Introduction

La multiplication des collectes de données personnelles, que ce soit lors de transactions commerciales, de déplacements, d'interactions avec les institutions ou de communications, est accompagnée d'un recours tout aussi massif à des systèmes algorithmiques. En particulier, ces systèmes algorithmiques sont de plus en plus utilisés pour prendre des décisions impactant les individus, les groupes et la société dans son ensemble. Ainsi, ils prennent une place de plus en plus prépondérante dans de nombreux domaines comme outil d'assistance à la prise de décision ou comme agent décisionnel à part entière, que ce soit pour évaluer les curriculums vitae de candidat•es à Amazon [START_REF] Dastin | Amazon Scraps Secret AI Recruiting Tool That Showed Bias against Women[END_REF], pour apparier étudiant•es et établissements avec Parcoursup [START_REF] Frouillou | Le Rôle Des Instruments Dans La Sélection Des Bacheliers Dans l'enseignement Supérieur. La Nouvelle Gouvernance Des Affectations Par Les Algorithmes[END_REF], pour évaluer le risque de récidive des accusé•es comme le fait le logiciel COMPAS [START_REF] Brennan | Correctional Offender Management Profiles for Alternative Sanctions (COMPAS)[END_REF] utilisé aux États-Unis ou pour l'attribution d'organes par Eurotransplant [START_REF] Schulte | Analysis of the Eurotransplant Kidney Allocation Algorithm: How Should We Balance Utility and Equity?[END_REF] en Europe. Cette ubiquité soulève d'importantes préoccupations éthiques et sociales, notamment concernant la protection de la vie privée et l'équité de ces approches, comme le montrent les nombreuses réglementations établies comme le RGPD [START_REF]Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the Protection of Natural Persons with Regard to the Processing of Personal Data and on the Free Movement of Such Data, and Repealing Directive 95/46/EC (General Data Protection Regulation)[END_REF] ou en préparation comme l'AI Act [START_REF]Proposal for a Regulation of the European Parliament and of the Council on Laying down Harmonised Rules on Artificial Intelligence (Artificial Intelligence Act) and Amending Certain Union Legislative Acts[END_REF], ainsi que les multiples ouvrages abordant ces questions [START_REF] Zhou | A Survey on Ethical Principles of AI and Implementations[END_REF][START_REF] Birhane | The Forgotten Margins of AI Ethics[END_REF].

La particularité sociale de ces enjeux, à la fois par leur contexte d'application et par la nature des questions, rend leur étude sous l'angle technique parcellaire. Si des approches comme l'évaluation des risques permettent de quantifier dans une certaine mesure l'impact tant pour la vie privée que la discrimination, celle-ci ne peut se faire qu'en connaissance des dangers. Cependant, les définitions de ces risques évoluent à travers le temps, les cultures et les positions politiques. Les étudier nécessite de pouvoir modéliser des systèmes sociotechniques complexes et cette modélisation peut être elle-même porteuse de biais de par le choix de consider ou non certains éléments [START_REF] Aven | Selective critique of risk assessments with recommendations for improving methodology and practise[END_REF]. Par ailleurs, les dimensions d'interprétation et de contexte de concepts comme la "vie privée" et la "non discrimination" ne peuvent pas être retranscrites facilement de façon formelle, laissant place à des redéfinitions algorithmiques comme discuté par des travaux récents [START_REF] Ochigame | The Invention of "Ethical AI[END_REF][START_REF] Hoffmann | Where Fairness Fails: Data, Algorithms, and the Limits of Antidiscrimination Discourse[END_REF][START_REF] Kulynych | POTs: protective optimization technologies[END_REF]. Des conférences comme ACM FAccT [START_REF]ACM Conference on Fairness, Accountability, and Transparency[END_REF] tentent de répondre à cet enjeu en faisant se rejoindre les expertises et outils de plusieurs communautés, avec de nombreux articles mélant des auteur•ices issues des domaines de l'informatique, de la linguistique, du droit ou encore de la sociologie. Cette première réponse nous semble cependant souffrir de sa limitation au champ technique, amenant certes la dimension sociale en perspective, mais se limitant à des réponses techniques. On peut prendre par exemple le cas du cadre analytique de l'intersectionnalité [START_REF] Kimberlé | Demarginalizing the Intersection of Race and Sex: A Black Feminist Critique of Antidiscrimination Doctrine, Feminist Theory and Antiracist Politics[END_REF], identifié par l'universitaire afroféministe américaine Kimberlé Williams Crenshaw, qui permet de considérer les différentes formes de domination, d'oppression et de discrimination non pas comme cloisonnées, mais comme un système interconnecté, avec l'idée que celles-ci sont "plus grandes que la somme de leurs parties" [START_REF] Preddie | More than the Sum of Its Parts: Intersections of Sexual Orientation and Race as They Influence Perceptions of Group Similarity and Stereotype Content[END_REF]. Si cette notion est décrite et étudiée en longueur par de multiples ouvrages en sciences humaines et sociales, elle est formalisée dans plusieurs articles techniques [Fou+20 ; Mor+19 ; Kan+22 ; ML22] comme la simple considération d'une multiplicité d'attributs sensibles, par exemple l'intersection de genre et de race. Bien que ces propositions de formulations permettent un premier pas vers l'utilisation d'un concept abstrait dans un environnent technique, la réduction en inéquations, par exemple formulée sur le modèle de la confidentialité différentielle et appliquée à des sous-groupes de la population dans [START_REF] Foulds | An Intersectional Definition of Fairness[END_REF], ne peut conserver le contexte et les principes méthodologiques de l'approche. La complexité d'utilisation est illustrée par [START_REF] Bauer | Intersectionality in Quantitative Research: A Systematic Review of Its Emergence and Applications of Theory and Methods[END_REF] qui compare l'utilisation du cadre intersectionnel par plusieurs études quantitatives, ou par [START_REF] Varley | Untangling Synergistic Effects of Intersecting Social Identities with Partial Information Decomposition[END_REF] qui propose une première définition empirique. Les considérations sociales fortes rendent particulièrement crucial d'évaluer ce qui est perdu lors de la technisisation des concepts, au delà de l'exemple de l'intersectionnalité utilisé ici.

Cette thèse cherche à étudier la protection de la vie privée et l'équité sous l'angle de leur utilisation pratique, mais aussi de leurs implications sociétales. S'il est difficile de parler de technique sans être technicien•e et tout autant de parler d'éthique sans être éthicien•ne, on gardera ici une approche d'informaticien, en ayant pour objectif d'aller vers une ouverture sur les sciences sociales.

Protection de la vie privée

Pour résumer, la vie privée dans le domaine de l'informatique vise à y formaliser des desiderata sociaux en des objectifs numériques (formels), qui génèrent ensuite leurs propres questions techniques. On transforme ainsi des concepts parfois élusifs de respect de la vie privée et d'équité, issues d'attentes individuelles, de la morale et du droit, en des propriétés mathématiques des algorithmes et de leurs sorties [START_REF] Ochigame | The Long History of Algorithmic Fairness[END_REF].

Discrimination

Du côté de la non-discrimination, aussi appelé parfois équité, il s'agit d'un enjeu étudié plus récemment en informatique que le domaine de la vie privée, mais on retrouve des similarités dans l'approche suivie. Ainsi, les nombreuses façons de conceptualiser la discrimination et sa résolution se sont matérialisées dans de multiples redéfinitions algorithmiques de notions complexes. Les définitions les plus communes sont celles dites "de groupe", qui prennent majoritairement comme cadre les décisions issues d'algorithmes d'apprentissage supervisé, en cherchant à comparer la sortie d'un système (c'est-à-dire ses décisions) à des données de référence. Dans ce cadre, la discrimination de groupe se définit comme l'écart entre un certain indicateur, comme le taux de décisions positives ou le taux d'erreur, évalué entre deux groupes. De nombreuses définitions appartenant à cette famille existent [START_REF] Mehrabi | A Survey on Bias and Fairness in Machine Learning[END_REF] et sont généralement mutuellement exclusives [START_REF] Sorelle | On the (im)possibility of fairness[END_REF]. Pour résumer, la question est de démontrer que le système n'ajoute pas de discrimination, et non pas qu'il n'est pas discriminatoire à l'origine.

Une autre conception propose une vision "individuelle" [START_REF] Dwork | Fairness through awareness[END_REF] de non-discrimination, en exigeant que deux profils "proches" reçoivent des décisions similaires, étant donnée une notion de similarité entre les individus. La définition de non-discrimination repose alors sur cette mesure de similarité qui doit être elle-même non-discriminante, sans qu'elle puisse être définie à l'aide de ce modèle, posant naturellement un problème d'applicabilité. Il est intéressant de noter que Cynthia Dwork, première autrice de ce travail, est aussi à l'origine de la définition de confidentialité différentielle. On peut aussi citer les définitions utilisant un modèle causal, l'objectif étant alors de supprimer l'impact des variables confondantes dans la prise de décision, c'est-à-dire des variables qui influencent à la fois la décision et les variables d'entrée.

Cette multiplicité de définitions et de concepts est similaire dans une certaine mesure à ce qu'il est possible d'observer pour la vie privée, à la différence qu'aucune définition n'émerge comme préférable techniquement ou socialement comme le fait la confidentialité différentielle en protection de la vie privée. En particulier, il n'est pas désirable ou possible d'aboutir à une telle convergence, à la fois de par les incompatibilités formelles [START_REF] Sorelle | On the (im)possibility of fairness[END_REF] de concepts "désirables", ainsi que par la dépendance entre l'intérêt pour un concept et l'agent qui le formule [START_REF] Narayanan | Tutorial: 21 Fairness Definitions and Their Politics[END_REF], laissant de fait place à des biais dans le choix de la définition elle-même.

Nous faisons donc le constat de la difficulté à définir, choisir et utiliser des définitions de vie privée et de discrimination dans des contextes sociotechniques. Ne cherchant pas, ni ne pensant qu'il soit possible ou désirable de trouver, de réponse technique à ces enjeux, on s'attache dans cette thèse à étudier les limites et à présenter des solutions dont l'utilisation pourrait se faire en dehors du cercle technique, dans une perspective d'y lier les personnes sociales et les outils techniques.

Compromis entre vie privée et transparence

Dans une première contribution, nous examinons le conflit entre la vie privée et la transparence lors de la publication de procédures judiciaires. Plus précisément, nous proposons de considérer la transparence comme un desideratum humain d'une part et la vie privée comme une exigence pour le traitement automatisé d'autre part. Les décisions de justice sont aujourd'hui publiées massivement en ligne, à la fois dans un objectif de transparence et d'accessibilité de la justice, et pour répondre à la demande d'accès à des données nécessaire au développement d'outils automatisés pour le droit appelés "legal techs". Cet objectif de publication se heurte aux obligations de respect de la vie privée, pour lesquels nous identifions deux axes de tensions : individuel et massif.

La protection "individuelle" des jugements correspond à l'approche adoptée aujourd'hui de caviardage d'informations qui permette de façon évidente d'identifier les participant•e•s d'un jugement selon les règles qui encadrent la protection de chacun•e. Cette transformation peut être appliquée directement aux jugements en langage naturel et permet une grande utilité en conservant le sens à l'exception de quelques termes comme les noms qui peuvent être remplacés par exemple par des initiales. Cependant, si cette approche empirique rend potentiellement moins évidente une réidentification, elle n'apporte aucune garantie. Nous présentons plusieurs extraits de jugements qui permettent de mettre en valeur ces lacunes, ainsi que les difficultés techniques qui se présenteraient pour y répondre. Une technique comme la confidentialité différentielle permet de répondre à ce problème, mais n'est pas applicable directement au langage.

Concernant la protection "massive", nous partons du constat que le traitement massif de documents correspond au traitement automatique, et que celui-ci se fait en grande majorité après un traitement visant à structurer les données, par exemple en fréquence de mots. Cette séparation des pratiques nous permet de proposer une séparation des besoins en matière de vie privée, et de suggérer que des techniques de protection adaptées au traitement massif comme la confidentialité différentielle soient utilisées pour ce qui est de la publication massive, tout en conservant des méthodes limitées, mais répondant à l'exigence d'utilité pour ce qui est de l'accès individuel aux jugements. Nous proposons finalement une description d'architecture qui permettrait de mettre en place un tel système et identifions les composants particuliers qui seraient nécessaires à sa conception dans un cadre réel.

Évaluation empirique de la protection de la vie privée

Dans une deuxième contribution, nous proposons un cadre pour organiser des compétitions d'attaque de mécanismes de publication de données respectueux de la vie privée afin de mieux établir leurs comportements et limites dans la pratique. Les compétitions sont couramment utilisées dans la communauté de l'apprentissage automatique pour stimuler le développement de solutions pratiques et efficaces à des problèmes difficiles ou nouveaux, ainsi que dans la communauté de la sécurité à des fins de formation ou pour l'évaluation de sécurité des infrastructures existantes. Une telle tradition n'existe par contre pas encore dans le domaine de la protection de la vie privée. Toutefois, ces dernières années, plusieurs défis axés sur les mécanismes de publication respectueux de la vie privée ont été organisés.

Certains se sont principalement concentrés sur l'aspect de la défense, attendant des algorithmes respectant des garanties formelles telles que la confidentialité différentielle tout en atteignant des niveaux d'utilité élevés. D'autres considèrent de plus une phase d'attaques sur les ensembles de données assainis générés par les participants. Dans ce cas, la première phase est ainsi consacrée à la conception d'algorithmes d'assainissement, tandis que la seconde consiste à attaquer les données assainies à l'aide des algorithmes développés au cours de la première phase.

Si les phases d'assainissement ont permis de comprendre les compromis entre protection de la vie privée et utilité, et ont conduit à la mise en oeuvre des algorithmes d'assainissement de pointe ou même de nouveaux algorithmes, les résultats des phases d'attaque ont généralement été plus mitigés. Par exemple, les organisateurs du défi "Hide-and-Seek" ont rapporté des résultats d'attaques équivalents à des réponses aléatoires [START_REF] Jordon | Hide-and-Seek Privacy Challenge: Synthetic Data Generation vs. Patient Re-identification[END_REF]. Nous pensons que l'une des raisons de cette situation est que pour réussir, la phase d'attaque doit permettre aux participants de consacrer suffisamment de temps à la conception, à la mise en oeuvre et à l'évaluation de l'attaque.

Pour ce faire, nous proposons de mettre en place une série de défis en matière de publication de données respectueuse de la vie privée, spécifiquement adaptés aux attaques contre les algorithmes d'assainissement [START_REF] Allard | SNAKE Challenge: Sanitization Algorithms under Attack[END_REF]. En particulier, nous souhaitons concentrer l'énergie et l'expertise des participants contre un petit ensemble d'algorithmes d'assainissement de pointe soigneusement choisis en donnant suffisamment de temps aux participants pour concevoir et tester soigneusement leurs attaques. Finalement, nous souhaitons aussi permettre l'exploration d'un large éventail de modèles d'adversaires possibles et de connaissances de base. Cette proposition se veut complémentaire des compétitions qui se concentrent sur la partie assainissement, permettant ainsi d'aboutir à un pipeline complet de conception et d'évaluation défense-attaque.

Pour permettre un système complet et générique adaptable aux différents cadres d'attaques, nous définissons un ensemble d'abstractions sous la forme d'un graphe de règles de transformation des données d'entrée et soumissions des participant•es vers leur évaluation. La première édition du défi [START_REF] Allard | 1 st Edition of the Snake Challenge. Snake #1: SaNitization Algorithm under attacK ". collocated with the 13 ième[END_REF], qui sert d'implémentation de référence, se concentre sur l'inférence d'appartenance [Hu+22 ; SOT22] dans les données tabulaires générées synthétiquement tout en satisfaisant la confidentialité différentielle [START_REF] Dwork | The Algorithmic Foundations of Differential Privacy[END_REF].

Simuler l'impact de la discrimination à long terme

Dans une troisième contribution, nous nous concentrons sur les limites des définitions techniques d'équité et utilisons une simulation ancrée dans la réalité comme moyen d'observer leur impact à long terme sur l'ensemble d'un système. Les définitions de dis-crimination utilisées aujourd'hui proposent souvent une conception "instantanée" dans laquelle la sortie de l'algorithme est considérée sans prendre en compte son futur, à savoir le contexte dans lequel la prédiction pourra être utilisée pour prendre d'autres décisions. Notre premier pas vers la considération de l'impact d'une décision et de contraintes de non-discrimination sur les individus, les groupes et le système dans son ensemble se fait par une simulation. En effet, cet outil technique permet de répondre, bien que de façon imparfaite, à la nécessité de pouvoir observer le comportement de mécanismes de nondiscrimination sans pour autant nécessiter d'expérimentation immorale sur des individus ou impossible sur une société.

Plus précisément, nous adaptons un modèle proposé par Reardon et al. [START_REF] Reardon | What Levels of Racial Diversity Can Be Achieved with Socioeconomic-Based Affirmative Action? Evidence from a Simulation Model[END_REF] et issu du domaine de l'éducation dont les valeurs sont adaptées du système américain et dans lequel des étudiant•es de différents groupes sensibles, compétences et revenus postulent et sont admis ou non dans des écoles de qualité variable. Ce modèle prend en compte l'évolution des écoles d'une itération à l'autre en fonction de la qualité de ses étudiant•es. En revanche, le modèle ne considère pas l'impact des décisions sur les étudiant•es. Nous intégrons ce mécanisme pour pouvoir ainsi prendre en compte la dimension temporelle. La simulation finale comprend deux mécanismes de dépendances temporelles : la boucle de renforcement des systèmes de prise de décision et l'effet "boule de neige" pour les individus. La première est similaire à sa définition dans un contexte d'apprentissage par renforcement, à savoir que les décisions produites par un système ont un effet sur l'environnent, qui lui-même impacte l'état du futur système. L'effet boule de neige consiste à considérer une accumulation de décisions du point de vue des individus : une décision reçue est potentiellement utilisée ensuite pour prendre une autre décision, comme c'est le cas par exemple pour le passage d'un lycée à une université. À partir de cette simulation reproduite et modifiée, nous souhaitons observer le comportement de mécanismes simples de non-discrimination et comment leur effet et mesuré ou nous par différentes métriques.

Contributions

Afin de mettre en évidence les questions fondamentales soulevées par la technicisation des défis sociaux et comme tentative de ramener ces défis sociotechniques dans leur contexte social, cette thèse apporte donc les contributions suivantes :

• [ABG20a ; ABG20b] étudient les difficultés en matière de respect de la vie privée rencontrées lors de la publication des décisions de justice, illustrant l'inadéquation de méthodes génériques dans certains contextes ;

• [START_REF] Allard | SNAKE Challenge: Sanitization Algorithms under Attack[END_REF] fait la description et la démonstration d'un cadriciel visant à faciliter la mise en place de compétitions d'attaque de mécanismes de protection de la vie privée pour mieux en cerner les limites. [START_REF] Allard | 1 st Edition of the Snake Challenge. Snake #1: SaNitization Algorithm under attacK ". collocated with the 13 ième[END_REF] est une compétition internationale d'attaque de mécanismes de génération de données synthétiques tabulaires respectueux de la vie privée, organisée sur cette base à l'été 2023 ;

• [START_REF] Allard | ~Re] Simulating Socioeconomic-Based Affirmative Action[END_REF] décrit notre effort de reproduction d'une simulation que nous adaptons par la suite pour étudier les impacts à long terme des mécanismes discriminatoires.

Une liste complète est présentée en page 11.

List of contributions

Authors are listed in alphabetical order, with the exception of [START_REF] Allard | La Confidentialité Différentielle, Garante de l'anonymat[END_REF] for which the authorship sequence was decided by the editors. I am the main author of [START_REF] Allard | Online publication of court records: circumventing the privacy-transparency trade-off[END_REF][START_REF] Allard | Publication of Court Records: Circumventing the Privacy-Transparency Trade-Off[END_REF][START_REF] Allard | ~Re] Simulating Socioeconomic-Based Affirmative Action[END_REF][START_REF] Allard | 1 st Edition of the Snake Challenge. Snake #1: SaNitization Algorithm under attacK ". collocated with the 13 ième[END_REF][START_REF] Allard | SNAKE Challenge: Sanitization Algorithms under Attack[END_REF].

Peer-reviewed international journals

[ABG23a] Tristan Allard, Louis Béziaud, and Sébastien Gambs.

Organization of international challenges

[ABG23b] Tristan Allard, Louis Béziaud, and Sébastien Gambs. "1 st Edition of the Snake Challenge. Snake #1: SaNitization Algorithm under attacK ". collocated with the 13 ième Atelier sur la Protection de la Vie Privée (APVP'23). 2023.

Introduction

Algorithms in society

The proliferation of personal data collection, whether for commercial transactions, travel, interactions with institutions or communications, is accompanied by an equally massive use of algorithmic systems. In particular, these algorithmic systems are increasingly used to perform decisions impacting individuals, groups, and the society as a whole. As a result, they have taken an increasingly prominent role in many fields as a decision-making tool, or as a decision-making "agent" in their own right. For example, they are used to assess the curricula vitae of candidates for Amazon [START_REF] Dastin | Amazon Scraps Secret AI Recruiting Tool That Showed Bias against Women[END_REF], to match students and institutions with Parcoursup [START_REF] Frouillou | Le Rôle Des Instruments Dans La Sélection Des Bacheliers Dans l'enseignement Supérieur. La Nouvelle Gouvernance Des Affectations Par Les Algorithmes[END_REF], to assess the risk of recidivism of accused persons as done by the COMPAS [START_REF] Brennan | Correctional Offender Management Profiles for Alternative Sanctions (COMPAS)[END_REF] software used in the USA, or for organ allocation by Eurotransplant [START_REF] Schulte | Analysis of the Eurotransplant Kidney Allocation Algorithm: How Should We Balance Utility and Equity?[END_REF] in Europe. This ubiquity raises important ethical and social concerns, especially regarding the privacy and fairness of these approaches, as shown by the many regulations established such as the RGPD [START_REF]Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the Protection of Natural Persons with Regard to the Processing of Personal Data and on the Free Movement of Such Data, and Repealing Directive 95/46/EC (General Data Protection Regulation)[END_REF] or in preparation such as the AI Act [START_REF]Proposal for a Regulation of the European Parliament and of the Council on Laying down Harmonised Rules on Artificial Intelligence (Artificial Intelligence Act) and Amending Certain Union Legislative Acts[END_REF], as well as the multiple works addressing these issues [START_REF] Zhou | A Survey on Ethical Principles of AI and Implementations[END_REF][START_REF] Birhane | The Forgotten Margins of AI Ethics[END_REF].

Sociotechnical issues

The social particularity of these issues, both in terms of their context of application and the nature of the questions, tends to make them difficult to study from a technical angle. While approaches such as risk assessment make it possible to quantify to some extent the impact on both privacy and discrimination, this can only be done in full awareness of the dangers. However, definitions of these risks evolve over time, across cultures, and political positions. Their study requires the ability to model complex sociotechnical systems, and this modeling can itself be biased by the choice of whether or not to consider certain elements [START_REF] Aven | Selective critique of risk assessments with recommendations for improving methodology and practise[END_REF]. Furthermore, the interpretive and contextual dimensions of concepts such as "privacy" and "fairness" cannot be easily transcribed formally, leaving room for algorithmic redefinitions as discussed by recent works [Kul+20; Och19; Hof19], which is to be understood as an unfaithful translation of the concept of discrimination that was originally presented. Conferences such as ACM FAccT [START_REF]ACM Conference on Fairness, Accountability, and Transparency[END_REF] attempt to meet this challenge by bringing together the expertise and approaches of several communities, with many papers mixing authors from the fields of computer science, linguistics, law, ethics, and sociology. Nevertheless, we believe that this first response suffers from its limitation to the technical field, bringing the social dimension into perspective, but limiting itself to technical answers. Social considerations make it particularly crucial to assess what is lost when concepts are technicized, beyond the example of intersectionality used here. This thesis seeks to study privacy and fairness from the angle of their practical use, but also of their societal implications. Since it is difficult to talk about technology without being a technician, and just as difficult to talk about ethics without being an ethicist, the approach adopted here is that of a computer scientist, with the aim of opening up to the social sciences. As far as computer science is concerned, the main aim is to formalize social desiderata into (formal) numerical objectives, which then generate their own technical questions. This transforms elusive definitions of privacy and fairness, derived from individual expectations, morality and law, into mathematical properties of algorithms and their outputs [START_REF] Ochigame | The Long History of Algorithmic Fairness[END_REF]. This point is the underlying theme of this thesis, by which we want to highlight the importance of the social context when considering these technical issues, thus requiring sociotechnical answers.

Decision systems

Interactions between individuals are governed by decisions, with decisions originating from institutions or individuals, applied to one self or another, applied directly to group of individuals or indirectly through its members. In addition, decisions taken in the past impact future decisions [START_REF] Nash | Bourdieu on Education and Social and Cultural Reproduction[END_REF]. Institutions change over time with history, norms, or jurisprudence for the example of the judicial system. Individuals evolve by themselves [START_REF] Arad | Past decisions do affect future choices: An experimental demonstration[END_REF] and through interactions with both institutions or groups of individuals, which themselves are thought over time with concepts such as social reproduction through culture and education [BP64; BP70], or economic factors [START_REF] Tombazos | La Reproduction Capitaliste Chez Marx[END_REF].

Computers have naturally become a part of decision processes [START_REF] Rouet | Algorithmes et Décisions Publiques[END_REF]. This can occur indirectly, when computers process and create information used to take decisions (e.g., producing statistics for weather forecasts) or as decision support system producing recommendations that can be used to take the final decision (such as the COMPAS software [START_REF] Brennan | Correctional Offender Management Profiles for Alternative Sanctions (COMPAS)[END_REF] used by judges to assess recidivism risk), or directly, which corresponds to the situation in which an algorithm is used to take the full decision which is then applied as-is. Note that the algorithm is rarely applying decisions that impact individuals, but rather it produces predictions, which are then used to make decisions by a human or by an automatic system. However, the frontier between direct and indirect decision-making is blurry, if for example judges blindly follow the COMPAS assessment, without ever refusing its prediction [START_REF] Simmons | Big Data, Machine Judges, and the Legitimacy of the Criminal Justice System[END_REF].

The first algorithmic decision systems were called expert systems. They can be defined as a translation of the set of rules followed by experts of some domain to produce a decision. A typical representation would be a set of rules evaluated by an inference engine, using a knowledge base constructed by domain experts. For example, JUDGE [START_REF] Tom | Judge: A Case-Based Reasoning System[END_REF] is a rule-based legal expert system introduced in 1986 that deals with sentencing in the criminal legal domain for offenses relating to murder, assault and manslaughter. SHYS-TER [START_REF] Tyree | James Popple, A Pragmatic Legal Expert System. Applied Legal Philosophy Series[END_REF] is another such tool introduced in 1996, highlighting by itself the issue of such tool: (algorithmic) pragmatism as a way to be unbiased. Since then, machine learning has progressively replaced expert systems, with the main difference being the reliance on data to extract a set of decision patterns, rather than requiring the process of experts to be encoded. Thus, the aim is to use previous existing decisions directly to infer the rules, skipping the knowledge base. The quality of the production will be impacted both by the choice of learning algorithm and parameters as well as by the quality of training dataset.

In decision-making, high-stakes decisions are described by the existence of a possible large loss and a high cost of reversing the decision-when even possible [START_REF] Rudin | Please Stop Explaining Black Box Models for High Stakes Decisions[END_REF]. For instance, high-stakes decisions can be found in the context of justice-such as a jail sentencing which has strong repercussions on one's life-in healthcare-such as ranking to access an organ-in education-such as enrollment into a given university-in workplacesuch as access to a promotion-and so on. High-stakes decisions are commonly regulated in some way by social expectations or laws, such as the GDPR [START_REF]Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the Protection of Natural Persons with Regard to the Processing of Personal Data and on the Free Movement of Such Data, and Repealing Directive 95/46/EC (General Data Protection Regulation)[END_REF]. However, highstakes decisions are only qualified as such when taken in the context of their application, and is not a qualifier of the process by which the decision is taken. Thus, issues regarding decisions require considering the decision system as a whole, which in the case of privacy and fairness issues often implies individuals, cohorts, and institutions.

The main focus of this thesis is therefore to look at both privacy and fairness issues in the context of algorithmic high-stakes decisions systems under the lens of both technical considerations and social obligations. A parallel is then drawn between this unification of concepts and the current state of fairness in computer science, with many models and definitions. These limitations are already heavily discussed in the humanity communities, as illustrated by the following quote from [BP08]:

The system of inequalities is characterized not only by the interactions between all its constituent elements, but also by the retroactive nature of some of these interactions. This feedback can be positive, with the effect reinforcing its own cause, which will in turn reinforce it in a cumulative process; or, on the contrary, negative, leading in this case to the establishment of a certain equilibrium by self-regulation.

However, the computer science and particularly the machine learning community have not really researched thoroughly this topic.

Privacy

In privacy protection, the first responses were syntactic models, transforming the data to providing some sort of record indistinguishability [START_REF] Clifton | On Syntactic Anonymity and Differential Privacy[END_REF]. For example, k-anonymity [START_REF] Sweeney | k-Anonymity: A Model for Protecting Privacy[END_REF], introduced in 2002, hides individual profiles by grouping them into clusters of at least one person. This model of privacy has been successively refined to control utility-more precisely, information loss-and to respond to attacks, leading to l-diversity [START_REF] Machanavajjhala | L-diversity: Privacy beyond k-anonymity[END_REF] and then t-closeness [START_REF] Li | t-Closeness: Privacy Beyond k-Anonymity and l-Diversity[END_REF].

In parallel, differential privacy [START_REF] Dwork | Differential Privacy[END_REF] was proposed in 2006 with the objective of proposing a more robust definition of privacy. More precisely, a mechanism respects privacy-in the sense of differential privacy-if the presence or absence of an individual in the database provided as input to the mechanism impacts the calculation in an indiscernible way, obeying to an allocated budget, which is used to bound the privacy loss, that is, the amount of information that can leak In other words, given the result of a query, an attacker gains only limited information to infer the presence or absence of the individual.

If this definition is considered, in this work as in many others, as the reference, a great many variations have been proposed [START_REF] Desfontaines | SoK: Differential privacies[END_REF] in order to respond to different models of adversary, auxiliary knowledge or other data structures. To the issue of adopting a definition is added that of choosing a budget-and additional variables when considering variations-with attempts at academic recommendation [LC11; Hsu+14] and different practical uses in industry [START_REF] Tang | Privacy Loss in Apple's Implementation of Differential Privacy on MacOS 10[END_REF].

Fairness

Non-discrimination, or fairness, is a more recent topic in computer science than privacy, but there are similarities in the approach taken to address it. Indeed, the many ways of conceptualizing discrimination and its resolution have materialized in multiple algorithmic redefinitions of complex notions. The most common definitions are the so-called "group fairness", which mostly take as their framework the decisions made by supervised learning algorithms, seeking to compare the output of a system-i.e., its decisions-with ground truth data. Within this framework, group discrimination is defined as the difference between a certain statistical indicator, such as the rate of positive decisions or the error rate, evaluated between two groups. This notion of group is defined through so-called protected attributes-such as race, religion, gender, etc. A sensitive group is then a group, identified by one or multiple protected attributes, which is discriminated against. It is to be distinguished from the concepts of statistical majority and minority Many definitions belonging to this family exist [START_REF] Mehrabi | A Survey on Bias and Fairness in Machine Learning[END_REF], with some of them being mutually exclusive [START_REF] Sorelle | On the (im)possibility of fairness[END_REF].

Another proposition is "individual fairness" [START_REF] Dwork | Fairness through awareness[END_REF], which requires that two "similar" profiles receive similar decisions, given some notion of similarity between individuals. The definition of fairness then relies on this measure of similarity, which must itself be non-discriminating. It is interesting to note that Cynthia Dwork, the first author of this work, is also behind the definition of differential privacy. We can also cite definitions based on causal models [START_REF] Plecko | Causal Fairness Analysis[END_REF], in which the aim is to remove the impact of confounding variables in the decision-making process, which are variables influencing both the decision and the input variables.

This multiplicity of definitions and concepts is similar to some extent to what can be observed for privacy, with the difference that no definition emerges as technically or socially preferable as does differential privacy in privacy protection. In particular, it seems neither desirable nor possible to achieve such convergence, both because of the formal incompatibilities [START_REF] Sorelle | On the (im)possibility of fairness[END_REF] of "desirable" concepts, and because of the dependence between interest in a concept and the agent who formulates it [START_REF] Narayanan | Tutorial: 21 Fairness Definitions and Their Politics[END_REF], leaving room for bias in the choice of the definition itself.

As an example we can take the intersectional analytical framework [START_REF] Kimberlé | Demarginalizing the Intersection of Race and Sex: A Black Feminist Critique of Antidiscrimination Doctrine, Feminist Theory and Antiracist Politics[END_REF], as coined by the American Afrofeminist scholar Kimberlé Williams Crenshaw, which allows to consider the various forms of domination, oppression and discrimination not as compartmentalized, but as an interconnected system, with the idea that these are "greater than the sum of their parts" [START_REF] Preddie | More than the Sum of Its Parts: Intersections of Sexual Orientation and Race as They Influence Perceptions of Group Similarity and Stereotype Content[END_REF]. While this notion is described and studied at length in several works in the human science community, it is formalized in multiple technical articles [Mor+19; Fou+20; Kan+22; ML22] as the simple consideration of a multiplicity of sensible attributes, such as the intersection of gender and race. Although these proposed formulations provide a first step towards the use of a mainly abstract concept in a technical environment, the reduction to inequalities, for example formulated on the model of differential privacy and applied to subgroups of the population in [START_REF] Foulds | An Intersectional Definition of Fairness[END_REF], cannot retain the context and methodological principles of the approach. The complexity of using this framework is illustrated by [START_REF] Bauer | Intersectionality in Quantitative Research: A Systematic Review of Its Emergence and Applications of Theory and Methods[END_REF] who compares it across several quantitative studies, or by [START_REF] Varley | Untangling Synergistic Effects of Intersecting Social Identities with Partial Information Decomposition[END_REF], who proposes a first empirical definition.

Contributions

To investigate the fundamental issues raised by the technicization of social challenges, and as an attempt to bring these sociotechnical challenges back into their social context, this thesis tackles a broad set of questions in both privacy and fairness.

Privacy contextual issue

The use of privacy-preserving mechanisms can be hampered by the context in which data is used. Examples at the technical level are undefined context such as with natural language, which makes background knowledge hard to quantify, or unstructured data which requires alternative privacy definitions and mechanisms. Natural language is indeed full of implicit knowledge and cultural bias which are difficult to encode in language model, rendering the background knowledge difficult to quantify. Additionally, contextual requirements such as strong utility requirements or output constraints can pose challenges greater than the common privacy trade-off. A first problem we study is therefore how formal privacy models can be used under strong societal constraints on their behavior and output properties. In particular, we look at the privacy-transparency trade-off faced when publishing legal proceedings, and ask ourselves how to navigate the trade-off between privacy and utility when publishing legal decisions.

To answer this first question, we examine in a first contribution [ABG20b; ABG20a] the conflict between privacy and transparency in the publication of legal proceedings, we illustrate the inadequacy of generic methods and the difficulty of their application under specific constraints. Court decisions are now published massively online, both for the sake of transparency and accessibility of justice, and to meet the demand for legal technologies building on the wealth of legal data available. However, the sensitive nature of legal decisions also raises important privacy issues. Current practices solve the resulting trade-off between privacy and transparency by combining access control with text redaction. We show that current practices are insufficient for coping with massive access to legal data-restrictive access control policies is detrimental to openness and to utility while text redaction is unable to provide sound privacy protection-and advocate for a integrative approach that could benefit from the latest developments of the privacy-preserving data publishing domain. We propose to consider transparency as a human desideratum on the one hand, and privacy as a requirement for automated processing on the other. Discarding the one-size-fits-all approach allows to circumvent the tensions between privacy and transparency. To this end we propose a straw man multimodal architecture paving the way to a full-fledged privacy-preserving legal data publishing system.

Privacy in practice

Formal privacy models and mechanisms often depend on parameters which control their guarantees. However, selecting these parameters is challenging and often depending on contextual, cultural and individual considerations. Although following common practice, adopting economical frameworks [START_REF] Hsu | Differential Privacy: An Economic Method for Choosing Epsilon[END_REF], or using risks based approaches can be a first step to guide the selection of a model and its parameters, understanding the actual guarantees offered by models and mechanisms still depends on many factors. The second problem we study is to how to address this issue of evaluating privacy mechanisms in practice.

To this issue, In a second contribution [START_REF] Allard | SNAKE Challenge: Sanitization Algorithms under Attack[END_REF], we propose a framework for organizing attack competitions of privacy-preserving data publishing mechanisms in order to better establish their behaviors and limits in practice. Recently, several challenges focusing on privacy-friendly publishing mechanisms were organized, improving the understanding of trade-offs between privacy and utility, and leading to the implementation of state-of-the-art or even new sanitization algorithms. Although some challenges integrate an attack phase, their results have generally been more mixed, such as equivalent to random guesses in one case [START_REF] Jordon | Hide-and-Seek Privacy Challenge: Synthetic Data Generation vs. Patient Re-identification[END_REF]. We aim to focus participants' energy and expertise against a small set of carefully chosen state-of-the-art sanitization algorithms, giving participants sufficient time to carefully design and test their attacks. To this end, we propose a comprehensive and generic system adaptable to different attack frameworks, and we define a set of abstractions in the form of a graph of rules for transforming input data and participant submissions into their evaluation. Using this framework, we organize an international attack competition [ABG23b] on privacy-preserving tabular synthetic data generation mechanisms, collocated with the French 13 ième Atelier sur la Protection de la Vie Privée.

Observing (un)fairness

Most definitions of fairness focus on the algorithmic part of the process by evaluating the fairness of the algorithm rather than that of the decision. That is, current fairness metrics allow the producer of decisions (i.e., the maker of a model) to guarantee that the model do not add unfairness (with respect to some definition), not that it is fair. This better fits a legal requirement than an ethical/societal goal of fairness. Indeed, group fairness definitions compare the output of the algorithm (the decision) to the input (the training data) and assess whether the algorithm introduced bias, not whether the output is biased is the first hand. These issues result in the absence of an "absolute fairness" notion, due both to the conceptual goal of decisions producerswho only need their product, (i.e., the model), to be fair-and a technical limitation of existing propositions being mutually exclusive. Furthermore, this results in "snapshot" of fairness notions, which offer a restricted consideration of the impact on the system and its individuals. For instance, this notion focuses on the decision itself and ignores the way this decision can be used in the future by other algorithms in conjunction with other information. To address this shortcoming of a lot of works in the fairness literature, we ask ourselves how to consider the long-term effect of deploying fairness-aware or fairness interventions methods.

To this end, in a third contribution, we focus on the limits of technical definitions of fairness and use a reality-based simulation as a means of observing their long-term impact on a system as a whole. The definitions of fairness used today often propose a "snapshot" conception in which the algorithm's output is considered without taking into account its future impact, i.e., the context in which the prediction can be used to make other decisions. As a first step towards considering the impact of a decision and of fairness constraints on individuals, groups and the system as a whole, we propose to use a simulation grounded in reality. To do so, we start by presenting our effort towards reproducing an educational policy model [START_REF] Allard | ~Re] Simulating Socioeconomic-Based Affirmative Action[END_REF] which we use to evaluate the behavior of technical fairness in a context grounded in reality. This technical tool meets the need, albeit imperfectly, to observe the behavior of non-discrimination mechanisms without requiring immoral experimentation on individuals or impossible experimentation on a society. We adapt this mechanism to take into account two dependency mechanisms: the reinforcement loop for decision-making systems and the "snowball effect" for individuals, which consists in considering an accumulation of decisions from the point of view of individuals: a decision received is potentially then used to make another decision, as is the case, for example, for the move from a high school to a university. From this model, we observe the behavior of simple non-discrimination mechanisms and how their effect is measured or measured by different metrics.

Outline

We first introduce the background in Chapter 2. In Chapter 3 we present our first contribution, in which we study the tension between transparency and privacy when publishing court decisions. Chapter 4 describes and demonstrates the practicality of a framework designed to facilitate the organization of attack challenges. We present our third contribution in Chapter 5, in which we study the long-term dimension of fairness. We conclude in Chapter 6.

Background

A background on privacy and fairness-though these topics should obviously be on the foreground.

Privacy

Historically many models

One of the first popular technical definition of privacy is k-anonymity [START_REF] Sweeney | k-Anonymity: A Model for Protecting Privacy[END_REF], first conceptualized in 1986 [START_REF] Dalenius | Finding a Needle in a Haystack or Identifying Anonymous Census Records[END_REF]. It relies on the idea of hiding single individuals into groups, requiring that the information of each person cannot be distinguished from at least k -1 other individuals. This means that each combination of quasi-identifierssuch as the combination of gender, birthdates and postal codes [Swe00]-must be present at least k times. Generally, to achieve k-anonymity, a space partitioning algorithm is applied to the data to compute generalized values, which are used in place of the real values for each individual. Figure 2.1 is an example of such a decomposition computed by following the Mondrian algorithm [START_REF] Lefevre | Mondrian Multidimensional K-Anonymity[END_REF]. The representation of values depends on the context and type of each column. When a taxonomy is available, categorical values can be generalized into a more general topic, while numerical values are typically replaced with a range or average value.

Multiple limitations of the k-anonymity model have been described. Without going into details, we describe the particular "complementary release attack" [START_REF] Sweeney | k-Anonymity: A Model for Protecting Privacy[END_REF], which will be particularly relevant when discussing differential privacy in the next section. When releasing data adhering to k-anonymity, quasi-identifiers are selected with respect to other external available information. Subsequent related releases must therefore consider all the released attributes as quasi-identifier to prevent linking. Similarly, if quasiidentifiers are wrongly defined then the guarantee falls, such as if unknown previous releases exist. The same issue arises if an attacker has access to background knowledge, which could be used as new quasi-identifiers.

The model of l-diversity [START_REF] Machanavajjhala | L-diversity: Privacy beyond k-anonymity[END_REF] was proposed to refine the definition and specifically counter the background knowledge made of negative sentences (e.g., "Bob does not have cancer") attacks and other attacks benefiting of cases where sensitive attributes have little diversity. It builds upon k-anonymity and adds intra-group diversity, requiring that classes contain at least l "well-represented" sensitive attributes, in which "wellrepresented" can be defined in several ways such has simply existing, or having a minimal entropy.

A third refinement to the definition is t-closeness [START_REF] Li | t-Closeness: Privacy Beyond k-Anonymity and l-Diversity[END_REF], which considers the loss of utility incurred by k-anonymity and l-diversity, which takes into account the distribution of values for the attributes. This representation also helps with an issue of l-diversity, which does not consider semantic closeness when enforcing diversity. While we will not be using these privacy definitions in the rest of this thesis, it is worth presenting them to review and discuss the analogy we make between the development of the privacy research and the development the fairness one.

As a key takeaway, it is interesting to take into account the incremental aspect of the development of privacy notions, and it was motivated by attacks against previous contributions.

Differential privacy

The approaches presented above define privacy as a property of the (sanitized) dataset, which can lead to information leakage. On the other side, differential privacy is defined as a property of the publication mechanism. It was inspired [START_REF] Desfontaines | SoK: Differential privacies[END_REF] by the privacy desideratum of Dalenius that "anything about an individual that can be learned from the dataset can also be learned without access to the dataset" [START_REF] Dalenius | Towards a Methodology for Statistical Disclosure Control[END_REF].

Feminist perspective Since we consider the social dimension of privacy, it is interesting to note that women have been especially present in the field, relative to other domains in computer science. Although not backed up by studies to our knowledge, this relative over-representation might be an indicator of the specificity of social issues. This was noted by Edwige Cyffers and discussed in [START_REF] Benbouzid | Fairness in machine learning from the perspective of sociology of statistics: How machine learning is becoming scientific by turning its back on metrological realism[END_REF], giving the prominent examples of Helen Nissenbaum, Cynthia Dwork, and Latanya Sweeney. [The+21] also provides a feminist perspective of the field of privacy. The same observation can be made for fairness, with discriminated communities having a strong interest and being concerned by the issue, as highlighted by initiatives such as Queer in AI [START_REF] Diaz | Scaling Laws Do Not Scale[END_REF].

Today's gold standard Differential privacy is used by many organizations. It was adopted by the U.S. Census Bureau for the 2020 Census [START_REF] Abowd | The U.S. Census Bureau Adopts Differential Privacy[END_REF], by Google as a way to report usage statistics in its Chrome web browser [START_REF] Erlingsson | RAPPOR: Randomized Aggregatable Privacy-Preserving Ordinal Response[END_REF], by Apple to do analytics on emoji usage [START_REF] Tang | Privacy Loss in Apple's Implementation of Differential Privacy on MacOS 10[END_REF], by Microsoft for its telemetry [START_REF] Bolin Ding | Collecting Telemetry Data Privately[END_REF], or by the Google to anonymize statistics on the response to policies aimed at combating COVID-19 [START_REF] Aktay | Google COVID-19 Community Mobility Reports: Anonymization Process Description (version 1.0)[END_REF].

Differential privacy is defined as a property of a mechanism. In a nutshell, -differential privacy ensures that the presence or absence of the data of a single individual has a limited impact on the output of the computation, thus also bounding the inference that can be done by an adversary about a particular individual based on the observed output [START_REF] Allard | Publication of Court Records: Circumventing the Privacy-Transparency Trade-Off[END_REF].

Definition 1 ( -differential privacy [Dwo06]). A randomized mechanism M satisfies -differential privacy with > 0 if Pr[M (D 1 = O)] ≤ exp( ) • Pr[M (D 2 ) = O]
for any O ∈ Range(M ) and any tabular dataset D 1 and D 2 that differs in at most one row, in which each row corresponds to a distinct individual.

Differential privacy exhibits a set of composability properties that helps to analyze the impact on the overall privacy guarantees of using a differentially-private scheme.

Definition 2 (Sequential and parallel composability [START_REF] Dwork | The Algorithmic Foundations of Differential Privacy[END_REF]). Let M i be a set of mechanisms such that each provides i -differential privacy. First, the sequential composability property of differential privacy states that computing all mechanisms on the same dataset results in satisfying ( i i )-differential privacy. Second, the parallel composability property states that computing each mechanism on disjoint subsets provides (max i i )-differential privacy.

Definition 3 (Post-processing [START_REF] Dwork | The Algorithmic Foundations of Differential Privacy[END_REF]). The post-processing property of differentialprivacy states that it is always safe (i.e., at no risk of breaking the privacy guarantee) to perform arbitrary computations on the output of a differentially private mechanism. Formally, if M satisfies -differential privacy, then for any mechanism F , F • M satisfies -differential privacy.

Answering numerical queries Since differential privacy is a property of the mechanism rather than the data, its implementation depends on the actual query to be considered. A common approach to implement differential privacy is to use the Laplacian mechanism, which adds noise drawn from a carefully parameterized Laplace distribution to the true answer to a numerical query. We introduce this mechanism on count and sum queries as a way to illustrate the impact of the budget .

Definition 4 (Laplace mechanism [DR14]

). Given a numerical function f : X → Y ⊂ R, the function F : X → R defined as follows satisfies -differential privacy:

F (x) = f (x) + Laplace δ
in which δ is the sensitivity of f and Laplace(s) denotes sampling from the Laplace distribution with center 0 and scale s.

The sensitivity of a function is the maximal amount by which the output changes when a profile is removed or added to the original dataset. For example, adding an individual to a dataset will change the result of a count query by one, whereas the sensitivity of a sum query will depend on the domain of the values.

Randomized response

Another example of a mechanism is randomized response [START_REF] Stanley | Randomized Response: A Survey Technique for Eliminating Evasive Answer Bias[END_REF][START_REF] Bernard | The Unrelated Question Randomized Response Model: Theoretical Framework[END_REF]. It provides a simple example of using differential privacy, and-benefiting from its simplicity-also illustrates the need for privacy when considering social issues. First proposed by Stanley L. Warner in 1965-and subsequently redefined to increase its utility-it allows a single individual to locally anonymize its answer to a sensitive query. It was used to survey sensitive topics such as cheating to exams [START_REF] Scheers | Improved Estimation of Academic Cheating Behavior Using the Randomized Response Technique[END_REF], drug usage [START_REF] Michael | The Randomized Response Technique: A Test on Drug Use[END_REF], or abortions [START_REF] Abernathy | Estimates of induced abortion in urban North Carolina[END_REF], which all require respondents to be confident in the plausible deniability of the scheme. The way noise is applied locally, and therefore before the aggregation, results in a high loss of utility, which is one of the drawback when using local differential privacy [START_REF] Prasad | What Can We Learn Privately?[END_REF].

Many variations

Although differential privacy is often presented-including by this thesis-as the flagship privacy definition, many variations of the model have been proposed. For instance, [DP20] lists a hundred of such variants and extension of differential privacy, organized on seven dimensions. These variants allow data custodian to consider different risk models or contexts of application.

Privacy washing With the increase in privacy risks, privacy protection becomes a marketing factor. Similar to the concept of greenwashing, which is a form of advertising in which marketing is deceptively used to persuade that products, aims and policies are environmentally friendly, privacy washing is a recent phenomenon in which privacy (often along with security) awareness is made into a selling point without actual consideration of the issue. This is made easier by the technical knowledge required to evaluate an approach. Indeed, weak mechanisms can be used, or strong mechanisms misused. An example is the case of Apple discussed before, in which the "bold announcement" [START_REF] Tang | Privacy Loss in Apple's Implementation of Differential Privacy on MacOS 10[END_REF] of its deployment of differential privacy was criticized for its lack of transparency regarding the implementation and atypical parameters that lead to low privacy guarantees [START_REF] Tang | Privacy Loss in Apple's Implementation of Differential Privacy on MacOS 10[END_REF].

Privacy attacks

Privacy attacks of decision systems, and usually of machine learning models, are categorized into different types. For instance, [START_REF] Rigaki | A Survey of Privacy Attacks in Machine Learning[END_REF] proposes a taxonomy composed of reconstruction, property inference, model extraction, and membership inference attacks.

The aim of a reconstruction attack [FJR15; HZL19; Zha+20] (sometimes called attribute inference or model inversion) is to recreate part of the training data of a model. Some variations create class representatives rather than an actual reconstruction of the data.

Property inference attacks [START_REF] Ateniese | Hacking smart machines with smarter ones: How to extract meaningful data from machine learning classifiers[END_REF][START_REF] Song | Information Leakage in Embedding Models[END_REF] focus on the extraction of information that was not encoded in the dataset, that is, to infer a property on the whole dataset or on a subgroup of individuals. For instance, [START_REF] Rigaki | A Survey of Privacy Attacks in Machine Learning[END_REF] gives the example of a model that "performs gender classification and can be used to infer if people in the training dataset wear glasses or not", which can interestingly be linked to fairness issues.

A model extraction attack [START_REF] Tramèr | Stealing Machine Learning Models via Prediction APIs[END_REF][START_REF] Krishna | Thieves on Sesame Street! Model Extraction of BERT-based APIs[END_REF] considers black-box models-that is when the adversary has no knowledge of the model parameters, architecture, or training dataand attempts to extract parameters and potentially reconstruct a substitute model that behaves similarly to the model under attack. This attack can be used as a first step towards other attacks, such as membership inference attacks [NSH19] described hereafter.

A membership inference attack aims at determining whether a sample (i.e., an individual in our case) was used as part of the training data. As noted in [START_REF] Rigaki | A Survey of Privacy Attacks in Machine Learning[END_REF], this is a popular category of attacks, with more than 30 examples referenced in [START_REF] Hu | Membership Inference Attacks on Machine Learning: A Survey[END_REF] It was introduced in the machine learning setting by Shokri et al. [START_REF] Shokri | Membership Inference Attacks Against Machine Learning Models[END_REF]. The actual privacy risk of such an attack depends on the nature of the data at hand. For example, we can consider a model trained to predict a patient's medical procedure, which could then be attacked to infer whether a particular individual was hospitalized. It is interesting to note that the definition of membership inference is closely related to the definition of differential privacy, which is why it is often used as an empirical evaluation of the lower bounds of privacy guarantees in privacy auditing papers [START_REF] Nasr | Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning[END_REF][START_REF] Jagielski | Auditing Differentially Private Machine Learning: How Private is Private SGD?[END_REF].

We focus on this specific type of attack in the rest of this section, describe the initial attack as proposed by [START_REF] Shokri | Membership Inference Attacks Against Machine Learning Models[END_REF], and how its success can be evaluated.

Shadow models attack

The attack described by [START_REF] Shokri | Membership Inference Attacks Against Machine Learning Models[END_REF] and illustrated on Figure 2.2 is now a common design pattern for a lot of membership inference attacks [START_REF] Hu | Membership Inference Attacks on Machine Learning: A Survey[END_REF]. It is based on the intuition that a model will behave differently when operated on data that do not belong to the previously seen training data. This can be captured either in the model output or in the internal model representation, which the attack model then uses to distinguish members from non-members.

The actual attack model contains a collection of models, one for each output class of the model under attack, called "shadow models". Since we consider here binary decisions, only two models are required. To build the attack, we consider a number of datasets D shadow that are usually assumed to come from the same distribution as the target dataset. Each dataset D shadow is partitioned into sets D in and D out , and a model M shadow -typically with the same architecture as the target model-is trained on

Membership advantage

The evaluation of membership inference attack is commonly performed by computing the membership advantage which estimates the knowledge gained by the attacker from the release, as compared to a guess done only from prior knowledge [START_REF] Ye | Enhanced Membership Inference Attacks against Machine Learning Models[END_REF].

Definition 5 (Membership experiment Exp M (A, A, n, D) [START_REF] Yeom | Privacy Risk in Machine Learning: Analyzing the Connection to Overfitting[END_REF]). Let A be an adversary, A be a learning algorithm, n be a positive integer, and D be a distribution over data points (x, y). The membership experiment proceeds as follows:

1. Sample S ∼ D n , and let A S = A(S).

2. Choose b ← {0, 1} uniformly at random. Definition 6 (Membership advantage [START_REF] Yeom | Privacy Risk in Machine Learning: Analyzing the Connection to Overfitting[END_REF]). The membership advantage of A is defined as

Draw

z ∼ S if b = 0, or z ∼ D if b = 1 4. Exp M (A, A, n, D) is 1 if A(z, A S ,
Adv M (A, A, n, D) = 2 Pr[Exp M (A, A, n, D) = 1] -1, (2.1)
in which the probabilities are taken over the coin flips of A, the random choices of S and b, and the random data point z ∼ S or z ∼ D.

Equivalently, the right-hand side can be expressed as the difference between A's true and false positive rates

Adv M (A, A, n, D) = Pr[A = 0|b = 0] -Pr[A = 0|b = 1].
(2.2)

Fairness

The first step required to define fairness in a technical setting is to have a concept of discrimination. This can originate from legal definitions, cultural norms, ethical philosophy, and so on. Many such propositions exist, and are outside the expertise of this thesis. A second issue is to have a notion of group on which to evaluate the discrimination. These groups are then refereed as protected and defined by one or multiple protected attributes. Typical protected attributes are race, gender, religion, etc. and can again vary across societies. In the next section, we briefly present the main families of fairness definitions encountered in the technical literature.

Many concepts, many frameworks, many models

A first naive approach to defining non-discrimination is to require that the decision is made while ignoring all protected attributes. However, this idea of "fairness through unawareness" is ineffective due to the existence of redundant encodings [START_REF] Pedreschi | Discriminationaware data mining[END_REF], similar to the issue with quasi-identifiers in privacy. Many fairness definitions have been proposed [Meh+22; GB20], with two main families: individual and group-based.

Individual fairness [START_REF] Dwork | Fairness through awareness[END_REF] was proposed by Cynthia Dwork. It requires that similar individuals receive similar outcomes, for some similarity metric which itself needs to be fair. The key idea as defined by [START_REF] Gupta | Individual Fairness in Hindsight[END_REF] is to introduce a Lipschitz condition (i.e., a uniform continuity) on the decisions of a classifier, such that for any two individuals x, y that are at distance d(x, y), the corresponding distributions over decisions M (x) and M (y) are also statistically close within a distance of some multiple of d(x, y).

The group fairness family contains many metrics. At an abstract level, it requires some statistical indicator to be equal (or similar) across groups. These indicators are commonly expressed through a combination of variables from the confusion matrix when considering supervised learning. Common examples include equalized odds [START_REF] Hardt | Equality of Opportunity in Supervised Learning[END_REF], which requires subjects in the protected and unprotected groups to have equal true positive rate and equal false positive rate, and equal opportunity, in which subjects in the protected and unprotected groups must have equal false negative rate. Some definitions of fairness are formulated from privacy concepts, requiring that an attacker cannot infer the protected attribute from the output of a system [START_REF] Feldman | Certifying and Removing Disparate Impact[END_REF].

Finally, causality-based fairness notions are a more recent proposition [START_REF] Plecko | Causal Fairness Analysis[END_REF]. They differ from the statistical ones such as group fairness in that they consider additional knowledge about the structure of the world, in the form of a causal model [START_REF] Carey | The Causal Fairness Field Guide: Perspectives From Social and Formal Sciences[END_REF]. Simple causal models can be seen in Figure 2 In (a), Z is a collider since it is influenced by both X and Y. In (b), Z is a confounder since it impacts both the input variable X and the output variable Y. In (c), T is a mediator since it lies between the input variable X and the output variable Y in one possible path.

corresponding to different formal models of causality, as seen in [START_REF] Carey | The Causal Fairness Field Guide: Perspectives From Social and Formal Sciences[END_REF]. To give a brief overview of the family, we can consider that it views discrimination as the causal effect of the sensitive attribute (Z on Figure 2.3) on the output attribute (Y on Figure 2.3).

Then, multiple questions correspond to different causal fairness notions, such as total causal fairness [START_REF] Pearl | The seven tools of causal inference, with reflections on machine learning[END_REF] which measures the impact on the output of changing the value of the protected attribute, or counterfactual fairness [START_REF] Kusner | Counterfactual Fairness[END_REF] which attempts to evaluate how the output variable would have behaved in an "alternate" scenario in which the protected variable has no impact. It has to be noted that many papers do not fall in those categories and try to come up with their own definitions, such as [Fou+20; GB20], or mixing concepts from individual and group fairness.

Intervention Although we focus in this work on how fairness can be evaluated, we briefly review how these notions of fairness are achieved in machine learning. Fairnessaware machine learning is typically available at three levels In-processing algorithms encompass many techniques where the learning step is modified to account for fairness [ZLM18; Kea+19];

Post-processing algorithms, finally, adjust the probability of specific decisions being made [START_REF] Pleiss | On Fairness and Calibration[END_REF].

Long-term considerations Recent work has focused, as we do, on evaluating the longterm impact of fairness interventions. The most prominent of these attempts is MLfairness-gym [START_REF] Alexander | Fairness is not static: deeper understanding of long term fairness via simulation studies[END_REF], which was developed as a framework to understand long-term fairness via simulation. It implements scenarios proposed in previous works [Liu+19; HIV19], and leverages OpenAI Gym [START_REF] Brockman | OpenAI Gym[END_REF] framework. As noted by authors of the paper, the simulations available are "extremely simple" in both the behavior of agents and the dynamics between components. On the other hand, the focus on machine learning makes the system difficult to interpret when studying on the behavior of fairness notions rather than on particular machine learning models.

Practical limits

Although this thesis tackles both privacy and fairness, we did not look the intersection of the two. This occurs in several ways, which we briefly discuss hereafter even if they are out of our scope. Firstly, there is often a technical similarity between privacy and fairness. This is heavily apparent in the case of individual fairness, sharing authors and definitions. In fact, many researchers from the fairness community originate from the privacy one [START_REF] Benbouzid | Fairness in machine learning from the perspective of sociology of statistics: How machine learning is becoming scientific by turning its back on metrological realism[END_REF]. The question of the compatibility between fairness and privacy, be it as a trade-off or as equivalent objectives, is also the subject of many studies [START_REF] Cummings | On the Compatibility of Privacy and Fairness[END_REF].

In the vein of a trade-off, some studies have raised concerns regarding the unfair impact of using differential privacy, especially in the case of the CENSUS [Fio+22; Puj+20].

Snapshot discrimination

We use the term "snapshot fairness" to refer to notions of fairness that do not consider the long-term impact, but rather are limited to enforcing a set of fair decisions at some given time. This is particularly well illustrated when considering most group fairness metrics, which evaluate a given set of decisions against a previous set of labeled decisions. The need to consider long-term dynamics of data and mechanisms with respect to their fairness, and the difficulty of identifying guidance towards these approaches was highlighted by Chouldechova et al. [START_REF] Chouldechova | A snapshot of the frontiers of fairness in machine learning[END_REF].

Fairwashing In line with our previous remark considering privacy washing, the same issue obviously appear with fairness, as it would with any social desiderata. In [Aıv+21], Aıvodji et al. show that given an unfair black-box model, it is possible to derive an explanation-in the form of an interpretable model-which exhibits both high-fidelity with respect to the model under question and a fairer behavior. This issue highlights the risk of trusting fairness advertising, and the difficulty of further assessments.

Fair individual fairness

When looking at individual fairness, a particular difficultyamong others [Fle21; GK21]-is the reliance on a similarity measure between individuals and between treatments. The fairness rests on the fairness of this measure, which poses an obvious issue of applicability.

Inaccessible causal model

Turning to the causal model, as we discussed above, it requires some knowledge of the model, which can also be a source of bias, and is difficult to obtain when considering complex sociotechnical systems. [START_REF] Fawkes | Selection, Ignorability and Challenges With Causal Fairness[END_REF] argues that causal models used in fairness settings "entail a particularly strong causal assumption, normally only seen in a randomized controlled trial". This strong requirement is unlikely to hold and makes the applicability in real cases particularly difficult.

Formal incompatibility As a final technical issue, we go to the most obvious concern with technical fairness definitions: their incompatibility. Indeed, [START_REF] Sorelle | On the (im)possibility of fairness[END_REF] shows that, apart from unrealistic scenarios-such as rejecting everyone, most fairness metrics are incompatible between each others. This can be illustrated in the case of group fairness metrics by considering that most of these metrics are expressed from a limited set of variables available in the confusion matrix, with only unrealistic scenarios being able to satisfy more than 3 metrics. [START_REF] Chouldechova | Fair Prediction with Disparate Impact: A Study of Bias in Recidivism Prediction Instruments[END_REF] gives the example of predictive parity (the likelihood of positive outcome is the same across groups) and error rate balance (the false positive and false negative error rates are equal across groups), which can only be both satisfied if the prevalence is the same between groups. At a higher level, and as discussed in [START_REF] Sorelle | On the (im)possibility of fairness[END_REF][START_REF] Barocas | Fairness and Machine Learning[END_REF], this also corresponds to the deeper issue of different fairness concepts corresponding to different worldviews and assumptions. This is also reflected in [START_REF] Hoffmann | Where Fairness Fails: Data, Algorithms, and the Limits of Antidiscrimination Discourse[END_REF], which advocates that technical fairness fails to address the "very hierarchical logic that produces advantaged and disadvantaged subjects in the first place". In other words, these approaches consider an effect-the unfair model-rather than the cause-an unfair society.

Privacy impact As a final point and to link the two topics of privacy and fairness, recent work has looked at two similarities and tensions between the two.

[Fio+22] surveys the work at the intersection of differential privacy, and fairness, showing how the two can both align or contrast, with differential privacy being able to sometimes mitigate fairness issues, or to exacerbate disparities among groups of individuals. In [START_REF] Pujol | Fair decision making using privacy-protected data[END_REF] the issue is illustrated on the U.S. Census data where the use of differential privacy disproportionately impact some groups over others for, as an example, the assignment of voting rights benefits. [START_REF] Chang | On the Privacy Risks of Algorithmic Fairness[END_REF] uses membership inference attacks as a framework to quantify such disparate impact.

The privacy-transparency trade-off of legal decisions publication

This chapter is adapted from [ABG20b; ABG20a], and was presented as an invited talk at the webinar on the use of AI in the justice field organized by the European Commission in 2021. First in Section 3.1 we introduce the context under which legal decisions are published, and the associated utility, transparency and privacy constraints.

In Section 3.2 we state the problem by describing precisely the content of legal data and by explaining the open legal data desiderata. Afterwards, we present the privacy limitations of the current approach, redaction, in Section 3.3.2, before describing in Section 3.4 our proposal of architecture for the publication of legal data ensuring both privacy and utility.

Publication of court decisions

The opening of legal decisions to the public is one of the cornerstones of many modern democracies: it allows auditing and make accountable the legal system by ensuring that justice is rendered with respect to the laws in place. As stated in [START_REF] Bentham | The Works of Jeremy Bentham[END_REF], it can even be considered that "publicity is the very soul of justice". Additionally, in countries following the common law, the access to legal decisions is a necessity as the law in place emerged from the previous decisions of justice courts. Thus, it is not surprising that the transparency of justice is enshrined in many countries as a fundamental principle, such as the right to a public hearing provided by the Article 6 of the European Convention on Human Rights, the Section 135(1) of the Courts of Justice Act (Ontario) stating the general principle that "all court hearings shall be open to the public" or in Vancouver Sun (Re) "The open court principle has long been recognized as a cornerstone of the common law". The open data movement push for free access to law with for example the Declaration on Free Access to Law [START_REF]Declaration on Free Access to Law[END_REF]. Multiple open government initiatives also consider the need for an open justice, such as the "Loi pour une République numérique" in France, the Open Government Partnership, the Open Data Charter, the Canada's Action Plan on Open Government. This trend is studied in a report of the OCDE [START_REF]The Call for Innovative and Open Government: An Overview of Country Initiatives. Organisation for Economic Co-operation and Development[END_REF], in [START_REF] Mcdermott | Building Open Government[END_REF] for the USA or [START_REF] Mcclean | Not with a Bang but a Whimper: The Politics of Accountability and Open Data in the UK[END_REF] for the UK.

Combined with recent advances in machine learning and natural language processing, the (massive) opening of legal data allows for new practices and applications (called legal technologies). Nonetheless, not all legal decisions should directly be published as such due to the privacy risks that might be incurred by victims, witnesses, members of the jury and judges. Some privacy risks have been considered and mitigated by legal systems for a long time. For instance, the identities of the individuals involved in sensitive cases, such as cases with minors, are usually anonymized by default because they belong to a vulnerable subgroup of the population. In situations in which the risks of reprisal are high (e.g., terrorism or organized crimes cases), judges, lawyers and witnesses might also ask for their identities to be hidden [START_REF] Jacquin | Terrorisme : La Peur Des Magistrats[END_REF][START_REF] Fleuriot | Avec l'accès Gratuit à Toute La Jurisprudence, Des Magistrats Réclament l'anonymat[END_REF]. Finally, the identities of the members of a jury are also usually protected to guarantee that they will not be coerced but also to ensure that the strategy deployed by the lawyers is not tailored based on their background. Legal scholars are aware of the need for privacy when opening sensitive legal reports [Con+11; Jac02; BB16].

In the past, these privacy risks were limited due to the efforts that were required to access the decisions themselves. For instance, some countries require going directly to the court itself to be able to access the legal decisions. Even when the information is available online, the access to legal decisions is usually on a one-to-one basis through a public but restricted API rather than enabling a direct download of the whole legal corpus. Typical restriction mechanisms include CAPTCHAs (SOQUIJ), quotas (CanLII), registration requirement as well as policy agreement and limitation of access to research scholars (Caselaw). Furthermore, the fact that a legal decision is public does not mean that it can, legally, be copied and integrated in other systems or services without any restrictions.

A first approach to limit the privacy risks consists in redacting the legal decisions before publishing them. Redaction mostly follows predefined rules that list the information that must be removed or generalized and define how (e.g., by replacing the first and last names by initials, by a pseudonym) [START_REF] Opijnen | On-Line Publication of Court Decisions in the EU: Report of the Policy Group of the Project "Building on the European Case Law Identifier[END_REF]. Redaction is in general semi-manual (and sometimes fully manual) because automatic redaction is error-prone [START_REF] Marrero | Named Entity Recognition: Fallacies, challenges and opportunities[END_REF]. This makes it extremely costly, not scalable, and does not completely remove the risks of errors [START_REF] Opijnen | On-Line Publication of Court Decisions in the EU: Report of the Policy Group of the Project "Building on the European Case Law Identifier[END_REF]. For example, 3.9 million decisions are pronounced in France every year but only 180000 are recorded in government databases and less than 15000 are made accessible to the public [START_REF] Fouret | Open Justice[END_REF]. Moreover, even a perfect redaction would still offer weak privacy guarantees. A redacted text still contains a non-negligible amount of information, possibly identifying or sensitive, that may be extracted, e.g., from the background of the case or even from the natural language semantics.

Another approach is access control, such as non-publication (e.g., a case involving terrorism was held in secret in Britain [START_REF] Calamur | In a First for Britain, a Secret Trial for Terrorism Suspects[END_REF]), rate limit, or registration requirements. However, access control mechanisms are binary and do not protect the privacy of the texts for which the access is authorized. Furthermore, restricting massive accesses for blocking also restricts the development of legal technologies that require a massive access to legal data.

Problem statement

Legal data

Legal reports are defined as written documents produced by a court about a particular judgment, which is itself a written decision of a court regarding a particular case (oral judgments are transcribed). Although the content of a case report varies with respect to the court and the country, it can consist of elements such as [Uni09]:

1. the case name and case citation (identifier); 2. the date of judgment and the hearing dates; 3. the court and judges involved in the decision; 4. the appearances (parties and their representatives); 5. the statement of facts: identify-sometimes in great length-the relationship and status of the parties, the legally relevant facts (i.e., what happened), and the procedurally significant facts (e.g., cause of action, relief request, raised defenses);

6. the procedural history: describes-if applicable-the disposition of the case in the lower court(s), the damages awarded, the reason for appeal, etc.;

7. the issues: point of law in dispute;

8. the law of the case: elements of law that the court applies;

9. the concurring and/or dissenting opinions (of judges); 10. the orders: the decision itself.

We can broadly distinguish three different categories of judicial data depending: metadata, facts and reasoning. Metadata (elements 1, 2, 3 and 4) correspond to identifiers of the case and basic information (e.g., date, parties and judge) and is written mostly in a structured way. Facts (elements 5, 6 and 7) are information pertaining to the parties, disclosing their personal "story". Reasoning (elements 8, 9 and 10) is the logic of the case, which is not specific to the parties.

Desiderata for the opening of legal data

In this section we present the needs that arise with the publication of legal decisions. We start by presenting utility requirements for individuals in Section 3.2.2, and for algorithms in Section 3.2.2. Finally we discuss the need for privacy in Section 3.2.2.

Need for readability and accessibility

The access to legal decisions is required both for ethical (transparency) and practical reasons such as case law, which is the use of past legal decisions to support the decision for future cases. Thus, the judiciary system is built on the assumption that legal decisions are made public and accessible by default (open-court principle), so that (1) citizens are able to inspect decisions as a way to audit the legal system and (2) past decisions can be used to interpret laws, and as such must be known from legal practitioners and citizens. It follows that decisions must be made available in a form readable by humans (i.e., natural language). Natural language format can be opposed to machine-readable formats such as word-vectors representation or logical propositions, which we will discuss later. The need for openness, the current practice in terms of open court, and the associated risks are detailed in [START_REF] Conley | Sustaining Privacy and Open Justice in the Transition to Online Court Records: A Multidisciplinary Inquiry[END_REF][START_REF] Peter W Martin | Online Access to Court Records-from Documents to Data, Particulars to Patterns[END_REF]. They conclude that, although there are powerful voices in favor of open court, radical changes in access and dissemination require new privacy constraints, and a public debate on the effect of sharing and using information in records.

Accessibility is also an important issue. In the past, the access to decisions required attending public hearings or reading books called "reporters". Later, decisions have started to be shared on digital medium such as compact discs or DVDs for example before being accessible online more recently. For instance, in the USA, CourtListener [START_REF]Free Law Project[END_REF] shares 3.6M decisions and the Caselaw access project [START_REF]Caselaw Access Project[END_REF] 6.7M unique cases; the Canadian Legal Information Institute [Fed23] (CanLII) publishes 2.5M Canadian decisions. The aim of these services is to facilitate access to legal records to individuals-law professionals (judges, lawmakers and lawyers), journalists, or citizens. The online publication also enables the large-scale access and processing of records, in particular due to the standardized format.

Need for massive accesses (legal technologies)

The term legal technologies broadly encompasses all the technologies used in the context of justice. The website CodeX Techindex [Cod23], a project by the Stanford Center for Legal Informatics, references more than a thousand companies, and defines nine different categories: (1) Marketplace, (2) Document Automation, (3) Practice Management, (4) Legal Research, (5) Legal Education, (6) Online Dispute Resolution, (7) E-Discovery, (8) Analytics and (9) Compliance.

A subset of these categories-2, 4, 7, 8 and 9-requires some form of "understanding" of legal documents, usually performed through natural language processing (NLP) and machine learning (ML) approaches [START_REF] Praduroux | Legal Tech Start-Ups: State of the Art and Trends[END_REF][START_REF] Dale | Law and Word Order: NLP in Legal Tech[END_REF]. We focus here on these categories as they are based on the analysis of a large number of legal data. One of the main challenges we have faced is that usually companies provide very few technical details about their actual processing and usage of legal documents.

The automatic processing and analysis of legal records have multiple applications, such as computing similarity between cases [TKA17; MS18; Man+17], predicting legal outcomes [Ale+16; KBB17] (e.g., by weighing the strength of the defender arguments

Focus on text-based legal techs

Most of the technologies introduced in the previous section rely on the processing of large database of legal data. However, the unstructured nature of legal data is one of the main challenges of the application of artificial intelligence in law [START_REF] Alarie | How Artificial Intelligence Will Affect the Practice of Law[END_REF]. Consequently, the analysis of a legal text corpus first requires to apply some pre-processing to add structure to the text. Figure 3.1 represents an abstract processing pipeline for court files, extracted mostly from academic papers1 , and inferred from the current practice of text analysis and descriptions of associated technologies.

In the following, we assume that any application involving the use of machine learning (as highlighted by most legal tech companies) is applied to court records. The first NLP step transforms the unstructured data (i.e., natural language) into some structured representation (see below) by pre-processing it. Afterwards, the second ML step corresponds to the actual application, which is the training (i.e., processing) of the ML algorithm, whose output is represented by the "internal representation" block. The term instance represents the output of the model given some query (e.g., applicable laws given a set of keywords representing infractions).

The pre-processing can be diverse and depends on the task (e.g., extracting a citation graph between cases). However, most NLP-based applications usually rely on a text model. Many models are statistical-based ones, such as document-word-frequency matrix, in which the corpus is decomposed into a matrix in which each cell contains the number of times a particular word appears in a document. This model has multiple variations such as bag-of-words (BoW) [START_REF] Joachims | Text Categorization with Support Vector Machines: Learning with Many Relevant Features[END_REF], term frequency-inverse document frequency, or n-grams [START_REF] Zhang | Character-level Convolutional Networks for Text Classification[END_REF]. For example, a combination of those techniques are used in [START_REF] Aletras | Predicting judicial decisions of the European Court of Human Rights: a Natural Language Processing perspective[END_REF] to predict decisions from the European Court of Human Rights, and by [START_REF] Kim | Statute Law Information Retrieval and Entailment[END_REF] to identify law articles given a query or to answer to questions given a law article. More recent approaches follow a neural network architecture in which a model is trained on the corpus with the objective to predict a word given a context, which is called word embeddings [START_REF] Mikolov | Distributed Representations of Words and Phrases and their Compositionality[END_REF]. Multiple variations of this structure exist [Jou+17; Kim14; LQH16; Lai+15; Zhe+18]. This approach has been used for example in [START_REF] Raphael | Machine learning for explaining and ranking the most influential matters of law[END_REF] to rank and explain influential aspects of law, or by [START_REF] Mokanov | Facts2Law: using deep learning to provide a legal qualification to a set of facts[END_REF] to predict the most relevant sources of law for any given piece of text using "neural networks and deep learning algorithms".

Need for privacy

The massive opening of legal decisions for transparency and technological reasons must not hinder the fundamental rights such as the right to privacy as emphasized by current open justice laws. In particular in this setting, the privacy of a least three main actors must be protected: namely the individuals directly involved in decisions (i.e., the parties), the individuals cited by decisions (e.g., experts or witnesses), and the individuals administering the laws (i.e., magistrates).

However, the problem of publishing legal decisions in a privacy-preserving manner is a difficult one. For instance, authorship attacks [START_REF] Abbasi | Writeprints: A stylometric approach to identity-level identification and similarity detection in cyberspace[END_REF] may lead to the re-identification of magistrates behind written decisions, or the presence of quasi-identifiers2 within the text decisions may lead to the re-identification of the individuals involved in or cited. Famous real-life examples, such as the governor Weld's [START_REF] Sweeney | k-Anonymity: A Model for Protecting Privacy[END_REF] or Thelma Arnold's re-identification [START_REF] Arrington | AOL Proudly Releases Massive Amounts of Private Data[END_REF], both based on the exploitation of quasi-identifiers, are early demonstrations of the failure of naive privacy-preserving data publishing schemes. Thus despite the fact that legal decisions are written as unstructured text, structured information can be extracted from them, including the formal argument, the decision itself (e.g., "guilty" or "innocent"), as well as arbitrary information about the individuals involved (e.g., gender, age and social relationships).

Pseudonymization schemes simply consist in removing directly identifying data (e.g., social security number, first name and last name, address) and keeping unchanged the rest of the information (quasi-identifiers included). These schemes provide a very weak protection level, as acknowledged in privacy legislations (e.g., GDPR), which has led to the development of new approaches for sanitizing personal data in the last two decades (see for instance the surveys [Che+09; ML21]). In this paper, we focus on privacy-preserving data publishing schemes providing formal privacy guarantees that hold against several publications (as required by any real-life privacy-preserving data publishing system). These schemes are based on (1) a formal model stating the privacy guarantees the scheme as well as a privacy parameter for tuning the "privacy level" that must be achieved, and (2) a sanitization algorithm designed to achieve the chosen model.

Analysis of current practices

In the following section, we review the current practice for legal data anonymization and privacy regulations. We also make a connection with health data anonymization techniques on which most papers rely. To be concrete, we illustrate the privacy risks through examples of re-identification attacks. Finally, we argue that rule-based anonymization is not sufficient to provide a strong privacy protection and discuss the (formal) issues surrounding text anonymization.

Redaction in the wild

Redaction of legal data

The redaction process consists in removing or generalizing a set of predefined terms defined by law through a semi-manual process [START_REF] Opijnen | On-Line Publication of Court Decisions in the EU: Report of the Policy Group of the Project "Building on the European Case Law Identifier[END_REF]. Furthermore, access to legal documents or even public hearings can be restricted in well-defined cases. The common practice is to replace sensitive terms, as defined below, by initials, random letters, blanks or generalized terms (e.g., "Montréal" becomes "Québec"). The specific set of rules regarding protected terms and the associated replacement practice can differ between countries and courthouses [START_REF] Opijnen | On-Line Publication of Court Decisions in the EU: Report of the Policy Group of the Project "Building on the European Case Law Identifier[END_REF].

According to [START_REF] Plamondon | Anonymisation de Décisions de Justice[END_REF], the following information is to be systematically removed for any person (subject to a restriction on publication), as well as for each of his or her relatives (parents, children, teachers, neighbors, employers, colleagues, school, …):

1. names, 2. date and place of birth 3. contact details (number, street, municipality, postal code, telephone, fax, email, web page, IP address), 4. unique personal identifiers (social security number, health insurance number, medical file, passport, bank account, credit card, …),

personal possessions identifiers (license or serial number, cadastral designation, company name, …)

In some context, the following data is also removed if it can be used to identify one of the individuals aforementioned:

7. small communities or geographic locations, 8. the accused and co-accused if their identity is not already protected by law, 9. the intervenors (court experts, social workers, police officers, doctors, …)

unusual information (number of children if abnormally high, income if particularly high, exceptional occupation or function).

[Con+11] present numerous examples of legislation framing the publication of specific terms and putting restriction to the open-court principle. For instance, it is common by default to hide the identity of victims of sexual offenses or children in youth courts. The identity of jurors and witnesses is also kept secret to avoid coercion or parties tailoring their strategy. In addition, in the USA, the fear for national security or the possible prejudice to another trial can lead to a complete ban on reporting being issued.

E.B. Petitioner v. V.I. Respondent Judgment for Dissolution of Marriage

Quote 1: Droit de la famille-15334, 2015 QCCS 762 (CanLII), [START_REF]Droit de La Famille -15334[END_REF]

Paper versus digital

The main difference between paper and digital access is the "practical obscurity" of paper records on the one hand, and the easy accessibility of digital records, on the other. The awkwardness of accessing paper records stored in a public courthouse puts inherent limitations on the ability of individuals or groups to access those records. In contrast, digital records are easy to analyze, can be searched in "bulk" by combining various key factors (e.g., divorce and children) and can potentially be accessed from any computer. Thus, traditional distribution provides "practical obscurity" [START_REF]Open Courts, Electronic Access to Court Records, and Privacy: Discussion Paper[END_REF], in that it is inconvenient (i.e., time-consuming) to attend the courthouse or read case reports.

Anonymization of health data

The Health Insurance Portability and Accountability Act (HIPAA) in the USA defines the security and privacy requirements of health information for both health professionals and technologies involved in health data. The search for complying with HIPPA has led to an important body of work on the redaction of health records. In particular, automated redaction or generalization of the sensitive terms defined in HIPPA generally involves domain specific named-entity recognition and generalization of terms through medical ontologies. As a concrete anonymization tool, Scrub [START_REF] Sweeney | Replacing Personally-Identifying Information in Medical Records, the Scrub System[END_REF] uses template matching to detect sensitive terms, which are replaced with synthetic data of similar type (e.g., a name with a name, a disease with a similar disease). t-PAT [START_REF] Jiang | t-Plausibility: Semantic Preserving Text Sanitization[END_REF] replaces sensitive words or phrases-recognized by an ontology-with more general terms using an early privacy-preserving data publishing model, called k-anonymity [START_REF] Sweeney | k-Anonymity: A Model for Protecting Privacy[END_REF], to preserve the privacy of patients.

Limitations of current approaches

Our objective in this section is to provide examples of potential attacks in order to illustrate the technical difficulties of raw text anonymization. Quotes 1, 2, 3, 4, and 5 are translated excerpts from French and Canadian opinions.

A common redaction practice is to replace names by initials as shown in Figure 1. The uniqueness of initials [START_REF] Finseth | The Uniqueness of Unique Identifiers[END_REF] is increased by combining multiple parties, especially if the relationship between the parties is known (e.g., in a divorce case).

A combination of attributes, which can be extracted using dedicated named-entity recognition, is presented in Quote 2: names of parties, parties are divorced, date of marriage, parties have a daughter, birthdate of the daughter, date of divorce, reside in Ontario near a Dr. James. This combination could be used as a quasi-identifier by a re-identification attack. Quote 3 is anonymized according to the CNIL recommendations of 2006, which requires the last name of individuals to be replaced by its initial. However, widely available background knowledge on the "Real Madrid Club de Futbol" combined with the (reallife) pseudonyms of the "players" trivially leaks their identity.

The de-anonymization of Quote 4 relies on the text semantics instead of background knowledge. It requires the adversary (1) to identify the link (X) between "M. […] Abdel X" and "the use of the name 'X' to designate a drink", and (2) to infer that the drink is called "sango", thus leading to the conclusion that X = "sango". While this attack may not be easy to automatize due to the hardness of detecting the semantics inference, it is, however, trivial to perform for a human (e.g., by crowdsourcing it).

Similar to Quote 2, Quote 5 could be attacked through a combination of attributes and relationship (e.g., extracted with Snorkel [START_REF] Ratner | Snorkel: rapid training data creation with weak supervision[END_REF]). This opinion from the Youth court involves children and, as such, follows the strictest anonymization rules of the SOQUIJ: only the year's birthdate of children is given and names are replaced by random letters. However, an adversary can extract an extensive relationship graph (see Figure 3.2), which could be matched over a relationship database (e.g., Facebook). In this case, a quasi-identifier could be the relationship graph (or parts of it).

A study [START_REF] Anandan | Significance of Term Relationships on Anonymization[END_REF] has shown that generalization-based sanitization is vulnerable to correlation attacks in which the generalized terms can be correlated with other terms, seemingly non-identifying, in order to jeopardize the effects of the generalization and

the American company Coca Cola Company markets drinks under the French trade mark "Coca Cola light sango", of which it is the proprietor; that M. […] Abdel X, relying on the infringement of his artist's name and surname, has brought an action for damages against the Coca Cola Company […]

On the ground that Abdel X maintains that, as an author and screenwriter, he is entitled to oppose the use of the name "X" to designate a drink marketed by the companies of the Coca Cola group.

Quote 4: Civ. 1 re , 10 April 2013, n Besides the content of legal documents, stylometry [START_REF] Tempestt | Surveying Stylometry Techniques and Applications[END_REF] can be used to identify authors (i.e., magistrates) by their writing style. Mitigation for this kind of attack exist [FDM19; WK18] but their output is only machine readable. Similarly, it is possible to exploit decision patterns to re-identify judges, as done for the Supreme Court of the United States [START_REF] Daniel | A general approach for predicting the behavior of the Supreme Court of the United States[END_REF].

Reasons for the failure of rule-based redaction

The review of current practices for tackling the privacy of legal documents in Section 3.3.1 has highlighted the widespread use of rule-based redaction, in which a set of patterns is defined as being sensitive and is either removed or replaced. However, as shown in Section 3.3.2 (1) privacy can be violated even in "simple" instances and (2) identifying information remains in most cases. In other words, rule-based redaction does not provide any sound privacy guarantee. We observe that it suffers from the following main difficulties.

1. Missing rule difficulty. Many combinations of quasi-identifiers can lead to reidentification and the richness of the output space offered by natural language (i.e., what can be expressed) can hardly be constrained to a set of rules. Furthermore, identifying the sensitive terms is challenging and domain-specific. This issue is the subject of multiple studies in the context of health data [Swe96; Ata+00; Dou+05].

Missing match difficulty.

The current state of the art about relationship extraction and named-entity recognition makes it hard to ensure that all terms that should be redacted will be detected, in particular because of the many possible ways to express the same idea (e.g., circumlocution).

Although these observations make the rule-based redaction difficult, it is important to note that attacks, e.g., re-identification, remain simpler than protection. Indeed, an adversary has to find a single attack vector (i.e., a missing rule or a missing pattern) whereas the redaction process needs to consider all the possibilities.

Towards a multimodal publication scheme

In Section 3.2, we have shown that the publication of legal documents serves two distinct and complementary purposes: (1) the traditional objective of transparency and case law, and (2) the modern objective of legal technologies of providing services to citizens and legal professionals. These two purposes obey to different utility and privacy requirements. More precisely, the traditional use case requires human-readable documents while legal techs need a machine-readable format for automated processing. Moreover, transparency and case law involve the access to opinions on an individual basis (i.e., one-at-a-time), similarly to attending a hearing in person. In contrast, legal technologies rely on the access to massive legal databases. This difference in cardinality (i.e., one versus many) entails different privacy risks. In particular, the massive processing of legal data requires the use of a formal privacy framework with composability properties (see Section 3.2.2). All this suggests the inadequacy of any one-size-fits-all approach.

Access modes

As a consequence, we propose that the organization in charge of the publication of legal decisions should consider two modes of publication: the precise access mode and the massive access mode.

Precise access mode

To fulfill the "traditional" use case, the precise access mode provides full access to legal decisions that are only redacted using the current practices. This access mode is designed for the transparency and case law usages, and is to be used typically by individuals (e.g., law professionals, journalists and citizens). Similar to the "traditional" paper-based publication scheme, in the precise access mode [START_REF] Hartzog | The Case for Online Obscurity[END_REF], a user has access to text documents, either in full or only extracts (partial access is useful for crowdsourcing tagging in order to build a dataset). While the current practice of redacting identifiers could be combined with more automated approaches such as [START_REF] Sánchez | Automatic General-Purpose Sanitization of Textual Documents[END_REF][START_REF] Hassan | Automatic Anonymization of Textual Documents: Detecting Sensitive Information via Word Embeddings[END_REF]. The aim of this mode is to provide strong utility first. It is similar to the websites currently publishing legal documents (e.g., Legifrance or CanLII), as it allows browsing, searching and reading documents.

To prevent malicious users from diverting the precise access mode for performing massive accesses, users must be authenticated, and their access must be restricted (e.g., rate limitation or proof of work [START_REF] Dwork | Pricing via Processing or Combatting Junk Mail[END_REF]). The main objective of the restricted access is to make it difficult to rebuild the full (massive) database. In addition, this mode provides privacy through "practical obscurity" similarly to the paper-based system.

Massive access mode

The massive access mode gives access only to pre-processed data resulting from privacypreserving versions of the standard NLP pipelines available on the server, i.e., aggregated and structured data extracted from or computed over large numbers of decisions, as required for the "modern" use case. It should be compatible with most legal tech applications that traditionally use a database of legal documents (see Section 3.2.2). Note that the perturbations due to privacy-preserving data publishing schemes have usually less impact (in terms of information loss) when applied late in the pipeline (see Figure 3.3), at the cost of a loss of generality of the output.

Users need to be able to tune the pre-processing applied. For the sake of simplicity, we assume that the user (i.e., legaltech developer) provides the parameters for a given NLP pipeline (see Fig. 3.3). These parameters can be for instance the maximum number of features or n-grams range to consider in the case of a BoW model or the window for word embeddings. But more complex implementations can be designed, e.g., allowing experiments by the users, fine-tuning for each dataset/task, as well as customization (e.g., for cleaning the data). This can be done (1) by generating structured synthetic testing data (e.g., a set of features extracted from legal data) in a privacy-preserving manner (e.g., PATE-GAN [START_REF] Jordon | PATE-GAN: Generating Synthetic Data with Differential Privacy Guarantees[END_REF] or MST [START_REF] Mckenna | Winning the NIST Contest: A scalable and general approach to differentially private synthetic data[END_REF]) and (2) by designing a full preprocessing pipeline that embeds privacy-preserving calls to the server (e.g., through a privacy-preserving computation framework such as Ektelo [START_REF] Zhang | EKTELO: A Framework for Defining Differentially-Private Computations[END_REF]).

The massive access mode must also authenticate users in order to monitor the overall privacy guarantees satisfied for each user based on his disclosures log and on the composability properties of the privacy-preserving data publishing schemes used.

As a result, the data is protected using authentication and strong privacy definitions as presented in Section 3.2.2. Examples of applications of differential privacy to NLP models include [START_REF] Fernandes | Generalised Differential Privacy for Text Document Processing[END_REF], which adds noise to word-frequency-matrix to achieve differential privacy, or [START_REF] Weggenmann | SynTF: Synthetic and Differentially Private Term Frequency Vectors for Privacy-Preserving Text Mining[END_REF], which samples the dictionary of the model using the differentiallyprivate exponential mechanism [START_REF] Dwork | The Algorithmic Foundations of Differential Privacy[END_REF]. The aim of these two approaches is to protect against authorship attribution.

[FDM19] uses a relaxation of differential privacy, d χ -privacy [START_REF] Mário | Metric-based local differential privacy for statistical applications[END_REF] which allows the authors to consider a distance between documents computed using word embeddings, rather than the row-based distance presented in Definition 1. The objective is to modify BoW representation of documents "similar in topics" remain "similar to each other" (w.r.t. the metrics defined on word embeddings), irrespective of authorship. In practice, this is achieved by drawing BoW where the probability of each word being associated to a document is distributed according to a Laplace probability density function.

The goal of [START_REF] Weggenmann | SynTF: Synthetic and Differentially Private Term Frequency Vectors for Privacy-Preserving Text Mining[END_REF] is to derive a differentially private synthetic feature vectors, keeping the theme of each document while preventing authorship attribution. Feature vectors map a set of words (the dictionary) to probabilities of the word appearing in each document. The main idea of the approach is to sample the dictionary from a reference dictionary (e.g., using synonyms from WordNet's synsets) using the differentially private exponential mechanism.

In practice, the massive access mode can be plugged into the existing platforms that store massive number of legal documents and already support the precise access mode, such as CourtListener or CanLII.

Finally, another potential need is the annotation of documents, which is the addition to terms, sentences, paragraphs or documents of metadata such as syntax information (e.g., verb or noun), semantic, pragmatic (e.g., presupposition and implicature). This step is crucial in NLP, and is usually done manually, for example through crowdsourcing. Crowdsourcing-specific approaches for privacy-preserving task processing [START_REF] Kajino | Instance-Privacy Preserving Crowdsourcing[END_REF] require to split the task (i.e., annotation of a set of documents) between non-colluding workers (e.g., at the sentence level) before aggregating the result. Such approach is compatible with our architecture assuming the aggregation is done locally on the platform.

System overview

We now outline an abstract architecture for a privacy-preserving data publishing system for legal decisions. Our objective is not to provide exhaustive implementation guidelines, but rather to identify the key components that such an architecture should possess.

Figure 3.3 depicts the proposed architecture. The precise and massive access modes are both protected by the Authentication module. The Authentication module can be implemented by usual strong authentication techniques (e.g., for preventing impersonation attacks). Authentication is necessary for enforcing the access control policy through the Access Control module and for maintaining for each user his Disclosure Log. The log contains all the successful access requests performed by a user. It is required for verifying that the overall privacy guarantees are not breached, e.g., the rate limitation is not exceeded for the precise access mode, or the composition of the privacypreserving data publishing schemes, formalized in the Composability Properties, does not exceed the disclosure allowed. Finally, the Privacy Parameters contain the overall privacy guarantees that must always hold, defined by the administrator (e.g., rate limit or higher bound on the differential privacy parameter). The user may additionally be allowed to tune the privacy parameters input by a privacy-preserving data publishing scheme (e.g., the fraction spent in the higher bound on the differential privacy parameter) provided it does not jeopardize the overall privacy guarantees.

Conclusion

In this chapter, we analyzed the needs for publishing legal data and the limitations of rule-based redaction (i.e., the current approach) for fulfilling them successfully. We proposed to discard any one-size-fits-all approach and outlined a straw man architecture balancing the utility and privacy requirements by distinguishing the traditional, one-to-one, use of legal data from the modern, massive, use of legal data by legal technologies. This step back from current practice allowed us to-in some way-avoid the tension between social and technical requirements.

Although our proposition could easily be implemented on current platforms, components of our framework pose new research questions. For example, reasoning with composability properties from different privacy models such as presented in Section 2.1.2 is a challenge. Another issue, discussed in Section 3.2.2 is the difficulty to evaluate empirical properties of approaches such as pseudonymization. Such a question could be answered by following the approach we describe in the following chapter.

Empirical privacy evaluation

This chapter is adapted from [START_REF] Allard | SNAKE Challenge: Sanitization Algorithms under Attack[END_REF], and from the first edition of the SNAKE challenge [START_REF] Allard | 1 st Edition of the Snake Challenge. Snake #1: SaNitization Algorithm under attacK ". collocated with the 13 ième[END_REF] organized during Summer 2023.

SNAKE framework

Competitions and challenges are commonly used both in the machine learning communityfor boosting the design and development of practical and efficient solutions to hard or new problems-and in the security community-for training purposes or for the evaluation of existing infrastructures. In contrast, in the privacy community, there is not a long tradition of holding such challenges. However, in recent years several competitions focusing on data sanitization algorithms (also called data anonymization algorithms or privacy-preserving data publishing algorithms) have been launched [Bou+20; Jor+20; Mur+23; Rid+21]. Some of them, like the 2018 Differential Privacy NIST Challenge [START_REF] Ridgeway | Challenge Design and Lessons Learned from the 2018 Differential Privacy Challenges[END_REF], have focused primarily on the defense aspect. More precisely, their main requirements were that the proposed algorithms have to (1) meet formal guarantees such as differential privacy, (2) achieve high utility levels on real-life cases, and (3) are efficient enough for being run on today's off-the-shelf computer systems. Others, like the Hide-and-Seek challenge [START_REF] Jordon | Hide-and-Seek Privacy Challenge: Synthetic Data Generation vs. Patient Re-identification[END_REF], the INSAnonym competition [START_REF] Boutet | RESSI 2020 -Rendez-Vous de La Recherche et de l'Enseignement de La Sécurité Des Systèmes d'Information[END_REF] or the PWSCUP [START_REF] Murakami | Designing a Location Trace Anonymization Contest[END_REF] additionally consider attacks on the sanitized datasets generated by the participants. More precisely, these competitions were generally composed of two phases, in which the first one is dedicated to the design of sanitization algorithms (usually focus on a particular type of data and use case) while the second one usually consists in attacking the data sanitized with the algorithms developed during the first phase.

While the sanitization phases of past challenges have been successful, in the sense that it has provided insights on the privacy/utility trade-offs, lead to implementations from state-of-the-art sanitization algorithms or even novel algorithms, the outcomes of the attack phases were usually more mitigated. For example, the organizers of the Hideand-Seek challenge have reported attack results equivalent to random guesses [START_REF] Jordon | Hide-and-Seek Privacy Challenge: Synthetic Data Generation vs. Patient Re-identification[END_REF]. We believe that one of the main reason for this is that in order to be successful, the attack phase must allow participants (1) to dedicate sufficient time to the design, implementation and testing of their attack strategies and (2) to focus on a few algorithms.

In this chapter, we propose a series of privacy challenges called SNAKE specifically tailored to attacks against sanitization algorithms. More precisely, the general structure of the SNAKE challenge provides the following salient features: it (1) enables to concentrate the energy and expertise of participants against a small set of carefully chosen into target files, similar to makefile directives. SNAKE uses Snakemake to prepare all files required to define the challenge and outputs a "bundle" which can be uploaded to CodaBench to build a complete challenge.

Key properties

The design of the SNAKE framework adheres to the following key goals. They form the minimal set of properties that are both necessary to the successful organization of attack challenges targeting sanitization algorithms and general enough to adapt smoothly to the advances of the field.

Genericity

The SNAKE workflow (see Figure 4.1) is high-level and generic enough to allow the design of most challenges in which participants attack the privacy properties of a sanitization algorithm. Indeed, any sanitization algorithm inputs prepared data and output sanitized data (e.g., a perturbed model), and any task consists in a challenge given to the participants (public part) and in the answer to the challenge (private part). In particular, the definition of a task as a pair of public and private data allows the framework to support a broad range of attacks scenarios (e.g., varying the background knowledge, the parameters, the kind of sanitization algorithm attacked).

Extensibility

The SNAKE framework provides the two following levels of extensibility. First, it offers a simplistic challenge overview, which allows for further specification. For example, considering multiple attacks or allowing online attack evaluation is possible without deviating from the framework. Second, steps of the workflow are organized as Snakemake rules which (1) allow for generic input, output, script, execution environment, and parameters; and (2) can be broken down into sub-rules if the need arises.

Usability By leveraging the Snakemake workflow management, the SNAKE framework does not require strong development skills to be used and can be run on commodity hardware. It can thus be used easily by participants during a competition for building a complete local competition environment allowing local executions of the sanitization algorithms and of the participants' attacks. On the organizer side, existing challenges can easily be framed within SNAKE. Note that using SNAKE does not generate any computational overhead compared to a SNAKE solution. Moreover, the SNAKE framework outputs a package compliant with the CodaBench platform, which allows for userfriendly participation (documentation, live ranking, automated scoring using a Python script).

Reproducibility SNAKE allows a challenge to be reproduced in full, thanks to the functionalities offered by Snakemake. For example, inputs, outputs, execution environments, parameters and scripts are tracked through hashing. Random seeds used by Create task are saved in the private part of the related task. They can be kept by the organizers and disclosed to participants after the competition. The genericity and reproducibility properties allow the framework to support different threat models over both participants and organizers. For example, our framework is designed to allow participants to verify locally that their scores, given by the organizer, are indeed correct. This can simply be done by the participant through a local evaluation of the submission performed after the distribution of the private part of the tasks. Preventing any tampering of the private part of tasks can be done, e.g., by storing their hash in the public bundle and by letting participants check that they match with the private parts distributed by the organizers after the competition.

SNAKE1: Membership inference attack against differentially-private synthetic data generators

SNAKE 1 is the first challenge based on the SNAKE framework. It focuses on membership inference attacks (e.g., [HHB19; Hye+22; Hu+21]) over differentially private synthetic data generation algorithms (e.g., [START_REF] Mckenna | Winning the NIST Contest: A scalable and general approach to differentially private synthetic data[END_REF]), is co-located with APVP 2023 and takes place all along the Summer 2023. Figure 4.2 describes the workflow of the challenge for a single task. First, the data used by the challenge is prepared-as described afterwards. Each execution of a sanitization algorithm takes as input a private dataset consisting of random samples from the base dataset. The attack used as input the synthetic data, a set of targets for which membership is to be guessed, the base data and the parameters of the sanitization algorithm. The participants then perform their attack locally 1 and submit their guesses to the CodaBench space of SNAKE 1 . The CodaBench platform scores the tasks by comparing the submission with the (private) ground-truth.

Attacks algorithm

Each team has to design an attack algorithm as follows:

Input The following information is provided to the attack algorithms:

• The synthetic dataset generated by an execution of the targeted sanitization algorithm over a private dataset;

• The targets to attack;

• The base dataset from which the private dataset is sampled;

• The parameters of the execution of the sanitization algorithm attacked.

Output The output of the attack algorithm is a real number in [0, 1] indicating the predicted probability of each target being within the private dataset or not.

We detail below the computation of the private datasets, the targets, the sanitization algorithms as well as the attack success measure.

Base dataset and private datasets

SNAKE 1 makes use of the publicly available EPI CPS Basic Monthly data provided by the Economic Policy Institute [START_REF][END_REF]. The CPS dataset is divided in years in which a yearly dataset contains more than 10 6 records and 125 columns. A record contains information about a single individual in a household. Our base dataset is built by using a pre-processed sample of columns and rows from the original dataset. Full details is available in the competition documentation. From this base dataset, we generate one private dataset for each parameterized sanitization algorithm attacked.

Targets and background knowledge

In SNAKE 1 , any household that contains at least 5 individuals might be a target, with the target consisting of the full set of records of the household. SNAKE 1 considers the following background knowledge about each target. The adversary knows (1) the exact records of the household targeted, and (2) the full base dataset2 . The adversary is also given the information about the sanitization algorithm targeted as well as the parameters used for the executions and has access to its implementation. However, the randomness generated internally during the execution of the algorithm is unknown to the adversary. can evaluate their submission on their side before submission.

Sanitization algorithms under attacks

The sanitization algorithms under attack during SNAKE 1 are differentially-private synthetic data generation algorithms. More precisely, we have selected a set of algorithms according to the following two criteria: technical soundness assessed by a rigorous peer-selection process (e.g., published at top-tier conferences or winner of a dedicated competition) and available open-source implementation. In particular, we have used the implementation available in the Reprosyn package [START_REF] Mole | [END_REF]. Except for parameters related to differential privacy, we use the default values set in their implementations. The PrivBayes algorithm [START_REF] Zhang | PrivBayes: Private Data Release via Bayesian Networks[END_REF] generates synthetic data by capturing the underlying distribution of the private data through a specific Bayesian network. The MST algorithm [START_REF] Mckenna | Winning the NIST Contest: A scalable and general approach to differentially private synthetic data[END_REF] is a generalization of the NIST-MST algorithm, which has won the 2018 NIST Differential Privacy Synthetic Data challenge. It generates synthetic data by perturbing the marginals that capture the data distribution through the Gaussian mechanism and by post-processing them through the Private-PGM algorithm [START_REF] Mckenna | Graphical-model based estimation and inference for differential privacy[END_REF]. The PATE-GAN algorithm [START_REF] Jordon | PATE-GAN: Generating Synthetic Data with Differential Privacy Guarantees[END_REF] is an extension of generative adversarial networks [START_REF] Goodfellow | Generative adversarial networks[END_REF] based on the private aggregation of teacher ensembles framework [START_REF] Papernot | Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data[END_REF].

Success measure

The success of a team is computed by first measuring the successes of its attack on each parameterized algorithm attacked (e.g., MST parameterized by = 1.0, δ = 10 -5 ) and second by aggregating the success measures in a single final score. The success of a given attack for a given parameterized algorithm is evaluated based on the membership advantage measure [START_REF] Yeom | Privacy Risk in Machine Learning: Analyzing the Connection to Overfitting[END_REF] (see 6). The number of targets must be at the same time large enough for obtaining a sufficiently stable estimation and small enough for being practical (e.g., r = 100).

Execution environment

The design, coding, and executions of attacks are performed locally by each team with its own resources. As a result, there is no restriction on the computing environment used to develop the attacks and the choice of the programming language and the available resources are unconstrained. Note however that we provide to participants the SNAKE 1 Python environment.

Preliminary results

The initial ending of the challenge was planned to September 1 st , 2023, which we then moved to October 1 st . At that time, the challenge had 21 participants, as individuals or teams. Due to technical issues with the platform we had to postpone the ending by another month, to November 1 st , which prevents us for presenting the detailed results of the challenge. As a first feedback, we note that the best scores are not attained for the highest values of , as could have been expected. Figure 4.3 shows the best score among all submissions for each task. The results for MST are as expected, with scores increasing with a larger budget. The best results for PATEGAN are close to each other. A first hypothesis might be that the learning error is higher than the impact of the noise added by differential privacy. Lastly, the values for PrivBayes are the most surprising: three of the tasks have similar scores, with the exception of = 100 which exhibits a particularly high membership advantage.

Future editions

The SNAKE framework is dedicated to facilitating the organization of challenges stresstesting sanitization algorithms. Overall, we hope that the SNAKE challenges can help gain understanding of the empirical privacy guarantees of sanitization algorithms and of their parameters and pave the way to a greater democratization of sanitization algorithms providing strong privacy guarantees. We envision SNAKE as a series of challenges, with future editions focusing on different datasets, privacy mechanisms and frameworks, attack models, etc.

Though first envisioned as a proof of concept, we created a toy competition named EKANS which could be the basis for a future edition. It is thought as the "reverse" of SNAKE 1 , in that its objective is to propose a sample method for targets that are the easiest to attack. EKANS asks participants to submit Python code in charge of sampling "good" targets for membership inference attacks, which is executed by the CodaBench platform before performing a basic attack on the sampled targets using the TAPAS package [START_REF] Houssiau | TAPAS: a Toolbox for Adversarial Privacy Auditing of Synthetic Data[END_REF].

Finally, as a future work, and considering the technical issues that we faced, we are considering building a platform customized to the SNAKE framework. This platform could benefit from the generic architecture and automated production of tasks to, in addition, deploy an API and database tailored to the challenge as described in for Snakemake.

Simulating long-term discrimination

In this chapter we focus on the fairness issues presented in Section 1, in which we highlighted the importance of considering the long-term impact of (un)fairness, and associated concepts that are often discarded in formal formulations of fairness. We start by advocating for the need of using a simulation for studying long-term evolution of (un)fairness mechanisms. Then, we present the main components of the model we adopt [START_REF] Reardon | What Levels of Racial Diversity Can Be Achieved with Socioeconomic-Based Affirmative Action? Evidence from a Simulation Model[END_REF], followed by a overview of the efforts need to undertook its reproduction.

The case for a simulation

As a first step towards the addition of long-term concepts in fairness, we need to be able to observe and identify effects of fairness effects in a long-term context. To achieve this goal, we adapted an agent-based simulation originating from the domain of educational policy, which, we hope, ground our work in reality-although highly simplified.

The use of a simulation is motivated by multiple factors. First and foremost, experimenting with fairness-and therefore unfairness-in high-stakes contexts cannot be done in an ethical fashion. This, the ability to draw observations without impacting individuals is obviously needed in this context. Second, the long-term effects in which we are interested requires long-term observation, which itself poses challenges. In particular, following individuals on the long-term is expansive and difficult [START_REF] Guerin | Lessons Learned from Participatory Discrimination Research: Long-term Observations and Local Interventions[END_REF]. Additionally, with observations being done over a long time period, more factors have to be taken in account since the number of decisions taken or received by an individual will increase. This is an effect of the curse of dimensionality, and directly linked to the argument towards the consideration of a snowball effect. Indeed, if decisions accumulate with time, the presence of unforeseen or uncontrolled decisions will make observations noisy.

Studying the long-term impact of a chain of decision, as an observation and without intervention, still requires access to a dataset following individuals during their life. Accuracy of measurements would require many individuals while meaningful observations would require collecting data over a long duration. Additionally, experimenting using machine learning models requires a large training set. To the best of our knowledge, no such dataset exists at the best of our knowledge. For example, the United States Census reports a high number of features on many individuals [START_REF] Abowd | The U.S. Census Bureau Adopts Differential Privacy[END_REF], but longitudinal linkage is limited [START_REF] Helgertz | A New Strategy for Linking US Historical Censuses: A Case Study for the IPUMS Multigenerational Longitudinal Panel[END_REF].

As a result, observing the impact of the decision is not possible without simulationor would require taking high-stakes decisions with all models to be experimented with in real life, which would be unethical and impractical. A simulation has the benefit of providing any number of decisions over any period of time. Additionally, the bias introduced by the simulation is known and controlled. On the other hand, the realistic aspect of the simulation is a concern. We opted for the model described in [START_REF] Reardon | What Levels of Racial Diversity Can Be Achieved with Socioeconomic-Based Affirmative Action? Evidence from a Simulation Model[END_REF] that studies the impact of affirmative action-which refers to the set of practices that seek to include particular groups that were discriminated against-in college admissions using a simulation. This is motivated by several factors. First, this model comes from the education policy community and is used to draw conclusions on a real-life process. In particular, the authors note that "although [their] model falls short of being completely realistic, it captures important, dynamic features of the application/admissions/enrollment processes that enable the investigation of the ways that affirmative action might affect enrollments." In addition, parameters are chosen to fit observation of the Education Longitudinal Study of 2002 [START_REF]Education Longitudinal Study of[END_REF]. Second, the model is described extensively. Although we faced several difficulties regarding the implementation-typical of the issue of reproduction in science, particularly in an interdisciplinary context-the paper makes extensive efforts to describe the model in details. Finally, we required a simulation that considers bias. For that matter, we searched for models in social sciences that cited "race". This potentially fits any simulation using sensitive attributes (such as gender or race), but the case study of [START_REF] Reardon | What Levels of Racial Diversity Can Be Achieved with Socioeconomic-Based Affirmative Action? Evidence from a Simulation Model[END_REF] considers the issue of discrimination itself, doing so through the correlated attributes of race and socioeconomic status.

College-student model

This section provides an overview of the model described by [START_REF] Reardon | What Levels of Racial Diversity Can Be Achieved with Socioeconomic-Based Affirmative Action? Evidence from a Simulation Model[END_REF]. A more technical description is available in Appendix A. We first introduce the two types of agents that interact in this simulation: colleges and students. Then we go over the three successive decisions that are taken at each iteration:

1. students apply to a subset of colleges depending on their own preferences and probability of being accepted;

2. colleges admit the top applying students;

3. students enroll into the best college they are admitted into.

Figure 5.1 is a complete view of the causal relationships between variables, which are described in Table 5.1. We will present relevant subsets of this figure along the section when discussing specific parts of the model. Specifically, Figures 5.2, 5.2, and 5.4 focus respectively on the application, admission and enrollment decisions.

Colleges and students

The model is composed of J = 40 colleges and N = 10 000 students.

Colleges are represented by their quality attribute, initially drawn from a normal distribution and later updated depending on the enrolled students, as represented by the orange arrows on Figure 5 Students are represented by three variables: race, resources and achievement. The race of each students is drawn from a non-uniform distribution over the categories Asian, Black, Hispanic, and White. Note that, following [START_REF] Reardon | What Levels of Racial Diversity Can Be Achieved with Socioeconomic-Based Affirmative Action? Evidence from a Simulation Model[END_REF], the numerical majority is White, while Asian is the least represented race, and that "minority" corresponds to Black and Hispanic students. Resources and achievement are sampled from racespecific bivariate normal distributions with non-zero correlation. The students' mixture distribution is:

Race ∼ Categorical(θ Race ) Res, Ach | Race = N ormal 2 (µ z , Σ z )
(5.1) The second decision phase, illustrated by Figure 5.3, consists in the admission of students by colleges. Colleges observe the achievement of applicants with some noise, similar to the observation of quality by students and serves the same purpose of modeling uncertainty and idiosyncratic preferences. Colleges rank applicants according to this variable and select the top subset of applicants, with the objective of enrolling 150 students. The number of admitted students is computed by each college from its previous enrollment yield, represented by green arrows on Figure 5.1. The final decision, illustrated by Figure 5.4, is straightforward. Students enroll in the college with the highest estimated utility to which they were admitted.

Admission

Race

Enrollment

Fairness interventions

Three fairness mechanisms are considered by [START_REF] Reardon | What Levels of Racial Diversity Can Be Achieved with Socioeconomic-Based Affirmative Action? Evidence from a Simulation Model[END_REF]: two affirmative action bonuses and one "targeted recruitment". The methods are introduced after the 15 th iteration by a subset of colleges.

Affirmative action can be based on race or socioeconomic status (SES). During the observation of students' achievement by colleges, an additional weight is added. In the case of race-based affirmative action, the added weight is constant added to minority students, that is Black and Hispanic students as introduced in Section 5.2.1. In the case of SES-based affirmative action, a weight is applied in accordance with each student's resources, such as 150 × Res. The actual weight varies between experiments, and is a source of difficulties regarding the reproduction which we discuss later in Section 5.4.

Targeted recruitment is the targeting of underrepresented racial minority students. It is represented as an increase in the observed utility of attending a college by students. Its implementation in reality would require colleges to take action towards students through targeted communication.

Reproduction

Method

Although the model is described in details, missing detailsinformation and inconsistencies hindered its reproduction [START_REF] Allard | ~Re] Simulating Socioeconomic-Based Affirmative Action[END_REF].

We have mostly followed the detailed description of the model given in [Rea+18, Appendix C]. We have also referred to [START_REF] Reardon | Agent-Based Simulation Models of the College Sorting Process[END_REF], which is often cited by the authors as an inspiration for this work. Code for this latter model is available in the (proprietary) Stata programming language [START_REF] Reardon | Agent-Based Simulation Models of the College Sorting Process[END_REF]. We have read this code for clarification, but we were not able to run it1 . We have contacted the first two authors of [START_REF] Reardon | What Levels of Racial Diversity Can Be Achieved with Socioeconomic-Based Affirmative Action? Evidence from a Simulation Model[END_REF] (Sean F. Reardon and Rachel Baker) in May 2021 but got no answer as of today.

For our implementation, we have used the Python programming language, with most of the processing done using NumPy [START_REF] Stéfan Van Der Walt | The NumPy Array: A Structure for Efficient Numerical Computation[END_REF]. We have used scikit-learn [START_REF] Kramer | Scikit-Learn[END_REF] for learning the logistic regressionWe have experimented with the Xarray library [START_REF] Hoyer | Xarray: ND Labeled Arrays and Datasets in Python[END_REF], which provides labeled multidimensional arrays. Our motivation was to deduplicate the join between students and colleges. However, we have achieved no storage or time improvement by doing so.

In addition to the replication effort, we focused on execution speed to be able to use the model as a synthetic data generator in other works. For this purpose, random data for all future iterations is drawn at initialization, and we took extensive care to use vectorized operations provided by the Numpy framework. We do not provide an extensive benchmark but still enjoy more than 5 iterations per seconds on off the shelf hardware.

Difficulties

In this section, we present the main issues encountered with respect to the description of the model in the original paper [START_REF] Reardon | What Levels of Racial Diversity Can Be Achieved with Socioeconomic-Based Affirmative Action? Evidence from a Simulation Model[END_REF]. Hereafter, inconsistent descriptions are quoted, but we refer the reader to the full paper for more context. Afterwards, we describe the interpretation that we have followed for the results presented in this paper.

Socioeconomic status (SES)-based affirmative action.

• "It is during the calculation of A * * cs that colleges with an affirmative action policy apply additional weight to a student's perceived admissions desirability in accordance with that policy. This additional weight is captured by the term

T c × [G × (Black s | Hispanic s ) + H × resources s ].
In this term, T c indicates whether a college has an affirmative action policy, […] H is the size of the weight given to students under SES-based affirmative action policies, which is applied linearly in accordance with the student's resources, resources s ." [Rea+18, Section 3]

• "SES-affirmative action (corresponding to an increase in admissions consideration of 0, 50, 100 and 150 achievement points for each decrease of one standard deviation in applicants' resources);" [Rea+18, Section 4].

Using H × resources s as in the first quote would advantage high-income students, which is the opposite of the objective. Moreover, the weight would not follow the description of the next two quotes, which involve the standard deviation. In our implementation, a weight H × max(-zscore(resources s ), 0) is added to the score of low-income students, with zscore being the standard score2 with respect to resources, and H the size of the weight (e.g., 0, 50, 100 and 150). We keep only negative values of the standard score (see Figure B.2 for a version without truncation) to impact only low-income students ("for each decrease of one standard deviation"). Note that while our interpretation is not totally satisfactory (see Section 5.5), the formula of the first quote (see Figure B.1) was equally unsuccessful.

Quality and own achievement reliability.

• "Quality reliability and own achievement reliability bounded by minimum values of 0.5 and maximum values of 0.9." [Rea+18, Table 1] • "the reliability of student perceptions of their own achievement, is a function of student resources and bounded between 0.5 and 0.9" [Rea+18, Appendix C]

• "the reliability of student perceptions of college quality, is a function of student resources and bounded between 0.5 and 0.7 " [Rea+18, Appendix C]

We assume that 0.7 is a typo and clip the values between 0.5 and 0.9. However, the impact is quite negligible as seed by comparing Figures 5.7 Apart from a significant speed increase, we observe no difference when fitting on the previous year only (and using the previously fitted model's parameters to initialize the new fit). However, our code still uses the past five years.

Weights of affirmative actions and targeted recruitment. The actual mapping between weights and their label changes between "moderate SES-based affirmative action and moderate race-based recruitment, which corresponds to a weight of 100 and 75, respectively." [Rea+18, Figure C3] and "strong SES-based affirmative action and strong race-based recruitment, which corresponds to a weight of 100 and 75, respectively." [Rea+18, Figure C3]

Experiments

This section contains original figures from [START_REF] Reardon | What Levels of Racial Diversity Can Be Achieved with Socioeconomic-Based Affirmative Action? Evidence from a Simulation Model[END_REF] along with our replications, followed by discussions of each result. To ease readability, we force the positioning of figures, which is therefore not optimal. We apologize to the reader for the resulting empty spaces. Furthermore, please note that to ease comparison we attempt-to the best of our ability-to reuse the style of the original figures, even when not ideal. We also apologize for the gray color palette. Original figures in the published version of the paper [START_REF] Reardon | What Levels of Racial Diversity Can Be Achieved with Socioeconomic-Based Affirmative Action? Evidence from a Simulation Model[END_REF] are colored. However, because of the high costs of the reproduction rights, we chose to use the black and white versions of the preprint.

Our objective was to replicate the overall tendencies of the model. We compare below our replicated results to the original results. The results without using positive discrimination policy presented on Figure 5.5 are similar to the original paper. The first years appear to exhibit a reversed trend, which could be due to difference in the way the logistic regression is being fitted. Figure 5.6 shows that we replicated the main trends but still indicates an issue with race-based affirmative action: our replication leads to a greater impact on minority students. The behavior of quality on Figure 5.8 is similar to the original paper, indicating the adequacy of our quality initialization. A strong claim in the original paper is the decline in quality of colleges using policies, which is not as clear in our replication as only two of the four active colleges suffer such decline. This might be attributed to the lower impact of our actions, as highlighted by the other figures. It is important to note that this figure corresponds to an average over 10 runs (as the original paper), which requires the colleges to be sorted by quality.

The additional left-most arrow covers students who do not enroll. However, it is unspecified however whether it captures students who do not enroll while having applied, being admitted or without condition. We use the latter (see Appendix B). Figure 5.10 shows a successful replication of the general tendency of the model (i.e., the relative values) and a failure in replicating the absolute numbers, probably due to the issue related to the affirmative action observed above. Figures 5.11 and 5.12 shows a successful replication regarding the racial impact of targeted recruiting and SES-based affirmative action strategies, but highlight an issue with the socioeconomic composition (Figure 5.12). Indeed, the impact of our SES-based affirmative action is higher than the original paper. entirely faithful to the ones from the original paper, can still be useful to simulate data grounded in real-world in the context of college admission.

Towards an unfairness warning system

This section presents the next step towards building an unfairness warning system based on the model presented in this chapter.

Long-term dynamic

To observe the impact of fairness concepts on the long-term, we need to model two concepts of long-term impact: the feedback loop and a "snowball effect". The first is already encompassed in the dependencies across states, since variables which represent the context of the decision are impacted by previous decisions, mainly quality of the colleges and probability of admission.

Snowball: decision accumulation

To model the snowball effect, we add a dynamic aspect to the students, so that innate attributes of individuals are impacted by previous states. Our original definition of the snowball effect considers decisions at each steps as features for the next. This would imply (1) an ever-growing accumulation of features and (2) a require several modifications to the decision process itself, whereas we try to keep the model as is. Instead of following this individual formulation, we could apply the accumulation on the groups: the weights and distribution of resources and achievement of racial groups are modified according to their respective enrollment, as represented by the red arrows on Figure 5.1. This formulation is equivalent to our original definition if we consider an aggregating step: rather than appending decision y to the features X, we set the new features as a function of X and y. A way to view our proposition is to consider students as children of the parent cohort, benefiting from their successes. In short, we can model the impact on students in two ways: Representativity The weights of the race distribution can be modified to fit the enrollment success of each race. This impacts the representativity of each group, and models a unique cohort of students going to successive stages of selection.

Reproduction

The race-based weights of achievement and resources can be penalized or advantaged depending on the enrollment rate of past cohorts, potentially taking into account the quality of colleges in which students where enrolled. This would correspond to future students benefiting or not from their parents academical background.

Interventions

The model allows for a broad kind of fairness intervention policies. First, it mixes different kinds of decisions: (1) a numerical estimation of the probability of admission used by students, (2) an optimization problem to build the application portfolio, (3) a top-k ranking for the admission of students, as well as finally a top-1 ranking for the final enrollment. Since each mechanism can independently use noisy observation or perfect observation, different combinations can be envisioned. In particular, the point (3) allows our model to be used to evaluate fair rankings [START_REF] Zehlike | Fairness in Ranking, Part I: Score-Based Ranking[END_REF], while the point (1) would allow tools such as [START_REF] Rachel | AI Fairness 360: An extensible toolkit for detecting and mitigating algorithmic bias[END_REF] to be plugged in, providing multiple fairness-aware machine learning models. Apart from algorithmic interventions, the way bias is added through observation of variables by students and colleges (e.g., through the SAch, CAch, and SQual variables), allows for more classical fairness policies to be studied. Affirmative action, in the form of a bonus weight added to minority groups, is already part of the model. Random noise can be used as a baseline by removing the correlation between observation noise and race.

Reference scenarios

In addition to providing a playground for multiple fairness-aware algorithms, the model allows for many different fairness assumptions. As discussed in Section 2.2, the particular state of reference used to evaluate fairness is crucial, and a matter of social consideration. Leveraging a simulation is particularly interesting here since we can compare the impact of using a particular reference. Figure 5.15 illustrates such options with red arrows. Fair transitions of the alternate chain can be achieved by accessing the pre-bias students variables (i.e., "innate") resources and achievement, which are available as random variables before mixing. When evaluating fairness at state S i , several references can be used: Previous using the previous state is the typical choice of many fairness papers, with the data used to "train" the current state S i is used as reference;

Past similarly, an arbitrary state in the past S j can be used, increasing the window of observation;

Initial although generally undefined in real-world scenarios, we can also use the initial state of the simulation S 0 ;

Alternative more interesting, we can use for reference an alternative state S i which is the result of using an alternative transition, for example one implementing a specific fairness intervention, as defined above;

Parallel finally, as an option only available in the context of a simulation, we can use for reference a state S i , which is the state existing in a parallel chain in which every transition was using an alternative fair transition.

Red flag notion of fairness

The idea of "automated 'early warning system' for discrimination" is proposed in [START_REF] Benbouzid | Fairness in machine learning from the perspective of sociology of statistics: How machine learning is becoming scientific by turning its back on metrological realism[END_REF], and we share this objective with this model. This requires the definition of a "warning", therefore needing an indicator to be selected. However, selecting one fairness metric above the others would go against this thesis. As an agnostic fairness metric-if any-we advocate for a distance between distributions, such as the earth mover's distance for instance. This does not provide a fairness notion by itself-outside of the extreme case of requiring everyone to receive the same decision, but can be used as a "red flag" indicating a potential issue to be investigated. Indeed, it would be "raised" when disparity arises between the distributions of both attributes and decisions across groups, alerting experts of the field that a deeper inspection is required to ensure that the mechanism under scrutiny is fair. Rather than taking a state as reference as discussed above, we would then need to compare the evolution between two states of time, assessing the rate by which the distance increases (or, ideally, decreases) and comparing that rate to another one. This is represented with blue arrows on Figure 5.15, in which the distribution changes V -1 (between states S i and S i-1 ) can be compared to the change V or V . This approach would allow to define a "(un)fairness rate", which corresponds to a direction (i.e., increase or decrease of distance) and velocity of (un)fairness. Rather than adding to the existing fairness metrics, this could serve as a way to detect if an investigation by proper experts is potentially needed.

Conclusion

In this thesis, we have attempted to highlight the intrinsic difficulties of applying technical solutions to social issues, and how these solutions appear unsuited to the actual concern. More precisely, first we have looked at the difficulties encountered when state-of-the-art privacy-preserving mechanisms need to be applied under strong-if not immutable-utility and transparency constraints. This clash is arising with the will to develop technological tools benefiting from data that is, at the same time, subject to social requirements, as is the case with legal proceedings. Afterwards, we shifted towards the evaluation of privacy empirical guarantees, with the objective of better understanding how formal frameworks would impact real-life data and processes. This was done by creating a framework to help with the instantiation of attack challenges, and by organizing a competition on the issue of membership inference attacks of privacy-preserving synthetic data. Our hope is to organize further editions, and potentially to tackle further questions such as the aforementioned trade-off between privacy and utility of natural language sanitization, or the study of the fairness impact of privacy mechanisms.

As a last contribution, we finally turned to fairness issues with the objective of proposing a framework to observe long-term dynamics of technical fairness notions. To this end, we reproduced and adapted a model originating from the community of education policy, with the aim of providing a technical tool grounded in reality. This last point should be thought as both the ending note of this thesis and the broad direction that we identify for the future of this work: identifying not how to use social concepts in technical contexts, but rather identifying when and how technical tools can be used to help with those social issues. Therefore, keeping social issues in the domain of social sciences, and avoiding the technicization of social concepts as if the "technical" was outside the sociotechnical system. Abstract: Algorithms are increasingly used across all layers of society, including in highstake decision systems, impacting individuals, cohorts and society as a whole. This omnipresence of algorithms raises important ethical and social concerns, including in particular privacy and fairness. In this thesis, we study these two technical subjects under the lens of their practical usage and strong societal requirements. In a first contribution we investigate the conflict between privacy and transparency when publishing legal proceedings. In a second contribution, we propose a framework to organize privacy challenges with a fo-cus on attacking privacy-preserving data publishing mechanisms to better define their behavior in practice. As a third contribution, we focus on the limits of technical fairness definitions and leverage a simulation grounded in reality as a way to observe the long-term impact of fairness on a whole system. Overall, the main objective of this thesis is to highlight fundamental issues raised by the technicization of social challenges as well as to propose technical tools and analyses aimed towards bringing these sociotechnical challenges back to their social context.

Name

Figure 2

 2 Figure 2.1.: Example of Mondrian execution with k = 5 using the two-dimensional plane as quasi-identifiers.

  n, D) = b and 0 otherwise. A must output either 0 or 1.

Figure 2

 2 Figure 2.3.: Examples of connections in a causal model, with a collider, fork, and chain.In (a), Z is a collider since it is influenced by both X and Y. In (b), Z is a confounder since it impacts both the input variable X and the output variable Y. In (c), T is a mediator since it lies between the input variable X and the output variable Y in one possible path.

  [Meh+22; Bel+19]: Pre-processing algorithms are used before the actual model learning by modifying the training data [KC11; Zem+13; Fel+15];

Figure 3

 3 Figure 3.1.: High-level pipeline of court files processing for Legal Techs

Quote 5 :Figure 3

 53 Figure 3.2.: Relationship graph manually extracted from Quote 5

Figure 4

 4 Figure 4.3.: Best membership advantage scores on October 5 th , 2023

Figure 5

 5 Figure 5.3.: Simplified causal model of the admission decision.

Figure 5

 5 Figure 5.4.: Simplified causal model of the enrollment decision.

  Figures 5.5 to 5.14 present side by side examples of reproduction, with plots taken from the preprint version [Rea+17] of the original paper [Rea+18] (due to copyright issues) along with our replication results. We attempt to replicate a representative subset of the plots, covering the gist of the model: impact of race-based affirmative action (Figures 5.6, 5.13, and 5.14) and impact of SES-based affirmative action and race-based recruitment (Figures 5.7 to 5.12). Additional replications are available in Appendix B.

Figure 5

 5 Figure 5.5.: Minority enrollment without using affirmative action or recruiting.

Figure 5

 5 Figure 5.6.: Minority enrollment with the top four colleges using real-world race-based affirmative action (weight of 260).

Figure 5 Figure 5

 55 Figure 5.7.: Minority enrollment with the top four colleges using strong SES-based affirmative action (weight of 150) and strong race-based recruitment (weight of 100).

4

  Schools use Race AA Proportion low-income students Mean achievement of enrolled students schools that don't use AA/recruiting schools that use AA/recruiting (a) Original [Rea+17, Fig. D1]

Figure 5

 5 Figure 5.9.: Mean achievement and proportion low-income with top four colleges using (respectively not using) strong SES-based affirmative action (weight of 150) and strong race-based recruitment (weight of 100) on the left sides, and real-world race-based affirmative action (weight of 260) on the right sides.Arrows start at year 14 with no affirmative action, and end at year 29.

Figure 5 .

 5 Figure 5.9 suffers from the same issue of affirmative actions as Figure 5.6, but the general behavior is similar to the paper. Figures B.5 and B.6 in Appendix B present the

Figure 5

 5 Figure 5.10.: Black and Hispanic enrollment in colleges using SES-based affirmative action and race-based recruitment, as a share of estimated enrollment under race-based affirmative action (using estimated real-world affirmative action weight 260).

Figure 5

 5 Figure 5.11.: Racial composition of colleges using SES-based affirmative action and racebased recruitment, by affirmative action and recruitment weights.

Figure 5

 5 Figure 5.12.: Socioeconomic composition of colleges using SES-based affirmative action and race-based recruitment, by affirmative action and recruitment weights.

Figure 5 Figure 5

 55 Figure 5.13.: Racial composition of colleges using SES-based and racial affirmative actions, by affirmative action weights.

  Figure 5.15.: Scenarios

FigureFigure

  Figure B.1.: Minority enrollment with the top four colleges using strong SES-based affirmative action (weight of 150) and strong race-based recruitment (weight of 100) using the formula provided in [Rea+18] for the SES-based affirmative action. See Section 5.4

  "[~Re] Simulating Socioeconomic-Based Affirmative Action". In: ReScience C 9.1 (2023). Ed. by Olivia Guest. doi: 10.5281/zenodo.10255346. hal: hal-04328511.

	Dissemination
	[AGB21]	Tristan Allard, Sébastien Gambs, and Louis Béziaud. "La Confidentialité
		Différentielle, Garante de l'anonymat". In: Pour la Science. Hors-Série 112
		(July 8, 2021).
	Book chapters
	[All+20]	Tristan Allard et al. "Ouvrir La Boîte Noire Des Algorithmes de Person-
		nalisation". In: Le Profilage En Ligne : Entre Libéralisme et Régulation. Ed.
		by Alexandra Bensamoun, Maryline Boizard, and Sandrine Turgis. Libre
		Droit. Mare & Martin, Oct. 15, 2020, pp. 211-231. isbn: 978-2-84934-466-8.

Peer-reviewed international workshops and demonstrations

[START_REF] Allard | Online publication of court records: circumventing the privacy-transparency trade-off[END_REF]

Tristan Allard, Louis Béziaud, and Sébastien Gambs. "Online publication of court records: circumventing the privacy-transparency trade-off". In: CoRR (2020). Presented at ICML 2020's Law and Machine Learning Workshop. arXiv: 2007.01688. [ABG20b] Tristan Allard, Louis Béziaud, and Sébastien Gambs. "Publication of Court Records: Circumventing the Privacy-Transparency Trade-Off". In: AI Approaches to the Complexity of Legal Systems XI-XII -AICOL International Workshops 2018 and 2020: AICOL-XI@JURIX 2018, AICOL-XII@JU-RIX 2020, XAILA@JURIX 2020, Revised Selected Papers. Ed. by Víctor Rodríguez-Doncel et al. Vol. 13048. Lecture Notes in Computer Science. Springer, 2020, pp. 298-312. doi: 10.1007/978-3-030-89811-3_21. hal: hal-03225201. [ABG23c] Tristan Allard, Louis Béziaud, and Sébastien Gambs. "SNAKE Challenge: Sanitization Algorithms under Attack". In: Proceedings of the 32 nd ACM International Conference on Information & Knowledge Management. Birmingham, United Kingdom: Association for Computing Machinery, Oct. 21, 2023. doi: 10.1145/3583780.3614754. hal: hal-04228115.

  High level view of the workflow implementation for the first edition. Rectangles represent data with rules as edge labels. The Attack rhombus is the (local) attack algorithm created by the participant. The Score ellipse is computed on CodaBench.

		Original data	prepare	Base data	sample
		Targets		Train-data
					synthesize
		Membership		Synthetic data
	Scores	Guesses	Attack
		Public	Private	Input	Submission
	Figure 4.2.:

  .1.

	Variable Description
	Race	Student's quality
	Ach	Student's achievement
	Res	Student's resources
	Qual	College's quality
	SAch	Student's perception of their own achievement
	CAch	College's perception of a student's achievement
	SQual	Student's perception of a college's quality
	PAdm	Race Student's estimation of its probability of admission into a college
	Util	Student's perception of the utility of a college
	#App	Student's number of applications
	#Adm	Ach College's number of admissions Res	Qual
	EnrYield	College's estimation of its enrollment yield
	App	Student's application to a college
	Adm Enr	CAch College's admission of a student SAch Student's enrollment into a college SQual
	Table 5.1.: Description of variables of the college-student model
		PAdm		Util
				App	#App
			Adm	#Adm
		Enr	EnrYield
		Random variable	Decision
	Figure 5.1.: Simplified causal model. Colored arrows represent causal relationship on
	future states.

Table 5

 5

	.2.: Experimental parameters

  Table 5.2 describes the available parameters and the values used in the experiments.Figures 5.11 to 5.14 present the impact of the different strategies on the final (years 25 to 29) composition-racial and socioeconomic 3 -of enrolled students in colleges. Our replication effort involved experimenting with the full set of plots presented in the original paper. We also resorted to additional plots (not presented in the original paper or the current one) to assess the correlation between admission probability and academic achievement. Every experiment results from an average of 10 runs.

		.4
		.3
	Percent Minority	.2
		.1
		0			Colleges begin to use Affirm. Action	End of Standard Simulation
		0	5	10	15	20	25 Year	30	35	40	45	50
				Non-Affirmative Action College			Affirmative Action College
				Average of Non-Affirm. Action Colleges		Average of Affirm. Action Colleges

  Enjeux de vie privée et d'équité posés par les systèmes de décision algorithmiques Mot clés : vie privée, équité, système de décision, socio-technique Résumé : Les algorithmes sont de plus en plus utilisés pour prendre des décisions impactant les individus, les cohortes et la société dans son ensemble. Cette ubiquité soulève d'importantes préoccupations éthiques et sociales, notamment la protection de la vie privée et l'équité. Cette thèse étudie ces deux sujets techniques sous l'angle de leur utilisation pratique et de leurs exigences sociétales. Dans une première contribution, nous examinons le conflit entre la vie privée et la transparence lors de la publication de procédures judiciaires. Dans une deuxième contribution, nous proposons un framework pour organiser des compétitions d'attaque de mécanismes de protection de la vie privée afin de mieux établir leur comportement dans la pratique. Dans une troisième contribution, nous nous concentrons sur les limites des définitions techniques d'équité et utilisons une simulation ancrée dans la réalité comme moyen d'observer leur impact à long terme sur l'ensemble d'un système. L'objectif principal de cette thèse est de mettre en évidence les questions fondamentales soulevées par la technicisation des défis sociaux ainsi que de proposer des outils techniques et des analyses visant à ramener ces défis sociotechniques dans leur contexte social.

Titre :

Title: Privacy and fairness issues with algorithmic decision systems Keywords: privacy, fairness, decision systems, sociotechnical

En matière de protection de la vie privée, les premières réponses se sont faites avec des mécanismes empiriques. Par exemple, la k-anonymité introduite en 2002[START_REF] Sweeney | k-Anonymity: A Model for Protecting Privacy[END_REF] cache les profils individuels en les regroupant en grappes d'au moins k personnes. Ce modèle de vie privée a été successivement raffiné avec des volontés de contrôler l'utilité (plus précisément la perte d'information) et de répondre à des attaques, amenant à la l-diversité [Mac+07] puis à la t-proximité[START_REF] Li | t-Closeness: Privacy Beyond k-Anonymity and l-Diversity[END_REF].En parallèle, la confidentialité différentielle[START_REF] Dwork | Differential Privacy[END_REF] a été proposée en 2006 avec un objectif formel devant représenter le concept de vie privée auquel obéit la définition. Plus précisément, un mécanisme respecte la vie privée (au sens de la confidentialité différentielle) si la présence ou l'absence d'un individu dans la base de données fournie en entrée au mécanisme impacte le calcul de manière indiscernable, selon un budget alloué. En d'autres mots, étant donné le résultat d'une requête, un•e attaquant•e ne gagne qu'une information limitée permettant d'inférer la présence ou absence de l'individu.Si cette définition est considérée, dans ce travail comme dans beaucoup d'autres, comme la référence, de très nombreuses variations ont été proposées [DP20] afin de répondre à différents modèles d'adversaire, de connaissances auxiliaires ou d'autres structures de données. À la question du choix d'une définition s'ajoute celle du choix d'un budget (et des variables additionnelles d'autres modèles), qui définit la quantité d'information pouvant fuiter, avec des tentatives de recommandation académiques [Hsu+14 ; LC11] et des utilisations pratiques différentes dans l'industrie[START_REF] Tang | Privacy Loss in Apple's Implementation of Differential Privacy on MacOS 10[END_REF].

The majority of the legal technologies market consists in commercial applications. They do not give information about their inner working and underlying techniques.

A quasi-identifier is a combination of attributes that are usually unique in the population, thus indirectly identifying an individual. A typical example is the triple (age, zip code, gender).

Participants can use the framework to generate their own versions of the tasks locally and therefore

It gives teams the knowledge of the population distribution commonly assumed in membership inference attacks.

Executing this model seems to require Stata version 11, but the dependency "college sorting napps.do" is not distributed in the archive.

The zscore variable is the number of standard deviations by which the value of a raw score is above or below the mean value of what is being observed or measured

Our socioeconomic distribution is represented by the quintiles of the full students' resources distribution (not only the enrolled students) as doing otherwise results in incomparable categories.

Result Although we achieve mostly partial successes, the overall behavior is successfully reproduced, with discrepancies regarding the impact of positive discrimination policies. We attribute the latter to issues with our understanding of the paper, thus highlighting the difficulties of describing and understanding complex systems, and the need for open-access to code. The resulting implementation, while not producing results

state-of-the-art sanitization algorithms, (2) enables to give sufficient time to participants to design and test carefully their attacks, (3) enables the exploration of a wide range of possible adversary models and background knowledge and (4) complements nicely the current sanitization competitions that focus on the sanitization part, thus resulting in a complete defense-attack pipeline. The first edition of the challenge focuses on the Membership Inference Attack setting [Hu+22; SOT22] (see Section 2.1) over tabular data synthetically generated while satisfying differential privacy [START_REF] Dwork | The Algorithmic Foundations of Differential Privacy[END_REF].

The adversarial background knowledge consists in the record(s) of the target(s) of the attack. Further editions will showcase other inference attacks, adversarial background knowledge, privacy models and sanitization algorithms. The abstractions defined in SNAKE consist in a workflow and its functional specifications. Figure 4.1 describes the SNAKE workflow. The rule Prepare data creates the main dataset of the challenge, called Base data, from some existing source of data, called Original data. Each task of the challenge is created by Create task, as a set of Public and Private files, with the former shared to participants, and the former used organizers (and the platform) for evaluation. The third rule Create bundle packages all the competition files into (1) a bundle can be uploaded to CodaBench to instantiate the full competition, (2) a public bundle which is to be hosted online to share the data necessary for participants and (3) a private bundle that is to be shared after the challenge.

Technical Foundations

SNAKE leverages (1) the Snakemake workflow management system [START_REF] Köster | Snakemake -a scalable bioinformatics workflow engine[END_REF] for its inner working and (2) the CodaBench competition platform [START_REF] Xu | Codabench: Flexible, Easy-to-Use and Reproducible Benchmarking for Everyone[END_REF] for all the features necessary for running a competition in real-life (e.g., user registration, submission management, leaderboard). Snakemake is a bioinformatics workflow engine which provides a domain specific language implemented as an extension to Python to describes pipelines. Workflows are specified as a directed acyclic graph of rules which transform input files During the first stage, illustrated on Figure 5.2, students select a subset-called portfolioof colleges to apply to. Students observe colleges' quality with some amount of uncertainty representing "imperfect information and idiosyncratic preferences". This error depends on students' resources as a way to model high-resources families having better information about college quality. Based on their noisy observations of their own achievement and college quality, students estimate their probabilities of admission into each college using a logistic model fitted on admission patterns over the past five years, represented by blue arrows on Figure 5.1. Each student applies to a subset of colleges that maximize their overall expected utility. More precisely, the expected utility of an application portfolio is computed as:

Application

Race

with c 1 , c 2 , . . . , c n ordered by descending utility. The model assumes that all students are rational, utility-maximizing agents. The variability in student application portfolios originates from the imperfect observation of variables.

Race-targeted recruiting. "Colleges' binary recruitment statuses (S c )-which had previously all been 0-are set based on model parameters that determine which colleges will use recruitment […] Utility is then calculated using model-specific recruitment magnitude values (L):

There is no use of S c in the document apart from the definition, and R sc is also not defined. We assume that it is a typo and that S c is to be understood as R sc . The recruitment magnitude value L is not used (but is an experimental parameter later). No specific definition of race-targeted recruitment is given but from "Recruitment efforts work in part by making students aware of specific colleges and by making these colleges seem more appealing to prospective students through additional, targeted contact with those students." [Rea+18, Section 3] our understanding is that targeted groups have an increased chance of applying to colleges using race-targeted recruitment. However, the race attribute is not used in conjunction with L.

We assume that the race-targeted recruitment weight is added to the perceived utility of attending colleges for minority students, such that

in which T c indicates whether a college uses race-targeted recruitment and R is the recruitment weight.

Quality update window.

It is unclear whether the quality of colleges depends only on the previous year or on the past five years:

• "Colleges' quality values (Q c ) are updated based on the incoming class of enrolled students before the next year's cohort of students begins the application process" [Rea+18, Appendix C].

• "College quality is calculated as the five-year running average of enrolled student caliber." [Rea+18, Figure C5]

The later point is given in the caption of a figure, whereas the first is part of the model description. Using only the previous year to update colleges' quality gives satisfying results (see Figure 5.8). Thus, we assume that the five-year running average is not used for the model but only to smooth the plots.

Initial college quality. "Initial college quality (Q) is normally distributed" [Rea+18,

Appendix C], but the values of the mean and variance are not given. We relied on the values from [START_REF] Reardon | Agent-Based Simulation Models of the College Sorting Process[END_REF]: Q ∼ N (1070, 130 2 ).

Admission probability estimation.

After the fifth iteration, each iteration requires fitting a logistic model on submitted application over the past five years. This allows students to estimate their probability of admission into each college, which is required for them to select a subset to apply to.

Bibliography

A. College-Student model

In this appendix we present in details the model as described in [START_REF] Reardon | What Levels of Racial Diversity Can Be Achieved with Socioeconomic-Based Affirmative Action? Evidence from a Simulation Model[END_REF].

A.1. Notation and variables

Variables All used variables along with their dimensions and indexes are listed in Table A.1. We note S the set of students, C the set of colleges, and Y the set of iterations.

A.2. Model

For each run of the model, J = 40 colleges are generated. At each iteration, a new cohort of N = 10000 students is sampled. Colleges are represented by a quality attribute Q, initially drawn from a normal distribution N (130, 1100). Students have three attributes: race, resources R, and achievement A. The latter two are sampled from race-specific bivariate normal distributions with non-zero correlation.

Three successive decisions are taken at each iteration:

1. Students apply to a subset of colleges;

2. Colleges admit a subset of students from the applications;

3. Students enroll into the best college they are admitted to.

Race is samples from a categorical distribution as given in Table A.2. Resources and achievement are sampled from bivariate normal distributions with nonzero correlation and conditioned on race. The student model can be expressed as the following mixture model, with parameters of the components given in Table

Figure 5.1 proposes an abstract view of the model, dropping the educational context. Individuals have a sensitive attribute Z, which impacts their attributes I and E. Attributes I is innate-that is they cannot be acted upon-and attributes E are external (i.e. received). Those two kinds of attributes are combined as an observed feature set X, with the "objective" of X capturing I but suffering the impact of E. A decision Y is taken on X, and on some context Q independent of the individuals. The decision then impacts future contexts and future external variables.

A.2.1. Application

During the first stage, students select a subset-called portfolio-of colleges to apply to. Students observe colleges' quality with some amount of uncertainty which represents "imperfect information and idiosyncratic preferences". This error depends on students' resources as a way to model high-resources families having better information about college quality.

ρ Q S = max(0.5, min(0.9, 0.7 + 0.1 × Res)

Students use observed college quality to evaluate the utility of attending that college:

Students may augment their own achievement, and they perceive their own achievement with noise. Thus, their assessment of their achievement, for purposes of deciding where to apply, is:

where α s represents enhancements such as extracurricular activities. As above, the error in a student's assessment of his or her own achievement has a variance that depends on his or her family resources. Based on their noisy observations of their own achievement and college quality, students estimate their probabilities of admission into each college using a logistic model fitted on admission patterns over the prior 5 years. For the first 5 years of the simulation, the intercept is set to 0 and the slope to -0.015.

Each student applies to a set of NApp = 4 + [0.5 × R] colleges that maximize their overall expected utility.

A.2.2. Admission

The second decision phase consists in the admission of students by colleges.

Colleges observe the achievement of applicants with some noise, similar to the observation of quality by students and serves the same purpose. Colleges assess students' achievement with a reliability of 0.8.

A.2.3. Enrollment

Students enroll in the college with the highest estimated utility of attendance (U * ) to which they were admitted.

A.2.4. Iteration

Colleges' quality (Q) are updated each year based on the new enrolled students as:

where Ā is the average value of A among the newest cohort of students enrolled in college c.

A.2.5. Simulation duration

The scenarios in [START_REF] Reardon | What Levels of Racial Diversity Can Be Achieved with Socioeconomic-Based Affirmative Action? Evidence from a Simulation Model[END_REF] last 30 or 50 iterations, with the first 10 years used as "bootstrapping" to stabilizes iteration-dependent variables such as the logistic regression or colleges' quality.

B. Further questions of reproductibility