Gaurav Maheshwari

Pascal Denis

Mikaela Keller

Aurélien Bellet

Fair NLP Models with Differentially Private Text Encoders

Encoded text representations often capture sensitive attributes about individuals (e.g., race or gender), which raise privacy concerns and can make downstream models unfair to certain groups. In this work, we propose FEDERATE, an approach that combines ideas from differential privacy and adversarial training to learn private text representations which also induces fairer models. We empirically evaluate the trade-off between the privacy of the representations and the fairness and accuracy of the downstream model on four NLP datasets. Our results show that FEDERATE consistently improves upon previous methods, and thus suggest that privacy and fairness can positively reinforce each other.

Introduction

Algorithmically-driven decision-making systems raise fairness concerns [START_REF] Raghavan | Mitigating bias in algorithmic hiring: evaluating claims and practices[END_REF][START_REF] Van Den Broek | Hiring algorithms: An ethnography of fairness in practice[END_REF] as they can be discriminatory against specific groups of people. These systems have also been shown to leak sensitive information about the data of individuals used for training or inference, and thus pose privacy risks [START_REF] Shokri | Membership inference attacks against machine learning models[END_REF]. Societal pressure as well as recent regulations push for enforcing both privacy and fairness in real-world deployments, which is challenging as these notions are multi-faceted concepts that need to be tailored to the context. Moreover, privacy and fairness can be at odds with one another: recent studies have shown that preventing a model from leaking information about its training data negatively impacts the fairness of the model and vice versa [START_REF] Bagdasaryan | Differential privacy has disparate impact on model accuracy[END_REF][START_REF] Pujol | Fair decision making using privacy-protected data[END_REF][START_REF] Cummings | On the compatibility of privacy and fairness[END_REF][START_REF] Chang | On the privacy risks of algorithmic fairness[END_REF].

This paper studies fairness and privacy and their interplay in the NLP context, where these two notions have often been considered independently from one another. Modern NLP heavily relies on learning or fine-tuning encoded representations of text. Unfortunately, such representations often leak sensitive attributes (e.g., gender, race, or age) present explicitly or implicitly in the input text, even when such attributes are known to be irrelevant to the task [START_REF] Song | Information leakage in embedding models[END_REF]. Moreover, the presence of such information in the representations may lead to unfair downstream models, as has been shown on various NLP tasks such as occupation prediction from text bios [START_REF] De-Arteaga | Bias in bios: A case study of semantic representation bias in a high-stakes setting[END_REF], coreference resolution [START_REF] Zhao | Gender bias in coreference resolution: Evaluation and debiasing methods[END_REF], or sentiment analysis [START_REF] Kiritchenko | Examining gender and race bias in two hundred sentiment analysis systems[END_REF].

Privatizing encoded representations is thus an important, yet challenging problem for which existing approaches based on subspace projection [START_REF] Bolukbasi | Man is to computer programmer as woman is to homemaker? debiasing word embeddings[END_REF][START_REF] Wang | Double-hard debias: Tailoring word embeddings for gender bias mitigation[END_REF][START_REF] Karve | Conceptor debiasing of word representations evaluated on WEAT[END_REF][START_REF] Ravfogel | Null it out: Guarding protected attributes by iterative nullspace projection[END_REF] or adversarial learning [START_REF] Li | Towards robust and privacy-preserving text representations[END_REF][START_REF] Coavoux | Privacy-preserving neural representations of text[END_REF][START_REF] Han | Diverse adversaries for mitigating bias in training[END_REF] do not provide a satisfactory solution. In particular, these methods lack any formal privacy guarantee, and it has been shown that an adversary can still recover sensitive attributes from the resulting representations with high accuracy [START_REF] Elazar | Adversarial removal of demographic attributes from text data[END_REF][START_REF] Gonen | Lipstick on a pig: Debiasing methods cover up systematic gender biases in word embeddings but do not remove them[END_REF].

Instead of relying on adversarial learning to prevent attribute leakage, [START_REF] Lyu | Differentially private representation for NLP: formal guar-antee and an empirical study on privacy and fairness[END_REF]; [START_REF] Plant | Cape: Context-aware private embeddings for private language learning[END_REF] recently propose to add random noise to text representations so as to satisfy differential privacy (DP), a mathematical definition which comes with rigorous guarantees [START_REF] Dwork | Calibrating noise to sensitivity in private data analysis[END_REF]. However, we uncover a critical error in their privacy analysis which drastically weakens their privacy claims. Moreover, their approach harms accuracy and fairness compared to adversarial learning.

In this work, we propose a novel approach (called FEDERATE) to learn private text representations and fair models by combining ideas from DP with an adversarial training mechanism. More specifically, we propose a flexible end-to-end architecture in which (i) the output of an arbitrary text encoder is normalized and perturbed using random noise to make the resulting encoder differentially private, and (ii) on top of the encoder, we combine a classifier branch with an adversarial branch to actively induce fairness, improve accuracy and further hide specific sensitive attributes.

We empirically evaluate the privacy-fairnessaccuracy trade-offs achieved by FEDERATE over four datasets and find that it simultaneously leads to more private representations and fairer models than state-of-the-art methods while maintaining comparable accuracy. Beyond the superiority of our approach, our results bring valuable insights on the complementarity of DP and adversarial learning and the compatibility of privacy and fairness. On the one hand, DP drastically reduces undesired leakage from adversarially trained representations, and has a stabilizing effect on the training dynamics of adversarial learning. On the other hand, adversarial learning improves the accuracy and fairness of models trained over DP text representations.

Our main contributions are as follows:

• We propose a new approach, FEDERATE, which combines a DP encoder with adversarial learning to learn fair and accurate models from private representations.

• We identify and fix (with a formal proof) a critical mistake in the privacy analysis of previous work on learning DP text representations.

• We empirically show that FEDERATE leads to more private representations and fairer models than state-of-the-art methods while maintaining comparable accuracy.

• Unlike previous studies, our empirical results suggest that privacy and fairness are compatible in our setting, and even mutually reinforce each other.

The paper is organized as follows. Section 2 provides background on differential privacy. Section 3 presents our approach. Section 4 reviews related work. Experimental results and conclusions are given in Sections 5 and 6.

2 Background: Differential Privacy Differential Privacy (DP) [START_REF] Dwork | Calibrating noise to sensitivity in private data analysis[END_REF] provides a rigorous mathematical definition of the privacy leakage associated with an algorithm. It does not depend on assumptions about the attacker's capabilities and comes with a powerful algorithmic framework. For these reasons, it has become a de-facto standard in privacy currently used by the US Census Bureau (Abowd, 2018) and several big tech companies [START_REF] Erlingsson | Rappor: Randomized aggregatable privacy-preserving ordinal response[END_REF][START_REF] Fanti | Building a RAPPOR with the unknown: Privacypreserving learning of associations and data dictionaries[END_REF][START_REF] Bolin Ding | Collecting telemetry data privately[END_REF]. This section gives a brief overview of DP, focusing on the aspects needed to understand our approach (see [START_REF] Dwork | The algorithmic foundations of differential privacy[END_REF] for an in-depth review of DP).

Over the last few years, two main models for DP have emerged: (i) Central DP (CDP) [START_REF] Dwork | Calibrating noise to sensitivity in private data analysis[END_REF], where raw user data is collected and processed by a trusted curator, which then releases the result of the computation to a third party or the public, and (ii) Local DP (LDP) [START_REF] Prasad Kasiviswanathan | What can we learn privately?[END_REF] which removes the need for a trusted curator by having each user locally perturb their data before sharing it. Our work aims to create an encoder that leads to a private embedding of an input text, which can then be shared with an untrusted curator for learning or inference. We thus consider LDP, defined as follows. Definition 1 (Local Differential Privacy). A randomized algorithm M : X → O is -differentially private if for all pairs of inputs x, x ∈ X and all possible outputs o ∈ O:

Pr[M (x) = o] ≤ e Pr[M (x ) = o].
(1)

LDP ensures that the probability of observing a particular output o of M should not depend too much on whether the input is x or x . The strength of privacy is controlled by , which bounds the log-ratio of these probabilities for any x, x . Setting = 0 corresponds to perfect privacy, while → ∞ does not provide any privacy guarantees (as one may be able to uniquely associate an observed output to a particular input). In our approach described in Section 3, x will be an input text and M will be an encoding function which transforms x into a private vector representation that can be safely shared with untrusted parties.

Laplace mechanism. As clearly seen from Definition 1, an algorithm needs to be randomized to satisfy DP. A classical approach to achieve -DP for vector data is the Laplace mechanism [START_REF] Dwork | Calibrating noise to sensitivity in private data analysis[END_REF]. Given the desired privacy guarantee and an input vector x ∈ R D , this mechanism adds centered Laplace noise Lap( ∆ ) independently to each dimension of x. The noise scale ∆ is calibrated to and the L1-sensitivity ∆ of inputs:

∆ = max x,x ∈X x -x 1 .
(2)

In our work, we propose an architecture in which the Laplace mechanism is applied on top of a trainable encoder to get private representations of input texts, and is further combined with adversarial training to learn fair models.

Approach

We consider a scenario similar to [START_REF] Coavoux | Privacy-preserving neural representations of text[END_REF], where a user locally encodes its input data (text) x into an intermediate representation E priv (x) which is then shared with an untrusted curator to predict the label y associated with x using a classifier C. Additionally, an attacker (which may be the untrusted curator or an eavesdropper) may observe the intermediate representation E priv (x) and try to infer some sensitive (discrete) attribute z about x (e.g., gender, race etc.). Our goal is to learn an encoder E priv and classifier C such that (i) the attacker performs poorly at inferring z from E priv (x), (ii) the classifier C(E priv (x)) is fair with respect to z according to some fairness metric, and (iii) C accurately predicts the label y.

To achieve the above goals we introduce FEDERATE (for Fair modEls with DiffERentiAlly private Text Encoders), which combines two components: a differentially private encoder and an adversarial branch. Figure 1 shows an overview of our proposed architecture.

Differentially Private Encoder

We propose a generic private encoder construction E priv = priv • E composed of two main components. The first component E can be any encoder which maps the text input to some vector space of dimension D. It can be a pre-trained language model along with a few trainable layers, or it can be trained from scratch. The second component priv is a randomized mapping which transforms the encoded input to a differentially private representation. Given the desired privacy guarantee > 0, this mapping is obtained by applying the Laplace mechanism (see Section 2) to a normalized version of the encoded representation E(x):

priv(E(x)) = E(x)/ E(x) 1 + , (3)
where each entry of ∈ R D is sampled independently from Lap( 2 ). We will prove that E priv = priv • E satisfies -DP in Section 3.4.

Adversarial Component

To improve the fairness of the downstream classifier C, we model the adversary by another classi- fier A which aims to predict z from the privately encoded input E priv (x). The encoder E priv is optimized to fool A while maximizing the accuracy of the downstream classifier C. Specifically, given λ > 0, we train E priv , C and A (parameterized by θ E , θ C , and θ A respectively) to optimize the following objective:

min θ E ,θ C max θ A L class (θ E , θ C ) -λL adv (θ E , θ A ), (4)
where L class (θ E , θ C ) is the cross-entropy loss for the C • E priv branch and L adv (θ E , θ A ) is the crossentropy loss for the A • E priv branch.

Training

We train the private encoder E priv and the classifier C from a set of public tuples (x, y, z) by optimizing (4) with backpropagation using a gradient reversal layer g λ [START_REF] Ganin | Unsupervised domain adaptation by backpropagation[END_REF]. The latter acts like an identity function in the forward pass but scales the gradients passed through it by -λ in the backward pass. This results in E priv receiving opposite gradients to A. We give pseudo-code in Appendix A.

Privacy Analysis

We show the following privacy guarantee.

Theorem 1. Our encoder E priv and the downstream predictions C • E priv satisfy -DP.

The proof is given in Appendix B. Theorem 1 shows that the encoded representations produced by E priv have provable privacy guarantees: in particular, it bounds the risk that the sensitive attribute z of a text x is leaked by E priv (x).1 These privacy guarantees naturally extend to the downstream prediction C(E priv (x)) due to the post-processing properties of DP (see Appendix B for details).

Error in previous work. We found a critical error in the privacy analysis of previous work on differential private text encoders [START_REF] Lyu | Differentially private representation for NLP: formal guar-antee and an empirical study on privacy and fairness[END_REF][START_REF] Plant | Cape: Context-aware private embeddings for private language learning[END_REF]. In a nutshell, they incorrectly state that normalizing each entry of the encoded representation in [0, 1] allows to bound the sensitivity of their representation by 1, while it can in fact be as large as D (the dimension of the representation). As a result, the privacy guarantees are dramatically weaker than what the authors claim: the values they report should be multiplied by D. In contrast, the L1 normalization we use in (3) ensures that the sensitivity of E is bounded by 2. We provide more details in Appendix C.

Interestingly, [START_REF] Habernal | When differential privacy meets NLP: the devil is in the detail[END_REF] recently identified an error in ADePT [START_REF] Krishna | ADePT: Auto-encoder based differentially private text transformation[END_REF], a differentially private auto-encoder for text rewriting. However, the error in ADePT is different from the one in [START_REF] Lyu | Differentially private representation for NLP: formal guar-antee and an empirical study on privacy and fairness[END_REF]; [START_REF] Plant | Cape: Context-aware private embeddings for private language learning[END_REF]: the problem with ADePT is that it calibrates the noise to L2 sensitivity, while the Laplace mechanism requires L1 sensitivity. These errors call for greater scrutiny of differential privacy-based approaches in NLP-our work contributes to this goal.

Related Work

Adversarial learning. In order to improve model fairness or to prevent leaking sensitive attributes, several approaches employ adversarialbased training. For instance, [START_REF] Li | Towards robust and privacy-preserving text representations[END_REF] propose to use a different adversary for each protected attribute, while [START_REF] Coavoux | Privacy-preserving neural representations of text[END_REF] consider additional loss components to improve the privacyaccuracy trade-off of the learned representation. [START_REF] Han | Diverse adversaries for mitigating bias in training[END_REF] introduce multiple adversaries focusing on different aspects of the representation by encouraging orthogonality between pairs of adversaries. Recently, Chowdhury et al. (2021) propose an adversarial scrubbing mechanism. However, they purely focus on information leakage, and not on fairness. Moreover, unlike our approach, these methods do not offer formal privacy guarantees. In fact, it has been observed that one can recover the sensitive attributes from the representations by training a post-hoc non linear classifier [START_REF] Elazar | Adversarial removal of demographic attributes from text data[END_REF]. This is confirmed by our empirical results in Section 5.

Sub-space projection. A related line of work focuses on debiasing text representations using projection methods [START_REF] Bolukbasi | Man is to computer programmer as woman is to homemaker? debiasing word embeddings[END_REF][START_REF] Wang | Double-hard debias: Tailoring word embeddings for gender bias mitigation[END_REF][START_REF] Karve | Conceptor debiasing of word representations evaluated on WEAT[END_REF]. The general approach involves identifying and removing a sub-space associated with sensitive attributes. However, they rely on a manual selection of words in the vocabulary which is difficult to generalize to new attributes. Furthermore, [START_REF] Gonen | Lipstick on a pig: Debiasing methods cover up systematic gender biases in word embeddings but do not remove them[END_REF] showed that sensitive attributes still remain present even after applying these approaches.

Recently, [START_REF] Ravfogel | Null it out: Guarding protected attributes by iterative nullspace projection[END_REF] propose Iterative Null space Projection (INLP). It involves iteratively training a linear classifier to predict sensitive attributes followed by projecting the representation on the classifier's null space. However, the method can only remove linear information from the representation. By leveraging DP, our approach provides robust guarantees that do not depend on the expressiveness of the adversary, thereby providing protection against a wider range of attacks.

DP and fairness. Recent work has studied the interplay between DP and (group) fairness in the setting where one seeks to prevent a model from leaking information about individual training points. Empirically, this is evaluated through membership inference attacks, where an attacker uses the model to determine whether a given data point was in the training set [START_REF] Shokri | Membership inference attacks against machine learning models[END_REF]. While [START_REF] Kulynych | Disparate vulnerability to membership inference attacks[END_REF] observed that DP reduces disparate vulnerability to such attacks, it has also been shown that DP can exacerbate unfairness [START_REF] Bagdasaryan | Differential privacy has disparate impact on model accuracy[END_REF][START_REF] Pujol | Fair decision making using privacy-protected data[END_REF]. Conversely, [START_REF] Chang | On the privacy risks of algorithmic fairness[END_REF] showed that enforcing a fair model leads to more privacy leakage for the unprivileged group. This tension between DP and fairness is further confirmed by a formal incompatibility result between -DP and fairness proved by [START_REF] Cummings | On the compatibility of privacy and fairness[END_REF], albeit in a restrictive setting. Some recent work attempts to train models under both DP and fairness constraints [START_REF] Cummings | On the compatibility of privacy and fairness[END_REF][START_REF] Xu | Removing disparate impact of differentially private stochastic gradient descent on model accuracy[END_REF][START_REF] Liu | Fair differential privacy can mitigate the disparate impact on model accuracy[END_REF], but this typically comes at the cost of enforcing weaker privacy guarantees for some groups. Finally, Jagielski et al. ( 2019) train a fair model under DP constraints only for the sensitive attribute.

A fundamental difference between this line of work and our approach lies in the kind of privacy we provide. While the above approaches study (central) DP as a way to design algorithms which protect training points from membership inference attacks on the model, we construct a private encoder such that the encoded representation does not leak sensitive attributes of the input. Thus, unlike previous work, we provide privacy guarantees with respect to the model's intermediate representation for data unseen at training time, and empirically observe that in this case privacy and fairness are compatible and even mutually reinforce each other.

DP representations for NLP. In a setting similar to ours, [START_REF] Lyu | Differentially private representation for NLP: formal guar-antee and an empirical study on privacy and fairness[END_REF] propose to use DP to privatize model's intermediate representation. Unlike their method, we actively promote fairness by using an adversarial training mechanism, which leads to more private representations and fairer models in practice. Importantly, we also uncover a critical error in their privacy analysis (see Sec. 3.1).

Concurrent to and independently from our work, [START_REF] Plant | Cape: Context-aware private embeddings for private language learning[END_REF] propose an adversarial-driven DP training mechanism. However, they do not consider fairness, whereas we focus on enforcing both fairness and privacy. Moreover, their method has the same incorrect analysis as [START_REF] Lyu | Differentially private representation for NLP: formal guar-antee and an empirical study on privacy and fairness[END_REF].

Experiments

Recall that we are interested in approaches that are not only accurate but also fair and private at the same time. However, these three dimensions are not independent and are not straightforwardly amenable to a single evaluation metric. Thus, we present experiments aiming at (i) showcasing the privacy-fairness-accuracy tradeoffs of different approaches and then (ii) analyzing privacy-accuracy and fairness-accuracy tradeoffs separately. We begin by describing the datasets and the metrics.

Datasets. We consider 4 different datasets: (i) Twitter Sentiment (Blodgett et al., 2016) consists of 200k tweets annotated with a binary sentiment label and a binary "race" attribute corresponding to African American English (AAE) vs. Standard American English (SAE) speakers; (ii) Bias in Bios (De-Arteaga et al., 2019) consists of 393,423 textual biographies annotated with an occupation label (28 classes) and a binary gender attribute; (iii) CelebA [START_REF] Liu | Deep learning face attributes in the wild[END_REF] is a binary classification dataset with a binary sensitive attribute (gender); (iv) Adult Income [START_REF] Kohavi | Scaling up the accuracy of naivebayes classifiers: A decision-tree hybrid[END_REF] consists of 48,842 instances with binary sensitive at-tribute (gender). Our setup for the first two dataset is similar to [START_REF] Ravfogel | Null it out: Guarding protected attributes by iterative nullspace projection[END_REF] and [START_REF] Han | Diverse adversaries for mitigating bias in training[END_REF]. Appendix D.2 provides detailed description of these datasets, including sizes, pre-processing, and the challenges they pose to privacy and fairness tasks. Due to lack of space, results and analyses for Adult Income and CelebA dataset are given in Appendix D.5, but note that they exhibit similar trends. The preprocessed versions of the datasets can be downloaded from this anonymized URL. 2Fairness metrics. For Twitter Sentiment we report the True Positive Rate Gap (TPR-gap), which measures the true positive rate difference between the two sensitive groups (gender/race) and is closely related to the notion of equal opportunity. Formally, denoting by y ∈ {0, 1} the ground truth binary label, ŷ the predicted label and z ∈ {g, ¬g} the sensitive attribute, TPR-gap is defined as: TPR-gap = P g (ŷ = 1|y = 1)-P ¬g (ŷ = 1|y = 1).

For Bias in Bios, which has 28 classes, we follow [START_REF] Romanov | What's in a name? Reducing bias in bios without access to protected attributes[END_REF] and report the root mean square of TPR-gaps (GRMS) over all occupations y ∈ O to obtain a single number:

GRMS = (1/|O|) y∈O (TPR-gap y ) 2 . (5)
Privacy metrics. We report two metrics for privacy: (i) Leakage: the accuracy of a two-layer classifier which predicts the sensitive attribute from the encoded representation, and (ii) Minimum Description Length (MDL) [START_REF] Voita | Informationtheoretic probing with minimum description length[END_REF], which quantifies the amount of "effort" required by such a classifier to achieve a certain accuracy. A higher MDL means that it is more difficult to retrieve the sensitive attribute from the representation. The metric depends on the dataset and the representation dimension, and thus cannot be compared across different datasets. We provide more details about these metrics in Sec. D.1.

Methods and model architectures. We compare FEDERATE to the following methods: (i) Adversarial implements standard adversarial learning [START_REF] Li | Towards robust and privacy-preserving text representations[END_REF], which is equivalent to our approach without the priv layer, (ii) Adversarial + Multiple [START_REF] Han | Diverse adversaries for mitigating bias in training[END_REF] implements multiple adversaries, (iii) INLP [START_REF] Ravfogel | Null it out: Guarding protected attributes by iterative nullspace projection[END_REF]) is a subspace projection approach, and (iv) Noise learns DP text When RT is increased, we select models with potentially lower accuracy on the validation set but are more fair (lower TPR-gap). Our approach FEDERATE consistently achieves better accuracy-fairness-privacy trade-offs than its competitors across all RTs.

representations as proposed by [START_REF] Lyu | Differentially private representation for NLP: formal guar-antee and an empirical study on privacy and fairness[END_REF] but with corrected privacy analysis: this corresponds to our approach without the adversarial component. These methods have been described in details in Section 4 and their hyperparametrs in Appendix D.4. We also report the performance of two simple baselines: Random simply predicts a random label, and Unconstrained optimizes the classification performance without special consideration for privacy or fairness.

To provide a fair comparison, all methods use the same architecture for the encoder, the classifier and (when applicable) the adversarial branches. In order to evaluate across varying model complexities, we employ different architectures for the different datasets. For Twitter Sentiment, we follow the architecture employed by [START_REF] Han | Diverse adversaries for mitigating bias in training[END_REF], while for Bias in Bios we use a deeper architecture. The exact architecture, hyperparameters, and their tuning details are provided in Appendix D.3-D.4. We implement FEDERATE in PyTorch [START_REF] Paszke | Pytorch: An imperative style, high-performance deep learning library[END_REF]. Our implementation, training, and evaluation scripts are available here. 3

Accuracy-Fairness-Privacy Trade-off

In this first set of experiments, we explore the tridimensional trade-off between accuracy, fairness, and privacy and the inherent tension between them. These metrics are potentially all equally important and represent different information on different scales. Thus, they cannot be trivially combined into a single metric. Moreover, this trade-off is influenced by the choice of method but also some of its hyperparameters (e.g., the value of and λ in our approach). Previous studies [START_REF] Han | Diverse adversaries for mitigating bias in training[END_REF][START_REF] Lyu | Differentially private representation for NLP: formal guar-antee and an empirical study on privacy and fairness[END_REF] essentially selected hyperparameter values that maximize validation accu-3 The work-in-progress version of the codebase is currently available at https://github.com/saist1993/ DPNLP.

racy, which may lead to undesirable or suboptimal trade-offs. For instance, we found that this strategy does not always induce a fairer model than the Unconstrained baseline, and that it is often possible to obtain significantly more fair models at a negligible cost in accuracy. Based on these observations, we propose to use a Relaxation Threshold (RT): instead of selecting the hyperparameters with highest validation accuracy α * , we consider all models with accuracy in the range [α * -RT, α * ].

We then select the hyperparameters with best fairness score within that range. 4Figure 2 presents the (validation) accuracy, fairness and privacy scores related to different RT for each method on Twitter Sentiment. The first thing to note is that FEDERATE achieves the best fairness and privacy results with accuracy higher or comparable to competing approaches. We also observe that setting RT= 0.0 (i.e., choosing the model with highest validation accuracy) leads to a significantly more unfair model in all approaches, while fairness generally improves with increasing RT. This improvement comes at a negligible or small cost in accuracy. In terms of privacy, we find no significant differences across RTs.

We now showcase detailed results with RT fixed to 1.0 which is found to provide good trade-offs for all approaches in Figure 2, see Table 1a for Twitter Sentiment and Table 1b for Bias in Bios (and Appendix D.6 for additional results). For both datasets, we observe that all adversarial approaches induce a fairer model than Unconstrained or Noise, with FEDERATE performing best. In terms of accuracy, all adversarial approaches perform similarly on Twitter Sentiment. Interestingly, they achieve higher accu- Table 1: Test results on (a) Twitter Sentiment, and (b) Bias in Bios with fixed Relaxation Threshold of 1.0. Fairness is measured with TPR-Gap or GRMS (lower is better), while privacy is measured by Leakage (lower is better) and MDL (higher is better). The MDL achieved by Random gives an upper bound for that particular dataset. Results have been averaged over 5 different seeds. Our proposed FEDERATE approach is the only method which achieves high levels of both fairness and privacy while maintaining competitive accuracy.

racy than Unconstrained. We attribute this to a significant mismatch in the train and test distribution due to class imbalance. On Bias in Bios, we observe a small drop in accuracy of our proposed approach in comparison to Adversarial, albeit with a corresponding gain in fairness. We hypothesize that this is due to the choice of possible hyperparameters for FEDERATE (we did not consider very large values of which would recover Adversarial), meaning that FEDERATE pushes for more fairness (and privacy) at a potential cost of some accuracy. We explore the pairwise trade-offs (fairness-accuracy and privacy-accuracy) in more details in Section 5.2.

In terms of both privacy metrics, FEDERATE significantly outperforms all adversarial methods on both datasets. In fact, in line with previous studies [START_REF] Han | Diverse adversaries for mitigating bias in training[END_REF], the leakage and MDL of purely adversarial methods are similar to that of Unconstrained. On both datasets, Noise achieves slightly weaker privacy than FEDERATE with much worse accuracy and fairness.

FEDERATE also consistently outperforms INLP in all dimensions.

In summary, the results show that FEDERATE stands out as the only approach that can simultaneously induce a fairer model and make its representation private while maintaining high accuracy. Furthermore, these results empirically demonstrate that our measures of privacy and fairness are indeed compatible with one another and can even reinforce each other.

Pairwise Trade-offs

In the previous experiments, we explored the tridimensional trade-off and found FEDERATE to attain better trade-offs than all other methods. Here, we take a closer look at the pairwise fairnessaccuracy and privacy-accuracy trade-offs separately. We find that FEDERATE outperforms the Adversarial and Noise approach in their corresponding dimension, suggesting that FEDERATE is a better choice even for bidimensional trade-offs. This experiment also validates the superiority of combining adversarial learning and DP over using either approach alone.

Fairness-accuracy trade-off. We plot best validation fairness scores over different accuracy intervals for the two datasets in Figure 3. The interval is denoted by its mean accuracy (i.e., [71.5, 72.5] is represented by 72). We then find the corresponding best fairness score for the interval. We observe:

• Better fairness-accuracy trade-off: FEDERATE provides better fairness than the Adversarial approach for almost all accuracy intervals. In the case of Bias in Bios, Adversarial is able to achieve higher accuracy (albeit with a loss in fairness). We note that this high accuracy regime can be matched by FEDERATE with a larger .

• Smoother fairness-accuracy trade-off: Interestingly, FEDERATE enables a smoother exploration of the accuracy-fairness trade-off space than Adversarial. As adversarial models are notoriously difficult to train, this suggests that the introduction of DP noise has a stabilizing effect on the training dynamics of the adversarial component.

Privacy-accuracy trade-off. We plot privacy and accuracy with respect to , the parameter controlling the theoretical privacy level in Figure 4.

In general, the value of correlates well with the empirical leakage. On Bias in Bios, FEDERATE and Noise are comparable in both accuracy and privacy. However, for Twitter Sentiment, our approach outperforms Noise in both accuracy and privacy for every . We hypothesize this difference in the accuracy to be a case of mismatch between train-test split, suggesting FEDERATE to be more robust to these distributional shifts. These observations suggest that FEDERATE either improves upon Noise in privacy-accuracy tradeoff or remains comparable. For completeness, we also present the same results as a table in Appendix D.6.

Conclusion and Perspectives

We proposed a DP-driven adversarial learning approach for NLP. Through our experiments, we showed that our method simultaneously induces private representations and fair models, with a mutually reinforcing effect between privacy and fairness. We also find that our approach improves upon competitors on each dimension separately. While we focused on privatizing sensitive attributes like race or gender, our approach can be used to remove other types of unwanted information from text representations, such as tenses or POS tag information, which might not be relevant for certain NLP tasks.

A possible limitation of this work is that it not tailored to a specific definition of fairness like equal odds. Instead, it enforces fairness by removing certain protected information, which can correlate with specific fairness notions. Similarly, we do not provide any formal fairness guarantees for our method, as we do for privacy. In the future, we aim to investigate fairness methods that explicitly optimize for a specific fairness definition and explore other privacy threats (e.g., reconstruction attacks).

We provide the pseudo-code of the training procedure of FEDERATE in Algorithm 1. Note that the combination of Steps 2-3-4 corresponds to E priv in Sec. 3.

B Proof of Theorem 1

Proof. We start by proving that our noisy encoder

E priv : X → R D satisfies -DP. Recall that for any input text x ∈ X E priv (x) = priv • E(x) = E(x)/ E(x) 1 + ,
where each entry of ∈ R D is sampled independently from Lap( 2), the centered Laplace distribution with scale 2/ . Let Ẽ(x) = E(x)/ E(x) 1 . The L1 sensitivity of Ẽ is

∆ Ẽ = max x,x ∈X Ẽ(x) -Ẽ(x) 1 .
Since for any x ∈ X we have Ẽ(x) 1 = 1, the triangle inequality gives ∆ Ẽ ≤ 2. The -DP guarantee then follows from the application of the Laplace mechanism [START_REF] Dwork | Calibrating noise to sensitivity in private data analysis[END_REF].

Pr[E priv (x) = e] Pr[E priv (x ) = e] = D d=1 p(e d -xd ) p(e d -xd ) (6) = D d=1 e -2 |e d -x d | e -2 |e d -x d | = e 2 D d=1 |e d -x d |-|e d -x d | ≤ e 2 D d=1 |x d -x d | (7) = e 2 x-x 1 ≤ e 2 ∆ Ẽ = e , (8)
where (6) follows from the independence of the noise across dimensions, (7) uses the triangle inequality, and (8) from the definition of ∆ Ẽ and the fact that ∆ Ẽ ≤ 2 as shown above.

The above inequality shows that E priv satisfies -DP as per Definition 1. The fact that C • E priv also satisfies -DP follows from the post-processing property of DP, which ensures that the composition of any function with an -DP algorithm also satisfies -DP [START_REF] Dwork | The algorithmic foundations of differential privacy[END_REF].

C Error in Privacy Analysis of Previous Work

As briefly mentioned in Section 4, we found a critical error in the differential privacy analysis made in previous work by [START_REF] Lyu | Differentially private representation for NLP: formal guar-antee and an empirical study on privacy and fairness[END_REF]. This error is then reproduced in subsequent work by [START_REF] Plant | Cape: Context-aware private embeddings for private language learning[END_REF]. In this section, we explain this error and its consequences for the formal privacy guarantees of these methods, and provide a correction.

Recall from Section 2 that to achieve -DP with the Laplace mechanism, one must calibrate the scale of the Laplace noise needed to the L1 sensitivity of the encoded representation (see Eq. 2). This sensitivity bounds the worst-case change in L1 norm for any two arbitrary encoded user inputs x and x of dimension D.

In order to bound the L1 sensitivity, Lyu et al. ( 2020) and [START_REF] Plant | Cape: Context-aware private embeddings for private language learning[END_REF] propose to bound each entry of the encoded input x ∈ R D in the [0, 1] range. Specifically, they normalize as follows:

x ← x -min(x)/(max(x) -min(x)),

where min(x) and max(x) are respectively the minimum and maximum values in the vector x. [START_REF] Lyu | Differentially private representation for NLP: formal guar-antee and an empirical study on privacy and fairness[END_REF] and [START_REF] Plant | Cape: Context-aware private embeddings for private language learning[END_REF] incorrectly claim that this allows to bound the L1 sensitivity by 1 and thus add Laplace noise of scale 1 . In fact, the sensitivity can be as large as D, as can be seen by considering the two inputs x = [0, 1, . . . , 1] D and x = [1, 0, . . . , 0] for which xx 1 = D. Therefore, to achieve -DP, the scale of the Laplace noise should be D (i.e., D times larger than what the authors use). As a consequence, the differential privacy provided by their method are D times worse than claimed by [START_REF] Lyu | Differentially private representation for NLP: formal guar-antee and an empirical study on privacy and fairness[END_REF] and [START_REF] Plant | Cape: Context-aware private embeddings for private language learning[END_REF]: the values they report should be multiplied by D, which leads to essentially void privacy guarantees. While [START_REF] Lyu | Differentially private representation for NLP: formal guar-antee and an empirical study on privacy and fairness[END_REF] claim to follow the approach of [START_REF] Shokri | Privacypreserving deep learning[END_REF], they missed the fact that [START_REF] Shokri | Privacypreserving deep learning[END_REF] do account for multiple dimensions by scaling the noise to the number of entries (denoted by c in their paper) that are submitted to the server, see Algorithm 1: Training procedure of FEDERATE (one epoch).

Input: Model architecture composed of encoder E (parameterized by θ E ), classifier C (parameterized by θ C ), adversary A (parameterized by θ A ), loss function L Output: Trained model Data: Samples S={x i , y i , z i } m i=1 where x i is the input text, y i is the task label, and z i is the sensitive attribute. 1 for i ← 0 to m do // For each sample in the dataset. This can be batch too.

2 Encode: x i ← E(x i ) 3 Normalize: x i ← x i x i 1 4 Privatize: x i priv ← x i +
, where each entry of the vector ∈ R D is sampled independently from a centered Laplace distribution with scale 2 5 Adversarial prediction: ẑi ← A(x i priv )

6

Update θ A by backpropagating the loss L(z i , ẑi )

7
Task classification: ŷi ← C(x i priv )

8

Update θ E and θ C by backpropagating the loss

L(y i , ŷi ) -λ • L(z i , ẑi )
pseudo-code in Figure 12 of [START_REF] Shokri | Privacypreserving deep learning[END_REF].

In contrast to [START_REF] Lyu | Differentially private representation for NLP: formal guar-antee and an empirical study on privacy and fairness[END_REF] and [START_REF] Plant | Cape: Context-aware private embeddings for private language learning[END_REF], our normalization in Eq. 3 guarantees by design that the L1 sensitivity is bounded by 2. We provide a complete and self-contained proof of our privacy guarantees in Section B.

D Experiments

This section gives more information on the experimental setup and also provides additional results.

D.1 Privacy metric

Leakage: We compute the leakage using a sklearn's MLPClassifier. We use the validation set of the original dataset as the train and the test set of the original dataset as the test.

Minimum Description Length (MDL) is a information-theoretic probing measure which captures the strength of regularity in the data. In this work, we employ the online coding approach [START_REF] Voita | Informationtheoretic probing with minimum description length[END_REF] to calculate MDL. Online coding captures the regularity by characterizing the effort required to achieve a certain level of accuracy. Here, a portion of data is transmitted to the receiver at each step, which then uses all the data in the previous steps to understand the regularity in the current step. The regularity is obtained by training the model on the previously received data and then evaluating it on the current portion of the data.

Borrowing, the terminology from [START_REF] Voita | Informationtheoretic probing with minimum description length[END_REF], consider a dataset D consisting of {(x 1 , y 1 ), • • • , (x n , y n )} pairs, where the x i 's are the data representation, and the y i 's are the task label. In our case, x i is the output of the encoder, and y i is the sensitive attribute associated with the underlying text. Following the standard information theory setting, consider a sender Alice who wants to transmit labels y 1:n = {y 1 • • • , y n } to a receiver Bob, and both of them have access to the data representation x 1:n = {x 1 • • • , x n }. In order to transmit labels y 1:n efficiently (as few bits possible), Alice encodes y 1:n using a model p(y|x). According to Shannon-Huffman code, the minimum bits required to transmit these labels losslessly is:

L p (y 1:n |x 1:n ) = - n i=1 log 2 p(y i |x i ).
In the online coding setting of MDL, the labels are transmitted in blocks of n timesteps t 0 < t 1 < • • • t n . Alice starts by encoding y 1:t 1 with a uniform code, then both Alice and Bob learn a model p θ 1 (y|x) that predicts y from x using data {(x i , y i )} t1 i=1 . Alice then uses this model to communicate the next data block y t 1 :t 2 , and both learns a new model using larger chunk of data {(x i , y i )} t2 i=1 . This continues till the whole set of labels y 1:n is transmitted. The total code length required for transmission using this setting is given as:

L online (y 1:n |x 1:n ) = t 1 log 2 C- n-1 i=1 log 2 p θ i (y t i +1:t i |x t i +1:t i ). ( 10
)
where y i ∈ {1, 2, • • • , C}. In our case, the online code length L online (y 1:n |x 1:n ) is shorter, if it is easier for probing model to perform well with fewer training instances. This implies that the sensitive information is more easily available in the encoder's representation.

We compute MDL using sklearn's MLPClassifier at timesteps corresponding to 0.1%, 0.2%, 0.4%, 0.8%, 1.6%, 3.2%, 6.25%, 12.5%, 25%, 50% and 100% of each dataset as suggested by [START_REF] Voita | Informationtheoretic probing with minimum description length[END_REF].

D.2 Datasets

Twitter Sentiment [START_REF] Su | Demographic dialectal variation in social media: A case study of African-American English[END_REF] consists of 200k tweets annotated with a binary sentiment label and a binary "race" attribute corresponding to African American English (AAE) vs. Standard American English (SAE) speakers. The initial representation of tweets are obtained from a Deepmoji encoder [START_REF] Felbo | Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm[END_REF]. The dataset is evenly balanced with respect to the four sentimentrace subgroup combinations. To create bias in the training data, we follow [START_REF] Elazar | Adversarial removal of demographic attributes from text data[END_REF] and change the race proportion in each sentiment class to have 40% AAE-happy, 10% AAEsad, 10% SAE-happy, and 40% SAE-sad. Test data remains balanced. This setup is particularly challenging regarding privacy and fairness, as the model may exploit the correlation between the protected attribute and the main class label, which is reinforced due to skewing. The mismatch between the train-test distribution is also relevant for our setup, where the system may be trained on publicly available datasets or collected via an opt-in policy and may therefore not closely resemble the test distribution. This dataset is made available for research purposes only. 5Bias in Bios [START_REF] De-Arteaga | Bias in bios: A case study of semantic representation bias in a high-stakes setting[END_REF] consists of 393,423 textual biographies annotated with an occupation label (28 classes) and a binary gender attribute. Similar to [START_REF] Ravfogel | Null it out: Guarding protected attributes by iterative nullspace projection[END_REF], we encode each biography with BERT [START_REF] Devlin | BERT: pre-training of deep bidirectional transformers for language understanding[END_REF], using the last hidden state over the CLS token. We use the same train-valid-test split as [START_REF] De-Arteaga | Bias in bios: A case study of semantic representation bias in a high-stakes setting[END_REF]. As the dataset was collected by scrapping the web, it tends to reflect common gender stereotypes and contains explicit gender indicators (e.g., pronouns), making it more challenging to prevent models from relying on these gendered words. It is also more complex than Twitter Sentiment in terms of the number of classes. Dataset is released under MIT License. 6CelebA [START_REF] Liu | Deep learning face attributes in the wild[END_REF] consists of over 200,000 images of the human face, alongside with 40 binary attributes labels describing the content of the images. Following the standard setting as described in [START_REF] Lohaus | Too relaxed to be fair[END_REF], we use 38 of these attributes as features, "Smiling" as the class label, and "Sex" as the sensitive attribute. We use 60% of the data as train, 20% as validation, and the remaining as the test split. The CelebA dataset is available for non-commercial research purposes. 7Adult Income [START_REF] Kohavi | Scaling up the accuracy of naivebayes classifiers: A decision-tree hybrid[END_REF] consists of a U.S. 1994 Census database segment and has 48842 instances with 14 features each. We apply the preprocessing as proposed by [START_REF] Wu | On convexity and bounds of fairness-aware classification[END_REF] resulting in a total of 9 features for each instance. The objective is to predict whether a given data point earns more than fifty thousand U.S. dollars or less. We consider sex (binary) as the sensitive attribute. Like CelebA, We use 60% of the data as train, 20% as validation, and the remaining as the test split. The license of the dataset is unknown, however it is commonly used in several fairness papers and is avaialbe at [START_REF] Dua | UCI machine learning repository[END_REF].

D.3 Model Architecture

Twitter Sentiment. The encoder consists of two layers with ReLU activation and a fixed dropout of 0.1. The classifier is linear, and the adversarial branch consists of three layers. We use a fixed dropout of 0.1 in all the layers with ReLU activation, apart from the last layer.

Bias in Bios. The encoder consists of three layers and a fixed dropout of 0.1. The classifier also consists of three layers, and the adversarial branch consists of two layers. We use a fixed dropout of 0.1 in all the layers with ReLU activation, apart from the last layer.

In case of Adult Income and CelebA we use the same model as for Twitter Sentiment.

D.4 Hyperparameters

For all our experiments, we use Adam optimizer with a learning rate of 0.001 and batch size of 2000. We give additional tuning details of the different methods below. A single experiment takes about 30 minutes to run on Intel Xenon CPU. We will also provide the PyTorch model description in the README of the source code for easier reproduction.

• Adversarial: We perform a grid search over λ varying it between 0.1 to 3.0 with an interval of 0.2. Moreover, following previous work [START_REF] Lample | Fader networks: Manipulating images by sliding attributes[END_REF][START_REF] Adi | To reverse the gradient or not: an empirical comparison of adversarial and multi-task learning in speech recognition[END_REF], instead of a constant λ, we increase it over the epochs using the update scheme λ i = 2/(1 + e -p i ) -1, where p i is the scaled version of the epoch number. We also experimented with increasing the λ linearly, as well as keeping it constant, but found the above update scheme to perform the best in various settings. We also use this scheme in all other adversarial approaches.

• Adversarial + Multiple: Similar to Adversarial, we vary λ between 0.1 to 3.0 with an interval of 0.2. Apart from λ, Adversarial + Multiple has an additional hyperparameter λ ort which corrresponds to the weight given to the orthogonality loss component. We vary λ ort between 0.1 and 1.0. Here, we do a simultaneous grid search over λ and λ ort resulting in 150 runs for each seed. We fix the number of the adversary to three which is the same as the original implementation by [START_REF] Han | Diverse adversaries for mitigating bias in training[END_REF].

• FEDERATE: In order to have comparable number of runs to Adversarial + Multiple, we experiments with following values: 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.0, 15.0, 16.0, 20.0. Similar to above approach, we do a simultaneous grid search over λ and resulting in 150 runs for each seed.

• INLP: In the case of INLP, we always debias the representation after the penultimate classifier layer and before the final layer, which is consistent with the setting considered by the authors [START_REF] Ravfogel | Null it out: Guarding protected attributes by iterative nullspace projection[END_REF]. We also observe that this choice empirically led to the best results. We vary the number of iterations as a part of hyperparameter tuning. For Bias in Bios we vary the iterations between 15 and 45, while for Twitter Sentiment we vary between 2 to 7. We found that in case of Bias in Bios, performing less than 15 iterations resulted in the same behaviour as Unconstrained model over validation set while more than 45 iterations resulted in a random classifier. We observed the same in the Twitter Sentiment before 2 and after 7 iterations, respectively.

D.5 Extended Evaluation

Tables 2-3 present detailed results on CelebA and Adult Income dataset respectively. In terms of fairness over both the datasets, we observe that adversarial-based approaches induce a more fair model than Unconstrained or Noise, with FEDERATE outperforming all other methods. Interestingly, unlike Twitter Sentiment and Bias in Bios, all approaches have comparable accuracy, including Noise and INLP. We believe this to be the case due to these datasets being relatively more challenging than CelebA and Adult Income. As observed previously, purely adversarial-based approaches leak significantly more information than the DP-based approaches in terms of privacy. We observe that Noise and INLP performs marginally better in privacy than FEDERATE; however, they suffer significantly in the fairness metric.

In fact, they induce fairness levels which are similar to Unconstrained.

Overall, the results show FEDERATE as the only viable choice to induce a fairer model and make its representation private while maintaining comparable accuracy. These observations are in line with previous experiments described in Sec. 5.1

D.6 Additional Results

Tables 4-6 present detailed results on Twitter Sentiment with different relaxation thresholds, which were summarized in Figure 2.

Table 7 provides the detailed privacy-fairness results which were summarized in Figure 4.

Method

Figure 1 :

 1 Figure 1: Overview of our FEDERATE approach. The text input x is transformed to E(x) ∈ R D by the text encoder E. The encoded input is then made private by the privacy layer priv, which involves normalization and addition of Laplace noise. The resulting private representation E priv (x) ∈ R D is then used by the main task classifier C. It also serves as input to the adversarial layer A which is connected to the main branch via a radient reversal layer g λ . The light red boxes represent the Differentially Private Encoder (Sec. 3.1), and the light blue boxes represent the Adversarial component (Sec. 3.2).

Figure 2 :

 2 Figure2: Validation accuracy, fairness and privacy of various approaches for different relaxation threshold (RT) (see Section 5.1) on Twitter Sentiment. When RT is increased, we select models with potentially lower accuracy on the validation set but are more fair (lower TPR-gap). Our approach FEDERATE consistently achieves better accuracy-fairness-privacy trade-offs than its competitors across all RTs.

Figure 3 :

 3 Figure 3: Fairness-accuracy trade-off on Twitter Sentiment (top) and Bias in Bios (bottom). A missing point means that the accuracy interval was not found within our hyperparameter search. FEDERATE provides better fairness across most accuracy intervals in comparison to Adversarial over both datasets.

Figure 4 :

 4 Figure 4: Privacy-accuracy trade-off on Twitter Sentiment (top) and Bias in Bios (bottom), with associated values of . FEDERATE gives lower leakage and better or comparable accuracy to Noise over both datasets.

  .d.f. of Lap(2/ ). Consider two arbitrary input texts x, x ∈ X and let x = Ẽ(x) ∈ R D and x = Ẽ(x ) ∈ R D be their normalized encoded representations. Then, for any possible encoded output e = (e 1 , . . . , e D ) ∈ R D , we have:

Table 2 :

 2 Unconstrained 85.70 ± 0.21 12.25 ± 2.07 81.3 ± 0.89 67.82 ± 1.46 INLP 84.81 ± 0.47 12.69 ± 4.66 66.00 ± 1.32 100.17 ± 1.65 Noise 85.12 ± 0.47 12.49 ± 0.58 59.01 ± 0.65 103.93 ± 0.24 Test results on CelebA dataset with fixed Relaxation Threshold of 1.0. Fairness is measured by TPR-Gap (lower is better), while privacy is measured by Leakage (lower is better) and MDL (higher is better). The MDL achieved by Random gives an upper bound for that particular dataset. The results have been averaged over 5 different seeds.

		Accuracy ↑	TPR-gap ↓	Leakage ↓	MDL ↑
	Random	50.00 ± 0.00	0.00 0.00	-104.64 ± 0.11
	Adversarial	85.34 ± 0.22	7.83 ± 0.97 87.00 ± 2.22	46.61 ± 5.52
	Adversarial + Multiple 84.92 ± 0.12	5.79 ± 1.44 84.38 ± 2.07	51.11 ± 4.06
	FEDERATE	84.81 ± 0.34	2.68 ± 0.60 65.49 ± 3.48	98.53 ± 4.51
	Method	Accuracy ↑	TPR-gap ↓	Leakage ↓	MDL ↑
	Random	50.00 ± 0.00	0.00 ± 0.00	-20.15 ± 0.083
	Unconstrained	83.41 ± 0.32 12.73 ± 7.17	78.19 ± 1.0	16.38 ± 0.46
	INLP	83.11 ± 0.51	3.91 ± 2.43 74.54 ± 0.67	19.93 ± 0.35
	Noise	82.87 ± 0.37	8.01 ± 1.18 68.12 ± 0.94	19.38 ± 0.33
	Adversarial	83.14 ± 0.53	7.02 ± 3.31	78.2 ± 0.18	16.1 ± 0.36
	Adversarial + Multiple 83.14 ± 0.25	3.55 ± 2.16 81.37 ± 0.98	13.5 ± 1.09
	FEDERATE	82.29 ± 0.9	2.73 ± 2.18 70.25 ± 4.81	18.1 ± 2.79

Table 3 :

 3 Test results on Adult Income dataset with fixed Relaxation Threshold of 1.0. Fairness is measured by TPR-Gap (lower is better), while privacy is measured by Leakage (lower is better) and MDL (higher is better). The MDL achieved by Random gives an upper bound for that particular dataset. The results have been averaged over 5 different seeds.

	Method	Accuracy ↑	TPR-gap ↓	Leakage ↓
	Unconstrained	72.54 ± 0.57 27.17 ± 1.76 87.18 ± 0.32
	Noise	71.87 ± 0.56 25.14 ± 3.47 71.75 ± 2.99
	Adversarial	75.49 ± 0.71	8.47 ± 3.5 88.03 ± 0.24
	Adversarial + Multiple	75.6 ± 0.53	7.74 ± 4.17 88.01 ± 0.28
	FEDERATE	75.34 ± 0.56	5.46 ± 3.59 62.31 ± 5.69

Table 4 :

 4 Test set results on Twitter Sentiment dataset (scores averaged over 5 different seeds, RT=0.0).

	Method	Accuracy ↑	TPR-gap ↓	Leakage ↓
	Unconstrained	70.57 ± 0.98 20.68 ± 0.99 82.91 ± 1.65
	Noise	70.47 ± 0.43 19.84 ± 0.91 66.83 ± 3.32
	Adversarial	74.09 ± 1.56	3.03 ± 2.65 88.14 ± 0.18
	Adversarial + Multiple 74.44 ± 0.62	1.07 ± 0.74 87.98 ± 0.36
	FEDERATE	74.24 ± 1.25	0.89 ± 0.46 61.92 ± 5.04

Table 5 :

 5 Test set results on Twitter Sentiment dataset (scores averaged over 5 different seeds, RT=3.0).

More generally, the DP guarantee bounds the risk that any attribute of x is leaked through Epriv(x).

https://drive.google.com/uc?id= 1ZmUE-g6FmzPPbZyw3EOki7z4bpzbKGWk

We can also incorporate privacy into our hyperparameter selection strategy but, for the datasets and methods in our study, we found no significant change in Leakage across different hyperparameters.

http://slanglab.cs.umass.edu/ TwitterAAE/

https://github.com/Microsoft/biosbias

https://mmlab.ie.cuhk.edu.hk/ projects/CelebA.html

The authors would like to thank the Agence Nationale de la Recherche for funding this work under grant number ANR-19-CE23-0022, as well as the ARR reviewers for their feedback and suggestions.

Method