J D Porter

Yulia Ilchuk

Quinn Dombrowski
email: quinndombrowskiqad@stanford.edu

Linguistic Fingerprints on Translation's Lens

Keywords: digital humanities, translation studies, syntax

What happens to the language fingerprints of a work when it is translated into another language? While translation studies has often prioritized concepts of equivalence (of form and function), and of textual function, digital humanities methodologies can provide a new analytical lens onto ways that stylistic traces of a text's source language can persist in a translated text. This paper presents initial findings of a project undertaken by the Stanford Literary Lab, which has identified distinctive grammatical features in short stories that have been translated into English. While the phenomenon of "translationese" has been well established particularly in corpus translation studies, we argue that digital humanities methods can be valuable for identifying specific traits for a vision of a world atlas of literary style.

INTRODUCTION

A central question in translation studies over the past two hundred years is whether there is anything German about German when we read it in English. This question preoccupied Friedrich Schleiermacher, who in his 1813 essay "On the Different Methods of Translation" argues that a good translator of literature must "keep the tone of the language foreign," with the "aim that a foreign spirit should blow towards the reader" (79-80). What's more, this is not just some vague "foreignness", no mere sense that the text is not from around here: The experienced reader ought to be able to "distinguish between works of Greek and Roman origin, or Italian and Spanish" (80). In other words, if our copy of his essay is any good, it ought to be recognizably German even if we read it in English. As Schleiermacher has it, a well-translated library should give readers something like a world atlas of literary style and linguistic possibility.

Schleiermacher argues that translation should follow this model, and even today theories often treat the question in rather ethical terms. Lawrence Venuti, for instance, suggests that a Schleiermachian concept of "foreignization" can help a translator resist ethnocentrism, racism, and imperialism (16). We leave these questions aside, to focus not on what translators ought to do, but on what they have, in fact, done. Foreignization as an inevitable outcome of translation has been the focus of recent corpus translation studies. Investigations of "translationese," per Gellerstam and others, show that it is often possible to tell whether a text originated in its present language or was translated, using only such features as the frequencies of words, parts of speech, and syntactic structures. It seems that in many or, perhaps, most translated works of literature, computational methods can now detect the foreign spirit.

Though this presence of some general foreignness in translations is well established, the world atlas of linguistic style that Schleiermacher envisioned has yet to be produced. Until recently such a thing scarcely seemed possible. André Lefevere argued in the 1970's that the "prospect of the application of his method on a massive scale, so that all different shades of foreignness might be represented in the translations, is… clearly utopian" (67). Schleiermacher himself seems to have had a similar idea, arguing that a reader's ability to detect specific language origins will "only be possible if he is able to make massive comparisons," and that barring this "even the most educated readers can achieve only a very deficient knowledge of what is foreign" (80). Anyone familiar with debates in the Digital Humanities about scale, comparisons, and individual readings can see that Schleiermacher might as well have had "distant reading" somewhere in his subtitle. To arrive at a comprehensive account of every feature of every language rendered in every other language probably remains impossible at present, but we now have the computational tools and the digital corpora to begin the task.

In this essay we take a step towards that vision with a specific focus on literary criticism. Beginning with a brief account of the current state of corpus translation studies, we then turn to a small experiment of our own, demonstrating what may need to happen for English or comparative literature departments to get in on the action. We show how information about the traces of particular languages across the barrier of translation can serve as a "textual grid" (to borrow a phrase from Bassnet and Lefevere) that both clarifies and illuminates the ideas of global reading. Finally, we propose a method by which translators (or even authors of texts that have not been translated) could calibrate their use of the stylistic features indicative of translation from a particular language to achieve the desired effect on the reader. In short, we aim to shed a little new light on the affordances of translation1 .

I A BRIEF SURVEY OF DIGITAL APPROACHES TO TRANSLATION STUDIES

Compared with many other areas of contemporary DH, the use of quantitative methods to determine whether a text has been translated has a long history. In the 1986 essay that first popularized the term "translationese", Martin Gellerstam examines word frequencies in novels written in Swedish (for clarity's sake, we will often use "L1" to refer to a text's original language) versus those translated into Swedish from English (Swedish L2). By the early 1990s, scholars working in corpus translation studies had theorized a number of detectable "translation universals," or features that tend to be present in a translated text irrespective of the particular languages involved. Many of these features revolve around clarity, including, to use the list Mona Baker compiled in 1993, "explicitation" (i.e., preferring explicit options even where ambiguous ones are available), simplification, conventional grammaticality, and avoidance of repetition. Baker also observes that translation might exaggerate certain characteristic features of the source language (243-245) 2 . The upshot is that translated texts contain the information necessary to determine that they are translated. Whatever the translators may have intended, statistical comparisons with untranslated work in a similar genre reveal foreignization at work. More specific applications of these principles are relatively easy to come by. Sessions of Canada's parliament, for example, contain English and French in both L1 and L2 guises; Kurokawa, Goutte, and Isabelle detect whether given transcripts are translated-their program guesses correctly 85% of the time when using parts of speech (POS), and 90% when using word bigrams 3 . Santos applies Gellerstam's translationese concept in a Portuguese context, finding that, although the quality of results depends on translation quality and language proximity (e.g., Portuguese is genealogically closer to Spanish than to Russian), it is still possible to identify translatedness using grammatical features. Xiao and Yue find consistent differences between Chinese L1 texts and Chinese L2 texts across a number of features, such as sentence length and various markers of explicitation. And among the many projects focusing on texts translated into English, Olohan has demonstrated particular features that go a long way toward producing a foreignization effect, including lower frequencies of contractions and higher frequencies of the word "that" used as a complementizer. We have zeroed in on these examples mainly to highlight a diversity of genres and languages, but the list of experiments like these is long and growing.

Notwithstanding these advances, the Schleiermachean vision of a translated library that retains linguistic specificity remains a bit utopian. In the first place, experiments that identify languagespecific markers of translation are, obviously, less numerous than those which identify translatedness per se, or what we might call general foreignness. Of course, these can be deduced by reading, say, Xiao and Yue for Chinese, Santos for Portuguese, etc., but the point is that the work remains somewhat piecemeal. The application of these methods to literary analysis in particular is also rarer. When we began this experiment a few years ago, we were able to find just one paper that did what we had in mind: Lynch and Vogel successfully identified the source languages of novels translated into English from Russian, German, and French. Their findings are quite suggestive, but their experiment is limited by their tiny corpus-just five novels for each source language.

More important, we believe that literary criticism has not yet taken full advantage of the new technical findings as a context for and aid to reading and analysis. For anyone working with literature in translation, this kind of work provides invaluable context about why the form of a text is the way it is. The application of digital research methods in translation studies can generate new insights for a broad range of research questions: what is the role of the author's or translator's "fingerprint" in transferring the meaning from the source to the target language? Does the style of the original text impose constraints on the translator? Are there linguistic or textual features associated with each language that affect the way that language's texts work in English? Whichever view one adopts in these discussions, it is clear that translation leaves a stylistic trace on a text. Of course, literary scholars, especially within the comparatist tradition, have long thought through the effects of translation, and accordingly we want to proceed with caution and humility about the novelty of our methods here. Our hope is simply that they may prove a useful example of bringing computational corpus translation studies into the literary critical realm.

need to retain aspects of the corresponding source invariants. This causes alien forms to appear in the target language, be it structures that are never encountered in the target language, or lexical items which belong to the cognates and foreign-language loans. For an additional study of the universals proposed by Baker, see Granger 2005. 3 Even when examining single sentences, their method guesses correctly 77% of the time.

II CORPUS BUILDING

The greatest technical obstacle to literary critical engagement with corpus translation studies is the corpus itself. The development of a standard, widely used corpus of translated texts remains a topic of contention and aspiration even among translation studies specialists4 . For the sorts of questions we are considering here, we need as many texts as we can get from multiple languages, authors, and translators, and we need them to be roughly comparable across all of those vectors. That is, if we have 100 well-translated 19th-century novels written in French, we need something like 100 well-translated 19th-century novels written in Russian. The difficulties are immediately apparent-for English readers, are there 100 well-translated 19th century novels written in Russian?

Two other standard practices of literary DH raise the bar even higher, to the disadvantage of some existing corpora of translated texts. First, there is the usual attention given to the author signal. An author's works tend to be very statistically similar to each other across a number of features, including many commonly used in translation studies (especially word frequencies). For researchers studying some other phenomenon-say, the statistical features associated with being Portuguese L1-care must be taken to mitigate the author signal, often by limiting the corpus to one text per author or by having enough texts to drown out any one author5 . Second, a similar, if far less pronounced, phenomenon may apply in rare cases to translators. Rybicki shows that translators do sometimes leave faint, detectable traces on the texts they translate6 . Ideally, then, a corpus would contain only one text per author and one text per translator. However, within a specific time period, there tends to be a limited number of active translators for a given pair of languages. Permitting only one text per translator quickly narrows the pool of possible texts.

Many commonly used translation corpora are highly limited if researchers want to control for these author and translator signals. Consider the well-known Translated English Corpus (TEC; Olohan uses it in her 2001 and 2003 work). The TEC fiction corpus contains dozens of short stories. Yet within particular languages, the corpora have substantial author and translator repetition. The metadata for the Portuguese corpus, for instance, shows just nine text files; four of these are by José Saramago, and six are translated by Giovanni Pontiero. For a researcher who seeks to avoid duplicate authors and translators, there are only four usable files in total. As Olohan's work shows, the TEC has proven its quality and usefulness in other respects; the issue is simply that this is a difficult problem to tackle. Consider the example of the Russian novels again-if we allow just one appearance per author and just one Constance Garnett translation, the total stack of public domain English titles might well fit in a single backpack-and the student wearing it could probably read them all in a semester.

For these reasons, we decided to build our own corpus for this experiment. We collected short stories from six different language origins as well as a comparison corpus of L1 English stories, as detailed in Table 1 7 . For the purposes of this experiment, we did not need the texts of the stories in their original languages; we only have them in English. They come from sources like the New Yorker, a few specialized anthologies, and the Akashic Noir series, which compiles detective stories set in single cities, e.g. Buenos Aires Noir, that are typically translated into English from a language spoken in that city. The Akashic stories are useful in that they are all new, meaning the dates for originals and translations are easy to find, but they do tilt our corpus toward detective fiction. Across the corpus as a whole, the authors range from well-known, high-prestige writers (Colson Whitehead, Isabel Allende, Clarice Lispector) to fairly obscure (or, in some cases, early career) writers. Each author and translator appears only once. This corpus is still far from ideal. Some language corpora proved more difficult to put together than others; we cannot be certain that they are all equivalent when it comes to quality of translation, and some corpora have much longer stories, on average, than others. It also proved difficult to gather stories from a tight date range: our English corpus is more recent than the others, dating mostly from the 2010s; the Portuguese corpus is the oldest, with some texts ranging as far back as the turn of the 20th century. Though most texts are from the past 50 years-by the standards of literary periodization, it is fair to describe this as a mostly contemporary corpus, with a few outliers-the results should be understood in the context of this suboptimal date spread8 . Still, this corpus should prove to be useful for computational translation studies, particularly given the non-repetition of authors and translators.

III FORMS OF DISTINCTION

In simple terms, we want to find features of our texts that distinguish them on the basis of their original language. The most obvious place to look is at the words. Unfortunately, if we let every word in, they distinguish the texts so well and in such obvious form that the results are difficult to use. For instance, we ran a Most Distinctive Words (MDW) analysis of our corpus, looking for words that show up statistically significantly more often in each corpus than they do in the English L1 corpus 9 . Some of the results are thought-provoking, if potentially side effects of the fairly small corpus. For instance, why do Spanish, Portuguese, and Russian (but not Korean, Italian, or Indonesian) L1 texts use "love" more often than English? Yet more often, we find things like the highly distinctive uses of "petersburg" and "moscow" in Russian L1 stories, of "euros" in Italian L1 and "won" in Korean L1, and even of "indonesian" in stories originally written in Indonesian. Analysis of this kind seems to lead us fruitlessly into sociology, elevating differences of cultural context rather than linguistic style. Although culture and language are certainly linked, our interest lies less in the "aboutness" captured by this capacious semantic analysis, and more in a kind of style that can be traced to language, no matter the currency characters use to buy biscotti or bibimbap. In other words, we are interested in the detectable linguistic traces of Russian that withstand translation; we are less interested in discovering that stories written in Russian are more apt to mention "petersburg".

To avoid the petersburg problem, we turned away from words as a whole, in two ways. First, we examined the grammar of the texts, tagging them for parts of speech (POS) as well as dependencies10 . Once again we checked these features for distinctiveness. Second, we examined stop words. Also known as function words, they are sometimes described as the alternative to "content words": these are the articles, prepositions, pronouns, conjunctions, and other workhorse words that sit at the top of most frequency lists for English texts-in our corpus, roughly one out of every 18 words is "the". We counted the frequency of every word included in the list of stop words that comes with the Natural Language Toolkit Python library and once again looked for distinctiveness by L1 corpus. To be clear, this filter still leaves a fairly high percentage of each text intact; about 49% of all words in our corpus are stop words11 .

In general, we believe that stop words are undertheorized in literary DH. They clearly bear some sort of important relationship to style. For instance, they are extremely useful in authorship attribution, and it is often possible to make reasonably good assessments of authorship using only stop words. For this to be true, they have to be doing a lot of formal work. Understanding the formal or aesthetic role of these words is quite difficult; most of the time they are virtually invisible, and it would be no small feat to construct a general theory of the aesthetics of words that seem to carry zero semantic content. Yet, in combination with grammatical features, we have found them quite useful for examining the affordances of foreignization.

III READING THE RESULTS

9 We calculate MDW by getting word scaled frequencies for each corpus, assigning an expected rate of scaled frequency based on the frequency of each word in the English L1 corpus, and using a Fisher's exact test to determine the significance of the difference between the observed and expected frequencies. For example, if the English L1 corpus has a word count of 100,000 and says "furthermore" 20 times, the expected rate of "furthermore" for all corpora is .0002. If the Portuguese L1 corpus says "furthermore" 35 times in 85,000 words, then our test would be (using the Python scipy.stats format): fisher_exact ([[a,b] ,[c,d]]) where a = 35, b = 84,965 (i.e. the number of times "furthermore" was not observed), c = 17 (or .0002 * 85,000) and d = 84,983 (the number of times "furthermore" was expected not to appear). The test returns a p-value, which describes the likelihood that the observed difference is attributable to chance. In this example, "furthermore" would appear statistically significantly more often in Portuguese, with p = .009.

In this section we focus on one finding, a confirmation of existing translation studies work, though it is an observation so small that an ordinary reader might miss it in the course of reading a story. We noticed early on that the word "that" is more common in all of the translated texts than it is in the English L1 texts (Table 2). This fact poses a substantial interpretive challenge, not just because of the usual ineffability of stop words, but because that is particularly multifunctional: It can be an adverb ("not that bad"), a determiner ("toss me that bag"), or a complementizer ("I see that your bag is bad"). The POS tags and dependency parses helped us to clarify the situation (Table 3): The word "that" is common in the translated stories across all of these uses, but the effect is especially pronounced when it comes to complementizers 12 . Narrowing the focus to this usage shows an even more pronounced effect across all of the L1 corpora. 3. Distinct uses of the word "that" as a subordinating conjunction Syntactic differences in L1 and L2 govern the variability of the translation of that. Unlike in many other languages, the use of that in English compound sentences is optional. It is optional if it is used to reproduce somebody else's speech (indirect speech); or one's own thoughts and beliefs (after verbs believe, think, know, hope, etc.); or in the constructions "to be+ adj. (sure, certain, right, important, afraid, pleased, sorry, surprised, worried)+ that" to express feelings, opinions, beliefs; or in the constructions "be+noun (belief, fact, hope, idea, possibility, suggestion, statement, claim, comment, argument)+that" to express somebody else's beliefs, opinions, attitudes.

Corpus

If we look at Russian as a specific example of usage outside of English, the complementizer that is mandatory in all kinds of compound sentences. It is associated with the pragmatics of 12 The information in this table is based on the uses of "that" tagged by Spacy as subordinating conjunctions and given the "mark" designation. Our hand tags showed that Spacy does not always correctly identify the uses we want here; nonetheless, it seems accurate far more often than not, and we thought it worth showing what a common automated tagger finds. 13 This column shows how many times the word "that" appeared, compared to the number of times we'd have expected it to appear if it was used at the same rate as in the English L1 corpus. For example, the Spanish corpus uses the word "that" 1.5 times as often as English, if we adjust for word count.

communication and reflects the speaker's attitude to the topic of communication, listener, or the utterance itself. The obligatory use of that makes Russian speech more emotional, emphatic, but also more structured. In the use of that to reproduce indirect speech ("he said that he won't come home"), the grammatical subject "he" can be omitted («он сказал, что не придет домой») because "that" ("что") can substitute it functioning as a grammatical subject. The retention of that in discursive terms also helps eliminate an ambiguity in Russian, that the double bind creates. In Russian, a sentence like "Peter said he would marry her" is ambiguous, as it remains unclear who "he" refers to -Peter or somebody else. We believe that the complementizer that partially clarifies the context.

In other words, the findings here replicate a well-known example of explicitation; Olohan's 2001 work shows the same overrepresentation of complementizer "that" in translated texts (425-426) This usage indicates a widespread translator preference for the more explicit option in a grammatical scenario that allows (and, at times, might even call) for a more ambiguous choice. In English, optional nature of the complementizer "that" means that the sentences "I know that the sky is blue" and "I know the sky is blue" are equally valid. As we have seen with Russian, however, many languages treat the complementizer as requisite. In Spanish, for instance, the word "que" must be included in the equivalent sentence, "Se que el cielo es azul." Translators have the option of eliminating the que when it turns into a that, and sometimes doing so may produce more natural or colloquial sentences, but it seems that they generally keep the word around. The reasons are not clear. It may be that, seeing the "que", they are more apt to reach for a one-to-one lexical translation, or maybe they just prefer to choose the clearest possible option (this is after all the theory behind explicitation). Schleiermacher might argue that retaining the complementizer is a subtle way to mimic the feel or a small taste of the weltanschauung of the original speaker's linguistic context.

When it comes to the text, however, there is less need for speculation. Whatever their origin story, the more frequent presence of the complementizer "that" in these translated texts is simply the fact of the matter. What is more, the uses of this stop word point clearly to a specific conceptual space. Following a hunch that developed over the course of creating many different sample sentences to explain what the complementizer form of that looks like, we set out to determine which verbs tend to govern its use. Given the complexity of the phenomenon, the best way to do this seemed to be hand-tagging-that is, reading the sentences and writing down what we thought was going on. We isolated the roughly 5,000 sentences in our corpus that, according to a Spacy parse, contained approximately the correct usage of "that", and hand-tagged six hundred of them, randomly selected14 . We then identified the lemma form of the governing verb and, following our hunch, classified it as a verb of communication (like "say" or "tell"), cognition (like "know" or "realize"), perception (like "see" or "feel") or something else15 . Of our 600 sentences, 447 turned out to have the kind of usage of "that" we wanted (many had other constructions, like "so angry that he choked" or "it is true that", and either slipped through our filters or were mistagged by Spacy). Table 4 shows the breakdown of the verbs according to our categories16 . 4. What sorts of verbs govern the complementizer "that"?

Category

These are, overwhelmingly, verbs that have to do with the relationship between an agent and some situation, whether reported, pondered, or sensed. In other words, they are verbs designed to treat propositional content; it is tempting to say that they represent facts (or at least, potential facts). The distinction between a fact and the ordinary objects of apparently transitive verbs is a bit slippery, but it is more less the difference between "a red apple" and "that the apple is red". The former is clearly not a fact so much as an object, albeit a complex one that contains multiple words and even parts of speech. The latter is a fact, and can only be subordinated to verbs that can handle facts. You can eat or throw "a red apple" but cannot eat "that the apple is red"-you can say, think, or see it.

Of course, this is really a way of restating the meaning of the relevant linguistic categories. A complementizer is designed to subordinate a clause, and a clause is a proposition, in the sense that it must have a verb and a noun. We belabor the distinction only because it appears to raise some thorny issues about the objects of belief, knowledge, discussion, and so on. As Quine says, in order to name what we believe: … we enlist a sentence as a subordinate clause. For example, we speak of the belief that Hannibal crossed the Alps, and that Neptune is a planet. We use a sentence, with "that" prefixed, as a name of the "thing" believed. Now what manner of thing is this believe thing-that Hannibal crossed the Alps? (4, emphasis in original) For Quine, the answer is basically "don't worry about it" 17 ; he thinks it is fine to consider sentences to be the basic units of belief. For our purposes, the upshot is that this seemingly miniscule side effect of different linguistic requirements-the presence or absence of a word that would be filtered out of many DH experiments-gets us into some weighty territory. If the contents of thought and communication are generated by prefixing the word 'that' to a subordinate sentence, then the empirical observation that these subordinate sentences are handled consistently and significantly differently in translated texts means that foreignization colors the treatment of a basic feature not just of language but of existence, propositions, or facts in general. If we took Quine literally, we would have to argue that translated Italian L1 texts, by prefixing the word 'that' to more subordinate sentences, have notably more "contents of belief" than English L1 texts. This obviously goes too far. We doubt that Quine would place as much importance on the "that" as on the subordinated sentence. Still, the fact that "subordinated sentences" are more clearly demarcated, separated from the subject and verb by an extra word-four additional letters and a space-is a good example of the kind of background information that we believe this kind of corpus translation studies can offer literary critics. A scholar ought to go into a reading of a story translated from Spanish with knowledge of the differences attributable to the fact that it is translated from Spanish. They form the textual grid over which a reading can be performed.

As a small example, consider the "Sophie and the Angel", written by Dora Alonso and translated by Beatriz Teleki. Sophie is a pious and lonely Catholic octagenarian who starts to receive regular visits from a beautiful angel who plays electric guitar and canasta, converses with her over lemonade, and gives her a unicorn. The erotic tension of these visits eventually grows unbearable for Sophie, and on the advice of the local priests, she apparently gets the angel to leave her alone. Alonso makes the diegetic reality of the angel ambiguous: the priests view the whole thing as alarming and unhealthy "illusions," but a twist ending describes "a long white feather of unquestionably angelic origin poking out luridly-and inexcusablyfrom beneath the sheets of her bed" ( 22). As for Sophie, "She often thought that she was living a delightful dream" (21). Or, as we have just primed ourselves to see it, she often thought a subordinate sentence beginning with the prefix "that."

If we were naive close readers, we might describe the inclusion of the optional "that" as a subtle tactic Alonso deploys to underline the propositional nature of "she was living a delightful dream," to establish that Sophie is engaging with the hypothetical nature of what is going on, as if Sophie is saying this sentence to herself. In our view, there is a slightly different sense (or, if you prefer, feel) to the phrasing "She often thought she was living a delightful dream", a bit of an elision of the space between the agent and the proposition, as though Alonso is giving us the subordinate sentence, but Sophie is experiencing its contents. It is admittedly a very small distinction, like an accent, the difference between pronouncing it "She often thought that she was living a delightful dream" and "She often thought she was living a delightful dream". Grammatically speaking, these are the same, but that does not mean readers do, or have to, experience them the same way; the existence of the option makes possible a separation. So, if Alonso includes the "that", perhaps she intends to place just a bit of distance between Sophie and the dream, "that" as a form of metacognition.

In the context of our corpus translation results, however, we have good reason to believe that Alonso did not make this choice, that in a practical sense she probably could not make this choice. Indeed, the sentence in the original Spanish is "En muchos momentos ella pensaba que vivía un delicioso sueño" (182, emphasis added). It is just as expected: Alonso includes the "que", because in Spanish a sentence like this must include it. The person who made a decision in the English text is Teleki, the translator. She would have been perfectly justified in leaving out the "that"; it might even have been a nice match for the simple, witty, light style of the story. The critic is also perfectly justified in observing the effect of the "that" in the story; everything we argued in the previous paragraph is true with respect to the English text that we have. The critic just needs to understand that the "that" is present because, in some order, 1) translated texts in general have more explicitation and specifically more complementizer "that" instances than English L1 texts, and 2) Spanish in particular requires the presence of the complementizer in sentences like these, and 3) Teleki made a choice (conscious or not) to include it. The subtle emphasis on severing agent from proposition, the little extra light on the sentential nature of objects of thought and speech, is an effect in this story, but it is also part of the textual grid of translation in general and translation from Spanish in particular 18 . These factors contextualize without erasing the formal features of the English text. A slight extra sharpness to the contents of thinking, perceiving, and saying may be one of the affordances of translation.

IV APPLICATIONS

The results from this initial study are suggestive of the potential for translators and writers to evaluate and shape the use of translationese features in their text. With support from digital tools, backed by data from larger and more nuanced corpora, translationese features could potentially take on a more significant role in shaping literary style, both for new translations and in original text meant to invoke a sense of foreign-ness.

For computationally-oriented digital humanists, there are already relatively easy-to-use tools for using the findings described here as benchmarks for comparing new text. As a proof-ofconcept, we developed a Jupyter notebook (an interface for juxtaposing executable code with human-readable text) that takes an English-language text as the input, and compares its use of 'that' to the results we identified for Italian, Portuguese, Russian, and Spanish. The resulting reports reveal traits of the text that it may be difficult for the translator or writer to see, and provide suggestions for how to increase or decrease the score for each linguistic feature. The writer or translator could, by adjusting the text and re-running the Jupyter notebook for updated scores, achieve a fine-grained level of control over the extent of translationese manifested within the text.

While Jupyter Notebooks may be a familiar, relatively "easy" environment for DH scholars who are used to working with code, they are still far outside the mainstream tool set for writers and translators. That said, both of these groups have increasingly welcomed computationally-driven tools into their workflows over the past five years, but for such tools to be successful, they must be incorporated into word processing environments. Companies such as Grammarly have built upon advances in machine learning and NLP to develop plugins for web browsers as well as Microsoft Office that offer suggestions around tone and style, in addition to spelling and grammar. PerfectIt is a similar tool offering "professional" proofreading, including locating undefined abbreviations and enforcing style manual conventions.

The large English-language data sets that serve as training data for existing commercial tools are relatively easy to obtain. To develop an effective tool for benchmarking translationese, one would first need to amass corpora much larger than the ones described here. Ideally, those corpora would reflect a range of different genres and time periods, to capture, for instance, differences in style between 19th century crime fiction and 20th century magical realism. At the same time, more variation and nuance inevitably leads to smaller corpora, potentially to a point where the "benchmark" for a particular combination of genre and language is just the sum of the noise caused by a small set of peculiar exemplars. Nonetheless, developing the kind of corpora necessary to develop a tool usable for writers and translators to modulate their use of translationese features is increasingly feasible. Based in the United States, HathiTrust provides access to scanned, OCRed versions of millions of books, including a large number of literary texts in translation. Texts that are still protected by copyright may be accessible for computational analysis via the HathiTrust "data capsule". Outputs of this analysis are vetted to ensure that they honor agreements about what kinds of derivatives can be extracted from the text, but calculations similar to those described here (relating to frequency of syntactic of our Spanish L1 stories is subtitled The Magic and the Real). It may just be a coincidence. Still, as many magical realists could tell you, coincidences count.

structures, rather than content) should be permissible. To expand beyond the strictly-literary, there are also large shared-interest communities online who post translations of other forms of cultural production, such as song lyrics and fan fiction. These could potentially serve as another source of data for translationese features and frequency when developing a tool for writers and translators.

Conclusion

In this paper we have laid out a very ambitious scheme for a world atlas of the features of translation as such, as well as decidedly modest examples of the kinds of findings such an atlas might contain, and the interpretive possibilities it may generate. The first goal is, of course, far beyond the scope of any one paper. It may even be impossible. Still, the advances in libraries, linguistics and NLP departments, and translation studies and DH labs have made the project seem a lot less utopian than it did when Lefevere levied that critique. We want to emphasize that the digital findings of this paper have largely replicated the work already done by specialists in the more technical side of this field (broadly conceived). Confirmation has value, but the point is that we have not attempted to overthrow any existing wisdom on that front. Rather, we believe that findings like these can be deployed in new and exciting ways.

Translators may be able to take advantage of these findings to assert greater control over how their translation reflects the source language. Moreover, these findings may open the door to extensive work by literary scholars. Knowledge of the ambiguities, limitations, quirks, and felicities of translation will open the way to more sophisticated and accurate engagement with texts from a wider variety of language origins. Ideally, it will empower more scholars to work, responsibly, in languages they don't know, drawing the world's scholarly literary language communities closer together. Those literary critics who have ideas about texts in languages where they lack fluency may find in this set of methods a way to justify and structure engaging those texts, to the benefit of all. Small findings and readings like those we have produced here may eventually add up to this utopian goal. The massive number of comparisons that Schleiermacher imagined may well tear down the distinction between we subjects and the delightful dream.

Table 1 .

 1 The corpora used in this study.

	Original Language	Number of stories	Total tokens
	English	20	106,963
	Indonesian	12	41,561
	Italian	20	73,314
	Korean	20	165,375
	Portuguese	20	74,008
	Russian	20	160,686
	Spanish	20	65,714

Table 2 .

 2 Distinct uses of the word "that"

		Times used	p-value	Observed/expected uses 13
	Spanish	946	7.28E-16	1.50
	Indonesian	553	2.90E-07	1.39
	Portuguese	957	8.90E-10	1.34
	Italian	931	1.06E-08	1.32
	Korean	2029	1.01E-13	1.28
	Russian	1682	0.014	1.08
	Corpus	Times used	p-value	Observed/expected uses
	Spanish	397	3.39E-21	2.27
	Indonesian	395	5.85E-17	2.03
	Portuguese	226	3.90E-10	2.00
	Italian	351	2.95E-10	1.71
	Korean	777	5.43E-20	1.69
	Russian	712	3.11E-15	1.59
	Table

We use the term "affordances" in the sense suggested byLevine.

The notion of "translation universals" is related to the unique language emerging at the meeting point of the source and target texts. An important characteristic of the notion of translation universals is that they are independent of the languages involved, although their expression is necessarily related to the source and target languages. Like a transfer of grammatical structures, the translation of lexical items is also governed by a

Journal of Data Mining and Digital Humanities http://jdmdh.episciences.org

Cf. Ustaszewski.

Note, however, that the choice of style in translation is not entirely motivated by the source text, because the stylistic features in the source text do not always reflect the author's conscious choices and because the author's intention is not easily discovered for the reader. The "authors' statements and suggestions could be seen as simply another source of information, like dictionaries, background reading, or the opinions of other readers" (Boase-Beier, 50). Therefore, a translator does not reconstruct what is inferred as the author's intention; their inference derives from evidence in the style of the text. In translation terms, style is not just a choice of words or syntactic structures; stylistic choices "reflect a speaker's (subjective) choice of a given conceptualization"(Tabakowska 1993:7), and thus are always a reflection of different content rather than just different expression(Leech & Short 1981:15ff.; see alsoPilkington 1996:159).

Rybicki actually emphasizes the near invisibility of translators, "an unexpected corroboration of Venuti's observation on translator's invisibility. Indeed, it is adding insult to injury". In Rybicki's tests, the author signal nearly always appears to overpower the translator signal. However, he does show at least one case where a particularly free translator leaves a trace (W.S. Kuniczak's translations of Henryk Sienkiewicz). Would this be true of someone like the prolific translator of Russian, Constance Garnett? (Joseph Brodsky supposedly once complained that "The reason English-speaking readers can barely tell the difference between Tolstoy and Dostoevsky is that they aren't reading the prose of either one. They're reading Constance Garnett." (quoted in Remnick)). We prefer to err on the side of eliminating these "translator signals".

Antonio Lenzo, Shana Hadi, and Eunji Lee were instrumental in building the corpus. Initially, we collected some stories in German and French as well, but since many of these were substantially older, we have excluded them from the analysis reported here.

When we could not find dates of initial publication, we gathered author birth and death dates.

http://jdmdh.episciences.org

We used the Spacy library for Python for both part of speech and dependency tagging.

Per corpora, the percent stop words ranges from 47.6% in English L1 to 50.7% in Spanish L1.

We were still using our French corpus at this stage of the experiment, even though we removed it in the rest of the experiments detailed in this paper. However, since the purpose of this tagging was simply to gather many examples of the complementizer "that" in action, we believe that the results are still worth reporting.

A lemma is a standardized form of a word that can appear in multiple forms, e.g. "say" for "said", "saying", "says", and so on.

The count in Table4refers to total (lemmatized) token appearances within a given category, rather than unique types (that is, a lemma like "say" is counted multiple times for the "communication" category, since it appears in many sentences).

He does not literally say "don't worry about it", although he comes close: "This, like various other philosophical questions, is better deflected than met head on" (4). Also apt is his observation that, "Foreign speakers, after all, are said to share the belief that Hannibal crossed the Alps, even when they do not understand the English sentence" (4).

It is probably saying too much to suggest that this sort of effect has an influence on the English-speakingworld's relationship to Latin American magical realism (the anthology containing the Alonso and several other


Wang Song

He Yiting

Kong Youyong

Zhu Xiaomei

Zhang Shaobo

Shao Pengfei

Jean-Louis Coatrieux

Jean-Louis Dillenseger

Li Shuo

Guanyu Yang

CPNet: Cycle Prototype Network for Weakly-supervised 3D Renal Compartments Segmentation on CT Images

.

Introduction

3D renal compartment segmentation is the process of extracting the 3D structure of renal cortex and medulla from abdominal CTA images, which has great significance on laparoscopic partial nephrectomy [START_REF] Shao | Laparoscopic partial nephrectomy with segmental renal artery clamping: technique and clinical outcomes[END_REF][START_REF] Chen | An automatic method for renal cortex segmentation on CT images[END_REF][START_REF] Shao | Precise segmental renal artery clamping under the guidance of dual-source computed tomography angiography during laparoscopic partial nephrectomy[END_REF][START_REF] Zhang | Application of a functional3-dimensional perfusion model in laparoscopic partial nephrectomy with precise segmental renal artery clamping[END_REF]. During operation, correct segmentation of renal compartments helps doctors control the proportion of nephrectomy [START_REF] Taha | Kid-Net: convolution networks for kidney vessels segmentation from CT-volumes[END_REF], reduce the loss of renal function. Post-operatively, it assists in monitoring the recovery of renal function [START_REF] Kutikov | nephrometry score: a comprehensive standardized system for quantitating renal tumor size, location and depth[END_REF][START_REF] Ficarra | Preoperative aspects and dimensions used for an anatomical (PADUA) classification of renal tumours in patients who are candidates for nephron-sparing surgery[END_REF], ultimately achieve the goal of reducing the cost of surgery, increasing the success rate of surgery, and providing patients with higher quality medical services.

Deep learning has achieved remarkable success in medical image segmentation [START_REF] Litjens | A survey on deep learning in medical image analysis[END_REF][START_REF] Wang | Interactive medical image segmentation using deep learning with image-specific fine tuning[END_REF] and have made some progress in renal tumors [9,[START_REF] Heller | The state of the art in kidney and kidney tumor segmentation in contrast-enhanced CT imaging: Results of the KiTS19 challenge[END_REF] and renal artery [START_REF] He | Dense biased networks with deep priori anatomy and hard region adaptation: Semi-supervised learning for fine renal artery segmentation[END_REF] segmentation, but deep-learning based renal compartment segmentation on CT images is a challenging task owing to the particularity of renal compartments: 1) The boundary of renal compartments is not clear. As shown in fig. 1(a), the CT values are similar between cortex, medulla and extra-renal tissues. Model will lose the ability to extract distinguishable features for compartment anatomy, therefore it is prone to over segment or under segment. 2) The structure of renal compartments is thin. As is shown in fig. 1(b), the cortex extends into the kidney structure, entangles with the medulla to form several thin structures with unstable morphology. This makes feature extractors with large receptive fields easy to lose fine-grained features. The model trained with these features is not sensitive to small structures, thus be unable to segment the small part of renal compartments. 3) The large anatomy variation and small dataset scale. As is shown in fig. 1(c), the renal medulla is divided into a number of random shapes. This anatomy varies between different kidneys, so fine annotation requires a lot of time for professional doctors, which limits the scale of the labeled dataset. Therefore, the labeled dataset cannot cover all compartment morphologies. Model is unable to learn generalized knowledge, will be sensitive to singular structures, and have poor segmentation capabilities for unseen morphological structures.

There is no automatic, label-efficient and high-accuracy renal compartment segmentation works on CT images being reported. Some semi-automatic works design image operation combined with manual annotation to achieve renal compartment segmentation [START_REF] Xiang | Cor-teXpert: A model-based method for automatic renal cortex segmentation[END_REF], requiring a lot of labor costs. Deep-learning based renal compartment segmentation methods perform segmentation automatically, but the small scale of labeled dataset seriously limits their performance [START_REF] Jackson | Deep learning renal segmentation for fully automated radiation dose estimation in unsealed source therapy[END_REF][START_REF] Li | Segmentation of renal structures for image-guided surgery[END_REF].

Therefore, we proposed an automatic and label-efficient renal compartment segmentation framework Cycle Prototype Network (CPNet), which efficiently extracts renal compartments with only a few labels. It has three innovations:

1) We proposed a Cycle Prototype Learning framework (CPL) to learn consistency for generalization. It uses labels as guidance to extract features accurately and forms regularization through a reverse process to improve the generalization of the model. Feature maps are extracted under the guidance of the support label, the obtained feature vectors of the same compartment have smaller differences, and those of different compartments are more distinguishable. Prototype vectors that represent the features of compartments will be obtained by combining feature vectors of the same compartment. Prototypes are then used as templates to segment query images and train the network, forcing the feature vector extracted by the network to aggregate to the prototype vector. The feature vector of the unclear boundary deviates further from the cluster center, thus a higher penalty will be imposed. Therefore, the network will extract more discriminative boundary features. After that, the framework uses query prediction to reversely segment support images in the reverse process. This process uses the same feature extractor and prototype space to encourage the network to extract consistent class features on different images, forming a regularization thus improves the generalization ability of the model.

2) We proposed a Bayes Weakly Supervised Module (BWSM) based on crossperiod prior knowledge to embed prior for pseudo label generation. Different renal compartments have different reactions on contrast agents, resulting in different performances on images of different periods. We take use of this prior to use CTA and CTU images, combined with network prediction, to obtain pseudo labels through Bayes optimization. The module first obtains noisy pseudo-labels from CTA and CTU images, which contain accurate location information, but noisy morphological information. Then it includes the network prediction as likelihood, which has relatively smooth morphological information but inaccurate location information. It uses prior knowledge Bayes theory to synthesize the two, and the obtained posterior probability weakens the error components in the two and forms a more accurate pseudo-label. Embed the posterior pseudo-label into the model indirectly expands the size of the training set. A larger training set can cover more possible compartment anatomy variations, forcing the model to reduce its attention to unstable spatial distribution features, thus improving the generalization ability of the model.

3) We proposed a Fine Decoding Feature Extractor (FDFE) that combines location information and detail information to extract fine-grained features. In the encoder-decoder stream, the decoder restores high-resolution image with the coordinate information recorded in the encoder, thus restores the global morphological information of the high-level feature maps. The cross-layer connection directly transmits the local detail information to the decoder, adds the detail feature lost in the encoder-decoder stream. Such structure combines global and local features, has better performance for segmentation tasks of renal compartments that focus on small volumes.

Methodology

As shown in Fig. 2, our CPNet uses a cycle prototype learning paradigm to efficiently implement weakly supervised segmentation of renal compartments. It has three related modules: a) The main CPL framework learns consistency with two processes. The forward process learns knowledge from pseudo labels to improve the robustness of the model. The reverse process achieves regularization and improves the generalization of the model. b) Our BWSM extracts prior knowledge and embeds pseudo-label into learning, thus improves the robustness of the model on images with large anatomy variation. c) Our FDFE combines the global morphological features recovered by the decoder and the local detail features passed by cross-connection transmission, thus make the resulting feature maps have sharper detail information.

CPL for consistency generalization

Advantages of CPL: 1) Stronger generalization ability. The reverse process in the framework forms regularization, forcing the extracted features to meet the consistency principle, making the network robust to the noise in labels. 2) Pay more attention to the boundary area. Our framework extracts class prototypes under the guidance of label, and then imposes high penalties on boundary feature vectors that deviate from the prototype vector so that more distinguishable boundary features can be extracted.

CPL structure for consistency regularization: As shown in Fig. 2, our framework trains an efficient feature extraction network on weakly supervised dataset, and consists of two processes: forward process and reverse process. It first uses FDFE to extract support feature x s and query feature x q from the support image i s and query image i q . In the forward process, it uses the support label y s to perform masked average pooling M ( • ) to obtain the support prototype t s = M (y s • x s ). Feature vectors in x q is classified by calculating cosine similarity CS( • ) with t s to obtain query prediction y q = CS(t s • x q ). Similarly, in the reverse process, it uses y q as query label, extracts the query prototype t q = M (y q • x q ) and predicts the support image to obtain the support prediction y s = CS(t q • x s ).

Forward and reverse learning process: We train our model through forward and reverse process. In the forward process, our model learns from query pseudo labels, so we set query loss L query to optimize the performance of our model. It is calculated between query prediction y q and query pseudo-label ŷq , and is used to measure the robustness of the model on various query images. In reverse process, our model learns consistency regularization. If the query prediction we get in forward process is accurate, reverse process will recover the correct support label with it. Therefore, the support loss L support calculated between support prediction y s and support label y s is set to measure the generalization of the model on recovering support label. Both losses are cross-entropy loss [START_REF] He | Dense biased networks with deep priori anatomy and hard region adaptation: Semi-supervised learning for fine renal artery segmentation[END_REF], the total loss of our learning process is as follows:

L total = θL query + L support ( 1
)
where θ is the query loss weight hyperparameter used to balance these losses.

BWSM for prior embedding

Advantages of BWSM: 1) Enlarges training dataset. It extracts prior knowledge from unlabeled data and embeds it into learning, which indirectly expands the scale of the training set and improves generalization. 2) Extracts accurate pseudo-labels. Prior pseudo-labels are optimized by network prediction, thus reduces the influence of noise and obtains more accurate pseudo-labels. BWSM process of pseudo label generation: As shown in Fig. 3, our BWSM has a prior knowledge extraction process and a Bayes correction process. The prior knowledge extraction process uses the different appearance of compartments in CTA and CTU images to produce a prior prediction of renal compartments. It first filters the CTA and CTU images and then subtracts them to obtain the prior feature map f q , which multiplies the kidney mask f q generated from network prediction y q to obtain the prior probability for correct and wrong predictions p correct and p wrong . The Bayes correction process combines network prediction to correct the prior pseudo-label to obtain a more accurate posterior pseudo-label. The softmax probability is used as the likelihood probability l correct and l wrong , then it is used to modify p correct and p wrong to obtain the posterior pseudo-label f correct and f wrong . The process of Bayes correction process is as follows:

         p correct =1/3 + ω p wrong =(1 -p correct )/2 f correct = pcorrect * lcorrect pcorrect * lcorrect+pwrong * lwrong f wrong = pwrong * lwrong pcorrect * lcorrect+pwrong * lwrong (2)
where ω is the prior probability difference hyperparameter to balance the influence of prior pseudo label.

FDFE for fine-grained feature extracting

Advantages of FDFE: 1) Emphasizes global morphology restoration. The decoder inherits the position information saved by the encoder thus performs spatial restoration more accurately. 2) Emphasizes the extraction of local detail features. The cross-layer connection transmits high-resolution features without downsampling, so the output of the network has sharper details. 3) Enhances segmentation on thin structures. A combination of morphology and detail information makes the output feature maps have sharp detail features. Such feature maps will make the model able to segment thin renal compartment structures.

Structure of FDFE: As shown in Fig. 2 (c), our FDFE combines global information and detail information to extract fine-grained features. Global information is restored by up-pooling in the encoder-decoder stream. Detail information is retained by skip connection between convolution blocks of the same dimension. Specifically, the encoder consists of several repeated blocks. Each block contains two 3 × 3 × 3 convolutional layers and a 2 × 2 × 2 pooling layer, each convolutional layer is followed by a group norm layer. The decoder consists of several convolutional blocks corresponding to the decoder.

Experiments and Results

Experiment settings: Our dataset is obtained by preprocessing the abdominal enhanced CT images of patients undergoing LPN surgery. The pixel size of these CT images is between 0.59 mm 2 to 0.74 mm 2 . The slice thickness is fixed at 0.75 mm, and the z-direction spacing is fixed at 0.5 mm. 60 kidney ROIs of size 160 × 160 × 200 were used in the research, half of which were used as the training set and the other half as the test set. 4 images in the training set and all 30 images in the test set are fine labeled. We trained our model for 20, 000 iterations. In each iteration, we randomly select two supporting images for two prototypes representing two renal compartments and one query image from the corresponding data set to form an episode for training. The support and query sets in the first 2, 000 iterations are all from 4 labeled training images. The source of the support set in the last 18, 000 iterations remains unchanged. The query set is taken from the remaining 26 unlabeled training images. During the test process, the support set is still extracted from 4 labeled training images, while 30 test images work as the query set aimed to be segmented.

We use the SGD optimizer with learning rate l r = 0.001, momentum of 0.9, and batchsize of 1. When using the Bayes algorithm to optimize pseudo-labels, we assign the prior probability difference hyperpatameter ω = 0.05. When using the fully-supervised data set to initialize the network, the query loss weight hyperparameter is set at θ = 1, and when the pseudo-label is introduced for training, the θ is changed to 0.1. We use mean Dice for medulla and cortex (Dice-M and Dice-C), and Average Hausdorff Distance (AHD-M, AHD-C) [START_REF] He | Dense biased networks with deep priori anatomy and hard region adaptation: Semi-supervised learning for fine renal artery segmentation[END_REF] to parametrically measure model performance. Network DICE-M(%) DICE-C(%) Avg DICE(%) AHD-M AHD-C Avg AHD SegNet [START_REF] Badrinarayanan | SegNet: a deep convolutional encoder-decoder architecture for image segmentation[END_REF] 62.5 ± 5.6 Unable -6.1 ± 1.3 Unable -U-Net [START_REF] Ronneberger | U-Net: convolutional networks for biomedical image segmentation[END_REF] 72.7 ± 8.6 2.2 ± 14.0 37.4 ± 35.4 4.1 ± 1.5 9.4 ± 2.4 6.8 ± 3.4 (P)PANet [START_REF] Wang | PANet: few-shot image semantic segmentation with prototype alignment[END_REF] 55.9 ± 9.2 56.7 ± 8.8 56.3 ± 9.0 3.7 ± 1.0 3.5 ± 1.1 3.6 ± 1.1 (P)CPNet-BWSM 58.1 ± 7.6 59.6 ± 6.8 58.9 ± 7.2 3.3 ± 0.9 3.1 ± 0.8 3.2 ± 0.9 (P)CPNet-FDFE 76.5 ± 9.0 77.0 ± 9.2 76. Comparative analysis: Our framework has the best performance compared to other methods. As shown in Tab. 1, given four labeled images, our mean Dice of cortex segmentation is 78.4, and 79.1 for medulla segmentation. Some methods achieve very low performance. SegNet and U-Net did not learn enough knowledge from the given four labeled images, so they cannot correctly segment renal compartments. Prototypical method PANet achieves Dice of 55.9 and 56.7. As shown in Fig. 4. SegNet and U-Net judge the entire kidney structure as medulla and cannot correctly segment the cortex. PANet roughly segments renal compartments, but there is serious detail loss. Our CPNet retains detail better and achieves fine segmentation.

Hyperparameter analysis: Prior probability difference ω: As shown in Fig. 5 (a), Model performance increases then decreases as ω increases and achieves the best outcome at ω = 0.01. ω = 0 means BWSM directly takes network prediction as the pseudo label, and high ω means BWSM directly takes prior probability as the pseudo label. Both of them are biased and need correction, so we set ω at a balanced point.

Data amount analysis: As shown in Fig. 5 (b), model performance increases with the increase of the number of labels. More labels contain more information, so it is obviously better for training. In order to prove our superiority under weak supervision, we set the number of labels at a relatively small amount.

Conclusion

In this article, we propose a new automatic renal compartment segmentation framework Cycle Prototype Network on 3D CT images. The main Cycle Prototype Learning framework uses labels as guidance to extract features accurately and forms regularization through a reverse process to improve generalization. In addition, by embedding the Bayes Weakly Supervised Module into the framework, it can learn from unlabeled data autonomously, which improves the generalization of the framework. A unique Fine Decoding Feature Extractor is adopted to further strengthen the framework's capability to extract fine-grained detail features. The experiment results show that this method has obtained satisfactory segmentation accuracy and has potential for clinical application.

Fig. 1 .

 1 Fig. 1. Challenges of our renal compartment segmentation. a) Unclear boundaries between renal compartments and extra-renal tissues which make model lose the ability to extract distinguishable features. b) Thin structures of renal compartments which make feature extractor lose fine-grained features. c) Anatomy variation between images which make model sensitive to singular structures.

Fig. 2 .

 2 Fig. 2. The structure of our CPNet framework a) CPL learns consistency for generalization. It forms regularization through forward and reverse processes thus enhance robustness. b) Bayes Weakly Supervised Module. It embeds prior knowledge for pseudo label generation to enhance generalization. c) Fine Decoding Feature Extractor. It focuses on fine-grained detail information thus extract feature maps with sharp detail.

Fig. 3 .

 3 Fig. 3. BWSM for prior embedding. Our BWSM combines prior knowledge and network prediction to generate precise pseudo label.

Fig. 4 .

 4 Fig.4. The visual superiority of our CPNet. Segnet and Unet only obtain the rough boundary of the whole kidney. Prototypical method PANet can roughly segment renal compartments but have serious detail loss. Our CPNet learns detail information on the four labeled images better and achieves fine segmentation. but there is serious detail loss. Our CPNet retains detail better and achieves fine segmentation.

Fig. 5 .

 5 Fig. 5. Our hyperparameter analysis. a) Model performance increases then decreases as ω increases, achieving the best performance at ω = 0.01. b) Model performance increases as the number of labeled images increases. The increase speed gradually decreases and finally stabilizes.

Table 1 .

 1 Our method achieves best result on AHD and Dice. Methods with (P) are prototypical methods.


Garcia Giani

Jérôme Roy

Ivan Pitta

Dulcinéia Abdalla

Andrea Grabe-Guimarães

Vanessa Mosqueira

Sylvain Richard

Polylactide Nanocapsules Attenuate Adverse Cardiac Cellular Effects of Lyso-7, a Pan-PPAR Agonist/Anti-Inflammatory New Thiazolidinedione


Théo Jourdan
email: theo.jourdan@insa-lyon.fr

Antoine Boutet
email: antoine.boutet@insa-lyon.fr

Carole Frindel
email: carole.frindel@creatis.insa-lyon.fr

Privacy Assessment of Federated Learning using Private Personalized Layers

Keywords: Privacy, Federated Learning, Inference Attacks

Federated Learning (FL) is a collaborative scheme to train a learning model across multiple participants without sharing data. While FL is a clear step forward towards enforcing usersâĂŹ privacy, different inference attacks have been developed. In this paper, we quantify the utility and privacy trade-off of a FL scheme using private personalized layers. While this scheme has been proposed as local adaptation to improve the accuracy of the model through local personalization, it has also the advantage to minimize the information about the model exchanged with the server. However, the privacy of such a scheme has never been quantified. Our evaluations using motion sensor dataset show that personalized layers speed up the convergence of the model and slightly improve the accuracy for all users compared to a standard FL scheme while better preventing both attribute and membership inferences compared to a FL scheme using local differential privacy.

INTRODUCTION

The development of Internet of Things (IoT) and wearables contributed to the progress of a wide range of quantified-self applications related to activity recognition. In this context, motion data extracted from connected devices (e.g., smartwatches) equipped with motion sensors (e.g., accelerometer and gyroscope) are sent to a central server which processes these data through machine learning models. According to the considered application, these learning models can classify which activity is performed by the user or compute other information such as the number of steps or burned calories.

While quantitative analysis of daily activities can bring benefits from the health perspectives [START_REF] Park | Use of Machine Learning Classifiers and Sensor Data to Detect Neurological Deficit in Stroke Patients[END_REF][START_REF] Qi | A Survey of Physical Activity Monitoring and Assessment Using Internet of Things Technology[END_REF], transferring all this data to a third-party server raises important privacy concerns. Indeed, data breaches, compromised servers or any unwanted exploitation of the data expose users to personal and sensitive information leakage such as health-related attributes [START_REF] Kröger | Privacy Implications of Accelerometer Data: A Review of Possible Inferences[END_REF].

To mitigate this risk, a Federated Learning (FL) architecture (also known as collaborative learning) has been proposed [START_REF] Mcmahan | Federated Learning of Deep Networks using Model Averaging[END_REF]. In this scheme, the personal data of the user stays locally on its device and only a learning model is exchanged with the server. Iterativelly, the server sends a model to devices, this model is trained and refined with the local data on each device. Model updates are sent to the server which aggregates them to maintain a global learning model which will be disseminates back a devices. While this iterative process works well in case of data sharing similar distribution, heterogeneity of data across user devices can severely degrade performance of standard federated averaging for ML learning applications, especially for atypical users. Indeed, one unique model cannot cope with the heterogeneity of data and provide the best utility for all users [START_REF] Boutet | DYSAN: Dynamically sanitizing motion sensor data against sensitive inferences through adversarial networks[END_REF]. To address this data heterogeneity, several local adaptation schemes have been proposed such as fine-tuning of personalized layers, multi-task learning, and knowledge distillation [START_REF] Arivazhagan | Federated Learning with Personalization Layers[END_REF][START_REF] Yu | Salvaging Federated Learning by Local Adaptation[END_REF] which depict a benefit for all participants in terms of accuracy.

While FL improves privacy by reducing the exposition of the personal data, it remains vulnerable to threats. For instance, FL is not robust to model poisoning which aims to destroy the convergence of the central model [START_REF] Bernstein | signSGD with Majority Vote is Communication Efficient And Byzantine Fault Tolerant[END_REF][START_REF] Blanchard | Machine Learning with Adversaries: Byzantine Tolerant Gradient Descent[END_REF]. Privacy leakages may also occur through membership inference attacks [START_REF] Shokri | Membership Inference Attacks Against Machine Learning Models[END_REF] which consist of inferring the presence of an individual data record in the training data, or attribute inference attack where the adversary is able to infer sensitive information about individuals [START_REF] Ganju | Property Inference Attacks on Fully Connected Neural Networks Using Permutation Invariant Representations[END_REF]. The adversary can be passive [START_REF] Xu | Subject Property Inference Attack in Collaborative Learning[END_REF] (i.e., only observing exchanges) or active [START_REF] Nasr | Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning[END_REF] (i.e., modifying the protocol), and can control users or the server. To mitigate the risks, several approaches have been proposed from using Differential Privacy locally at user level or server level [START_REF] Naseri | Toward Robustness and Privacy in Federated Learning: Experimenting with Local and Central Differential Privacy[END_REF], Homomorphic Encryption (HE) and Secure Multiparty Computation (SMC) [START_REF] Fang | Privacy Preserving Machine Learning with Homomorphic Encryption and Federated Learning[END_REF].

In this paper, we quantify the utility and privacy of a FL scheme using private personalized layers [START_REF] Arivazhagan | Federated Learning with Personalization Layers[END_REF]. In such a scheme, only the lower layers of the model (capturing coarse grain information) are exchanged with the server while the upper layers of the model (capturing fine grain information) are personalized and kept private on each user. This scheme is known to improve the accuracy of the model in presence of heterogeneous data across users. However, the privacy impact of sharing only a sub part of the model has never been measured. To assess privacy leakage of this scheme, we consider both an attribute and a membership inference attack. Evaluations have been conducted using two datasets of motion sensor data collecting in real-life conditions. Results show that FL with personalized layers speeds up the convergence compared to vanilla FL and slightly increases the activity accuracy between 1% and 5%, while decreasing the gender and the overweight inference between 10% and 20% and 15% on average for membership inference. This utility and privacy trade-off is better than a defense scheme using local differential privacy which decreases the inference of the gender and the overweight up to 12% but at the cost of the activity accuracy which reduces up to 10%. These results tend to show that minimizing the information exchanged with the server is an interesting avenue for both personalizing the model (i.e., improving accuracy) while limiting potential inferences (i.e., improving privacy).

The outline of the paper is as follows. First, background and related work are described in Section 2. The exhaustive evaluation is then presented in Section 3 before concluding in Section 4.

BACKGROUND AND RELATED WORK

In this section, we review background and related work on FL (Section 2.1) and inference attacks and mitigation schemes (Section 2.2).

Federated Learning

Deep Neural Networks are now the most effective algorithms for a lot of machine learning tasks. A new paradigm has been proposed by training Machine Learning (ML) models on the user devices, named Federated Learning (FL).

In a FL scheme following [START_REF] Mcmahan | Federated Learning of Deep Networks using Model Averaging[END_REF], participants keep their data locally on their device and exchange a model -targeting a specific learning task -with a server. The main objective of this algorithm is to iteratively train a learning model M maintained by the server by aggregating this model trained locally on each participant. At each learning round i, each client k trains its local model m k with its own data using Stochastic Gradient Descent (SGD) during several iterations j. In its synchronous version, once all the participants send their model update m to the server, this server then aggregates all these model updates using the following equations before to disseminate back this aggregated model to all devices:

M i+1 = C c=1 n c n m i+1 c ,
with n c the set of indexes of all the data points n on client c, m i+1 c the local update of a client c, calculated with the following equation:

m i+1 c = m i c -ηд i c
, with η a fixed learning rate (i.e., hyperparameter which controls the step size of the optimization) for each client and д i c the average gradient on the local data of the client c at the epoch i. Those learning rounds continue until the convergence of the central model. The development of FL highlighted other challenges [START_REF] Li | Federated Learning: Challenges, Methods, and Future Directions[END_REF] such as the heterogeneity of data across user devices leading to a degraded accuracy for less represented users. To overcome this limitation, [START_REF] Arivazhagan | Federated Learning with Personalization Layers[END_REF] studied an FL scheme using personalization layers. In this scheme, the local model on each participant is composed of lower layers trained following classical FL learning round, and upper and personalization layers trained locally and which stay private on the device and not exchanged with the server (Figure 1).

Inference Attacks and Defenses

Privacy concerns are another main challenge of FL. Even though the raw data is not shared but kept on each local device, the model updates exchanged between participants and the server can leak sensitive information. Specifically, two types of attack can be considered. Poisoning attacks from malicious users aim at preventing the convergence of the learning model [START_REF] Blanchard | Machine Learning with Adversaries: Byzantine Tolerant Gradient Descent[END_REF], or implanting a backdoor to control its behaviour [START_REF] Bagdasaryan | How To Backdoor Federated Learning[END_REF]. Inference attacks, in turn, attempt to infer sensitive information on the users through model updates exchanged during the training process. These attacks can be conducted by participants or by the server. In addition, they can be passive or active. For instance, a malicious server can infer sensitive information based on the local updated parameters w i sent by each user. To increase the observation, the server can send to users a fudged model in order to amplify the potential inference of sensitive information [START_REF] Nasr | Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning[END_REF]. There are mainly two types of inference attacks: the attribute and the membership inference attacks. Attribute inference attack consists of inferring a sensitive information of the user [START_REF] Ganju | Property Inference Attacks on Fully Connected Neural Networks Using Permutation Invariant Representations[END_REF] while membership inference attack consists of determining if a data record has been used for the training of a specific model [START_REF] Shokri | Membership Inference Attacks Against Machine Learning Models[END_REF]. To apply these attacks on a FL scheme, a supervised classification model (i.e., Random Forest) was trained to infer the sensitive information from the model's parameters shared as input.

To prevent these attacks, several approaches have been considered. Homomorphic Encryption (HE) and Secure Multiparty Computation (SMC) [START_REF] Fang | Privacy Preserving Machine Learning with Homomorphic Encryption and Federated Learning[END_REF] approaches have been adapted to FL but are difficult to apply at scale due to overhead. Local Differential Privacy (LDP) [START_REF] Naseri | Toward Robustness and Privacy in Federated Learning: Experimenting with Local and Central Differential Privacy[END_REF], in turn, consists of introducing a random perturbation on SGD during the learning phase on the user device. The method provides statistical guarantees on the inference capability of an adversary to infer information on an individual's data. A differential-private SGD can be written as:

w t +1 = w t -η t (∇l(w t , x t , y t ) + N t ),
with η t the learning rate, w the parameter that minimizes the objective function l for a data point (x,y) at time t, and N t a noise value that follows a Gaussian distribution.

EVALUATION

We exhaustively evaluate the utility and privacy of a FL scheme using private personalized layers in the context of activity recognition (details of the methodology are given Section 3.1). In this section, we show that personalized layers improves the utility (Section 3.2) and privacy (evaluated through attribute inference Section 3.3 and membership inference Section 3.4) compared to both a vanilla FL and a defense scheme using local differential privacy.

Experimental setting

System model: We consider a FL scheme using Stochastic Gradient Descent (SGD) addressing activity recognition. The learning model is based on 2 convolutional layers, and 3 fully connected layers. Only the 2 lower convolutional layers are exchanged with the server which aggregates and disseminates model updates to devices, the 2 upper fully connected layers stay private on the device and are personalized with the user data. The devices of users are considered as trusted but it is not the case of the server which is considered as an adversary trying to infer personal information of participants from their model updates.

Datasets: Two real-life condition datasets are used for the evaluation. They are both publicly available and heavily used in the literature. These datasets come from the extraction of motion sensor data during gait activities (i.e., based on step patterns) of different subjects.

• MotionSense [START_REF] Malekzadeh | Mobile Sensor Data Anonymization[END_REF] contains motion data captured from an accelerometer (i.e., acceleration and gravity) and gyroscope of an iPhone 6s kept in the front pocket at a frequency rate of 50Hz. Overall, six activities (i.e., walking, jogging, going upstairs, going downstairs, sitting and standing) have been made by 24 users during 15 trials in the same conditions and environment.

• MobiAct [START_REF] Vavoulas | The MobiAct Dataset: Recognition of Activities of Daily Living using Smartphones[END_REF] records the motion data from 58 subjects during more than 2500 trials, all captured with a smartphone also in the front pocket. This dataset includes signals recorded from the accelerometer and gyroscope of a Samsung Galaxy S3 smartphone. Nine different activities of daily living are performed by the users. We only used the trials corresponding to the same activities as MotionSense in order to do the evaluation with the exact same settings. Both datasets contain an equal number of men and women, and each activity is performed according the same conditions by all subjects. However, the walking activity is more represented than the others. For each user, we also have access to physical information (e.g., the gender, weight, height and age).

Baselines: We considered two baseline approaches to compare FL scheme using private personalized layers (FedPer) [START_REF] Arivazhagan | Federated Learning with Personalization Layers[END_REF]:

• Standard FL (Vanilla) [START_REF] Mcmahan | Federated Learning of Deep Networks using Model Averaging[END_REF] This is the most common FL scheme using SGD training on the device and average aggregation of all models at each learning round on the central server.

• Local Differential Privacy (LDP) [START_REF] Naseri | Toward Robustness and Privacy in Federated Learning: Experimenting with Local and Central Differential Privacy[END_REF] We consider an implementation based on an introduction of noise following a Gaussian distribution (N (0,0.01)) to the model updates computed through a classical learning phase (i.e., Vanilla). Evaluation metrics: We evaluated FedPer and the different baselines along both utility and privacy metrics.

• Utility: To measure the utility, we considered the accuracy of the predicted activity. More precisely, we produce a confusion matrix based on the output of the classifier and measure the number of correct predictions made by this classifier over all predictions made. The value of the accuracy ranges from 0 to 1, in which 1 corresponds to perfect accuracy. • Privacy: To assess the level of privacy, we rely on the accuracy of both the inference of sensitive attributes and the inference to be a member of the training set. These inference attacks implement the solution proposed by [START_REF] Ganju | Property Inference Attacks on Fully Connected Neural Networks Using Permutation Invariant Representations[END_REF] which leverages an invariant permutation representation of nodes at each layer to classify model updates received by the server through a random forest of 1000 trees with a maximum depth of 10. We consider the gender and the Body Mass Index (BMI) of the users as sensitive attributes. The BMI is a value defined by the weight of the user divided by the square of her height. This value allows to categorize a person as underweight, normal weight, overweight or obese. In our case we only focus on a binary classification: overweight (BMI > 25) or not (BMI < 25) for the sake of class balance. For the membership inference, the accuracy refers to the percentage of correct prediction (that a participant has been involved in the training of the model) over all predictions made. In both attacks, an accuracy of 0.5 corresponds to a random guess as our dataset is balanced. Implementation details: For each experiment, we run 10 times of 5-fold cross validation where each fold is tested based on the training of the other four. We considered 200 learning rounds and an early stopping that stops the learning process if the average test loss of the aggregated model sent locally on the user data does not decrease during 30 learning rounds. During each learning round, the training with SGD is done locally at user's level during 10 epochs. A constant learning rate is used with η = 0.001 for all the users. These parameters have been optimized independently for the tasks of activity recognition, gender and BMI inference.

Utility Evaluation

We measure the accuracy of the activity detection of FedPer and the baselines. Figure 2 reports the Cumulative Distribution Function (CDF) of this accuracy over the population of users for MotionSense and MobiAct dataset. First, results show that the local adaptation of FedPer on the upper layers sightly increases the accuracy compared to the Vanilla approach (from 1% to 7% of increase on average for MotionSense and MobiAct, respectively). Second, results show that LDP baseline degrades significantly the accuracy for both datasets (10% on average of MotionSense and 6% on average for MobiAct). Indeed, by introducing noise, the convergence of the model is greatly degraded leading to a loss of prediction for all users. This result comforts previous results [START_REF] Van Der Maaten | The Trade-Offs of Private Prediction[END_REF][START_REF] Yu | Salvaging Federated Learning by Local Adaptation[END_REF].

We also measured the convergence speed of the learning. Figure 3 depicts the accuracy of the activity detection as a function of learning rounds for FedPer and the Vanilla approach. Results show that FedPer drastically speeds up the convergence. For instance, FedPer achieves 90% of accuracy after 12 learning rounds on Mo-tionSense where the Vanilla approach achieves the same accuracy after 100 learning rounds. For MobiAct, FedPer achieves 90% of accuracy after 35 learning rounds where the Vanilla approach only reaches 86% of accuracy after 200 learning rounds. By using its personalized layers at each learning round instead of starting the learning from the aggregate model sent by the server, the accuracy increases faster. For LDP, we can observe that the noise introduced prevents the model from converging.

Privacy evaluation through attribute inference

We conducted an attribute inference attack to infer the gender and the BMI of users from their model updates sent to the server. In this attack, participants train their local model on 80% of their data. Once all the models are sent to the server, only the models from one class of the targeted attribute are aggregated (in our case, models from women for gender inference, and models from overweight users for BMI inference). Then the server sends back the aggregated model to all the users that fine-tune locally on the remaining 20% of their data (e.g., training from a model aggregating model updates from women) before returning the update to the server. The adversary then trains an RF classifier on these model updates to infer the sensitive attribute. This training exploits 80% of all the updates and the testing is done on the remaining 20%, with cross validation. Figure 4 evaluates, for both datasets, the accuracy of these both sensitive attribute inferences over the epochs of local learning. Firstly, results show that without any protection (i.e., the Vanilla approach), all sensitive attributes can be inferred with high accuracy for both datasets (e.g., around 90% of accuracy for the gender on MotionSense). FedPer reduces this accuracy between 10% and 20% according to the dataset and the sensitive attribute. Results also show that FedPer better protects users against inference attack compared to LDP regardless of dataset and sensitive attributes (from 5% to 10% of accuracy loss for MotionSense). Secondly, results show that the inference accuracy tends to increase over the epochs for all approaches. This is explained by the fact that attribute inference attack is closely related to overfitting [START_REF] Yeom | The Unintended Consequences of Overfitting: Training Data Inference Attacks[END_REF], the more the model learns on user's data, the more it adjusts the parameters to data structure and the more it may incorporate sensitive information.

Figure 5 reports the CDF of the inference accuracy over the participants. Results show that while each attribute can be inferred with high accuracy for a large part of the users, this accuracy drops for few percent of users. FedPer and LDP increase the percentage of users with a small inference accuracy.

Privacy evaluation through membership inference

Lastly, we conduct a membership inference attack to evaluate privacy. In this attack, 50% of the users follow a normal FL learning round with 80% of their data. The models are sent to the server which disseminates back the aggregated model to all the users. All of them fine-tune the aggregated model on their remaining 20% of data. The server then trains a RF to classify membership from model updates for all users (using 80% of all these updates for the training and 20% for the testing with cross validation as described in section 3.1). Figure 6 depicts the accuracy of this inference attack for both datasets and for all approaches. Similarly to the attribute inference attack, results show that the membership inference attack is more efficient on the Vanilla approach. FedPer provides the best protection compared to LDP (20% on average for MotionSense dataset). Interesting enough, FedPer depicts an accuracy close to 50% which correspond to a random guess (if the data of a specific user has been used to train the model) for MotionSense dataset.

CONCLUSION

We experimentally quantified the utility and privacy trade-off of FL using private personalized layers proposed by [START_REF] Arivazhagan | Federated Learning with Personalization Layers[END_REF] in a context of activity recognition. We consider both an attribute and a membership inference attack to measure privacy leakage. Results show that using private personalized layers provides a better utility and privacy trade-off compared to a FL vanilla approach and a defense scheme using local differential privacy. These results tend to show that minimizing the information exchanged with the server is an interesting avenue for both improving accuracy and limiting privacy leakage. To comfort these results, it would be interesting to use other quantitative privacy metrics such as average information leakage and maximum information leakage [START_REF] Du Pin Calmon | Privacy Against Statistical Inference[END_REF].

Figure 1 :

 1 Figure 1: Personalized FL approach: only the upper layers (colored in grey) are shared with the server while the personalization layers are kept private on the device.

Figure 2 :

 2 Figure 2: By personalizing upper layers of the model, FedPer sightly increases the accuracy of the activity prediction compared to a FL vanilla approach; local differential privacy, in turn, greatly degrades the accuracy.

Figure 3 :

 3 Figure 3: By using personalized layers instead of aggregated information, the learning is drastically speeds up.

Figure 4 :

 4 Figure 4: The increase of number learning epochs per user increases the accuracy of the attack on both sensitive attributes.

Figure 5 :

 5 Figure 5: FedPer and LDP increase the number of users with a small inference accuracy.

Figure 6 :

 6 Figure 6: FedPer and LDP significantly decrease the accuracy of the membership inference attack compare to Vanilla method