Mathieu D'aquin
email: mathieu.daquin@loria.fr

Finding Concept Representations in Neural Networks with Self-Organizing Maps

Keywords: CCS CONCEPTS, Computing methodologies → Artificial intelligence, Neural networks, Knowledge representation and reasoning Neural networks, conceptual representation, neuro-symbolic AI

In sufficiently complex tasks, it is expected that as a side effect of learning to solve a problem, a neural network will learn relevant abstractions of the representation of that problem. This has been confirmed in particular in machine vision where a number of works showed that correlations could be found between the activations of specific units (neurons) in a neural network and the visual concepts (textures, colors, objects) present in the image. Here, we explore the use of self-organizing maps as a way to both visually and computationally inspect how activation vectors of whole layers of neural networks correspond to neural representations of abstract concepts such as 'female person' or 'realist painter'. We experiment with multiple measures applied to those maps to assess the level of representation of a concept in a network's layer. We show that, among the measures tested, the relative entropy of the activation map for a concept compared to the map for the whole data is a suitable candidate and can be used as part of a methodology to identify and locate the neural representation of a concept, visualize it, and understand its importance in solving the prediction task at hand.

INTRODUCTION

It has now been clearly established that neural networks represent an effective, popular, and highly applicable approach to data-centric artificial intelligence, but that one of their key disadvantages is their interpretability [START_REF] Eschenbach | Transparency and the black box problem: Why we do not trust AI[END_REF]. They lack transparency in the sense that, even if one can inspect their inner working, a meaningful understanding of the relation between the (sometimes extremely large) vectors of weights and activations in the network and the conclusion being made is rarely achievable. However, some form of implicit representation of abstract notions, conceptual knowledge, is expected to exist within those vectors to support the task at hand [START_REF] Fong | Net2vec: Quantifying and explaining how concepts are encoded by filters in deep neural networks[END_REF]. For example, in a previous work [START_REF] Nikolov | Uncovering semantic bias in neural network models using a knowledge graph[END_REF], we could see that there was a strong relationship between the results of neural networks and the presence concepts extracted from a knowledge graph in the input data, showing, for example, that a network appeared to rely on knowledge of the country of origin or artistic movement of a painter to predict whether their work was exposed in major museums.

In this paper, we aim to explore a methodology to identify and locate the representation of conceptual knowledge directly within a neural network's layers, to find for example whether the concepts of 'Italian painters' or 'surealist painters' are actually present in the activation vectors of the network. In other words, we aim to propose a way to inspect those vectors so that they can be visually and computationally assessed and compared when presented with input data exemplifying different concepts. This can help to understand the importance, direct or indirect, of certain concepts in the decision and where the representation of those concepts might be located in the network. This is helpful in order to not only make the neural network more interpretable and communicable, but also to potentially identify biases in the data and in the way the network exploits the data. Indeed, in such cases, we can see biases as the use of information that should not be relevant to make a decision. By identifying a supposedly irrelevant concept that is implicitly represented in a neural network and how close the representation of such a concept is to the decision taken (i.e., to the output layer), we can therefore get an idea of the level at which that particular form of bias is present within the network.

To achieve that, we use self-organizing maps (SOMs [START_REF] Kohonen | The self-organizing map[END_REF]). SOMs are neural network architectures using competitive learning to create a low-dimensional map (generally a 2D grid) of highly dimensional data. It is often used for tasks such as clustering, with the units on the grid representing clusters organized based on their similarity. Here, we use SOMs to provide abstractions of the activation vectors of a neural network from which we expect it to be easier to spot patterns and regularities. We test a number of metrics on the constructed activation maps to identify which could be used to provide an assessment of the level of representation of a concept within a neural network layer and show how the selected measure behaves on two example neural network models: one (smaller scale) used for text classification and one (larger) used in a regression task from images. We show that our method can successfully identify the relative importance of the representation of concepts such as the ones associated with DBpedia categories, the gender of a person or their ethnicity, in the different layers of the neural networks.

RELATED WORK

This work is strongly related to the explanation and interpretation of neural networks, for which there are many techniques [START_REF] Adadi | Peeking inside the black-box: a survey on explainable artificial intelligence (XAI)[END_REF]. Some approaches use knowledge graphs (see Tiddi et al. [START_REF] Tiddi | Knowledge Graphs for Explainable Artificial Intelligence: Foundations, Applications and Challenges[END_REF] for an overview), but most explanation techniques focus on identifying the features (i.e., the part of the input) that have had the greatest impact on the output. Although this has shown to be helpful in identifying biases in the data involved in training [START_REF] Lapuschkin | Unmasking Clever Hans predictors and assessing what machines really learn[END_REF], approaches based on this idea cannot, by nature, help to identify concepts of higher level, such as the gender of the represented person. We show below that, through our method, it is possible to visualize and assess the level of representation of characteristics such as the gender, artistic movement or ethnicity of a person.

In addition to focusing on input features, many of the shortcomings of explanation and interpretation approaches come from the fact that they are external to the neural network and therefore do not really help to understand the decision process involved, only its behavior. In other words, they process the input and output of the network without looking at its inner working. For that reason, researchers have started looking at how analyzing neuron activations in a network can help to extract information about the way it reaches a conclusion, for example by processing networks of co-activations of neurons [START_REF] Vitor | Extracting knowledge from deep neural networks through graph analysis[END_REF].

Closer to the work presented here, researchers have tried to identify specific units (neurons) that appear to activate when specific 'concepts' are present in the data [START_REF] Matthew | Visualizing and understanding convolutional networks[END_REF]. While such works could find correlations between concepts and neural activations, it is expected that the neural representation of abstract concepts would receive contributions from multiple units (or filters in vision) [START_REF] Fong | Net2vec: Quantifying and explaining how concepts are encoded by filters in deep neural networks[END_REF]. In addition, most of the works in this area are focused on visual concepts: textures, patterns, or a particular type of objects present in an image, and are designed to work only on convolutional neural networks (CNNs). We aim here to be able to identify the presence of representations of more abstract concepts from all kinds of data, in all kinds of network (e.g. the artistic movement in a recurrent neural network trained on textual data about painters).

In Kim et al. [START_REF] Kim | Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav)[END_REF], although they still focus on visual concepts (e.g. 'stripes on a zebra'), the authors propose a more general approach (Testing Concept Activation Vectors, TCAV) to identifying whether 'concepts' appear in neural network. While there are many similarities between our work and TCAV, we explore here an alternative technique more suited to the visual exploration of the representation of concepts in the network. Finally, another recent work [START_REF] Krug | Visualizing Deep Neural Networks with Topographic Activation Maps[END_REF] also tested the use of SOMs (among other approaches) to visually represent activations in a network. It showed that such a representation could be suitable to inspect neural networks (already trained or during training), for the purpose mostly of visual inspection and model debugging, and without considering the way externally defined concepts could appear in those activations.

BUILDING SELF-ORGANIZING MAPS FROM NEURAL NETWORK ACTIVATIONS

In this section, we describe the proposed method for identifying and localizing representations of concepts in layers of neural networks using self-organizing maps. We start by informally introducing self-organizing maps and providing an overview of the process to build, compare, and assess SOMs from neural networks' activation vectors (activation maps). We then introduce an example, on which we will rely in the next section to test different measures of the level of representation of concepts in activation maps.

Self-Organizing Maps

SOMs are a family of neural network architectures mostly used for unsupervised learning, that is, to identify patterns and structures in potentially high-dimensional data. A typical SOM is a rectangular grid (although other shapes are possible) of units, all connected to the input. During training, the activation of each unit is a function of the similarity of the units' weight vector to the input vector. The weights are then updated following a competitive learning method in which the most similar unit (the winning neuron) is first selected, and its weights (as well as, to a lesser extent, the weights of the units in its neighborhood) are updated to increase the similarity with the input example. By iterating through the training data in this way, a self-organizing map will tend to converge towards one where hot-spots of similar data points will be grouped in particular areas of the map, and where distances on the map reflect distances in the higher-dimensional space of the input data. In this sense, SOMs can be seen as both a clustering/discretization method and as a dimension reduction method.

In the implementation of the process described in this paper, we use the library minisom1 to build SOMs from activation vectors. While minisom might not be the most efficient alternative, it is well established, robust, and highly parameterizable and, therefore, suitable for the needs of the research prototype developed as part of this research. A more efficient implementation that can, in particular, rely on GPUs will be integrated in future evolutions.

Overview of the process

An overview of the overall process for analyzing the presence of concept representations in the activations of a neural network models using SOMs is presented in Figure 1. The code used in this paper to carry out this process in two case studies is available at https://github.com/mdaquin/actsom. This process assumes the availability of a trained neural network model, of the dataset on which it was trained, and of concept labels for the elements of this dataset. In principle, there are no restrictions on the architecture of the network used, as long as activation vectors can be extracted from it. In the use cases presented below, we apply it to networks containing fully connected, convolutional, and recurrent (LSTM) layers. The process also assumes that the dataset is labeled with relevant concepts, i.e. that it is possible to create subdatasets containing examples (data points) that are representative of the concepts we aim to assess. The first step in the process is to create SOMs based on the activation vectors of each layer of the network when applied to the entire dataset. This corresponds to using as input to the SOM training process for a given layer the set of activation vectors for that layer that are extracted when running a forward pass on each example in the dataset. Each of those SOMs is trained (through the minisom library) using the typical competitive learning approach. Here, since our objective is not to fully analyze the effects of those parameters, we fixed the values of hyperparameters based on preliminary tests, using a cosine distance as the metric for activation, the 'Mexican hat' neighborhood function, a neighborhood radius (sigma) of 8, and SOM grids of 15 × 15 units. Also, for layers of more than one dimension, we specify the aggregation function to use in order to transform their activation matrices or tensors into activation vectors. In the use cases below, we aggregated using the mean of activation values for recurrent and embedding layers, and by flattening the activation tensor for convolutional layers.

We call the resulting SOMs the 'base SOMs' as they correspond to the patterns of activation in each layer when all concepts are present, and therefore will form a base for comparison. We represent those SOMs by a matrix of the activation frequency of each unit in the SOM on the entire dataset (i.e., for a given unit, the frequency at which it was the winning unit for the examples in the dataset). This matrix is represented visually as a heatmap, as visible in Figure 1, where darker tones represent higher frequencies.

Once the base SOMs are built, we build similar matrices (and heatmaps) for each of the subsets of the dataset corresponding to a concept. In practice, this simply means that, for a given layer and a given concept, we activate the base SOM corresponding to the layer for all examples of the concept (i.e., we 'populate the map') and build the corresponding frequency matrix. The idea is that those 'concept maps' can then be analyzed and compared to the corresponding base SOM (as well as to each other) to find whether specific patterns are present in the activations for a given concept. Our goal here is, in particular, to find what metrics could be used to assess a given concept map to understand how well the corresponding concept is represented in the corresponding layer.

Illustration on an Example

As a first use case and the basis for our tests, we use an example similar to that used in our previous work [START_REF] Nikolov | Uncovering semantic bias in neural network models using a knowledge graph[END_REF]. All the code to collect the dataset, the concept annotations for the dataset and to train the model (as input to the process presented above) is available in the provided online repository.

In this use case, the objective of the model is to predict, given the biography of a painter, whether this painter has at least one painting displayed in at least one major museum. Data are obtained by querying DBpedia2 for the abstracts of entities of type painter, together with the number of museums that refer to objects related to the given painter. We train a simple recurrent network made of an embedding layer, an LSTM layer, and a fully connected layer with relu as activation function in a binary classification task (i.e., with two classes: the painters with at least one painting in a major museum and the ones without). After a few trial-and-error-based tests to identify good values for hyperparameters and balancing the dataset through undersampling, we obtained a network that achieved 72.6% accuracy on the test set (20% of the balanced dataset). 3 While there is no doubt that better results could be obtained, our objective here is to understand what subconcepts of 'painters' might have been identified and represented in the network to support its predictions, which is useful even if those predictions are not very accurate (possibly even more).

Concerning the concepts to be considered, we also retrieve at the time of querying DBpedia, for each painter, their nationality, the artistic movement(s) to which they belong, and their categories (according to the DBpedia category taxonomy). This allowed us to group painters in our dataset according to concepts such as 'Italian Painters', 'Realist Painters', or 'American Women Painters', in addition to the target classes, i.e. painters with paintings in major museums and painters without.

Figure 2 shows the base SOMs produced for each of the three layers of the model, as well as the SOMs corresponding to the target class 0 (painters with no painting in major museums), to the target class 1 (painters with paintings in major museums) and to the concept of French painters (painters of nationality 'French'). When comparing those SOMs, clear differences can be seen. It is in particular visible that activations for the two target classes are substantially different in the fully connected layer (FC, the last one before the output layer in the network), but not so much in the previous layers (which is something we will use in the next section). Focusing on the concept of French painters, we can see that the heatmaps for the concept SOMs are more scattered than the base ones, which is unsurprising considering that French painters correspond to a small subset (151 painters) of the overall dataset (23.8K painters). However, we can also see that the distribution of frequencies in the units of the SOMs might show a pattern indicating that the concept of French painters has a stronger representation in the FC layer (many darker units in the lower right corner) than it does in the embedding layer (in which darker units tend to be situated where they are also darker in the base layer). Therefore, the key question here is: How to assess the level of representation of a concept in each layer more formally, beyond the visual inspection of the heatmaps?

ASSESSING THE LEVEL OF REPRESENTATION OF A CONCEPT

In this section, we show the results of testing different measures in the use case presented above to better understand which metrics could be used to assess how well a concept is represented in a given layer of a neural network. We discuss the measures selected based on how they could intuitively help identify well-represented concepts and test those measures applied to the SOMs as represented by their activation frequency matrices (as in the heatmaps in Figure 2). 4Understanding whether a measure is a good indicator of the level of representation of a concept in the activation vectors of a neural network is not trivial. To achieve this, in our tests below, we rely on a reasonable hypothesis: If we consider that the target classes of the model correspond to concepts, such concepts should be better represented in the layers closer to the output of the network than they are in those closer to the input layer. This hypothesis is based on the idea that the different layers of the network have a role to progressively abstract the examples provided so that they can be classified into the target classes in the end (in our example, painters with their work in major museums and painters without their work in major museums). This hypothesis can be visually verified in Figure 2 where the SOMs for the target classes are progressively more distinct from the corresponding base SOMs moving from the embedding layer to the FC layer, as previously mentioned.

In addition to testing, for each measure, whether they validate this hypothesis in the presented use case, we consider two requirements for those measures: 1-that they should have values in intervals that facilitate comparisons, and 2-that the evolution of their values through the different layers should be consistent across the two target classes.

Tested Measures

We consider two categories of measures here, depending on whether they apply directly to a SOM (or, more precisely, to the activation frequency matrix for a concept in a SOM) or whether they are measures comparing a concept SOM to a base SOM for the same layer.

Base measures.

Entropy: SOMs tend to group together in a given area of the map examples (in our cases, activation vectors) that are similar. Therefore, the frequency matrix for a well-represented concept would be expected to have high values concentrated within a small area of the map. Shannon's entropy is a measure of the level of uncertainty associated with the outcomes of a random variable. Here, considering the unit active in the SOM as the random variable, it would thus measure how close to a random distribution the frequency values in the concept SOM are. In order to obtain high values for high levels of representation, we use the inverse of entropy, i.e. if 𝑠 is the frequency matrix for a concept SOM represented as a vector of probabilities for each unit to be activated, and 𝐸 (𝑥) is Shannon's entropy applied to a probability distribution 𝑥, then the measure used is 1/𝐸 (𝑠).

Max FM: Another intuition on which we could rely is that if a concept is well represented by a layer, then many of the instances of the concept will activate the same unit of the SOM, which would also not activate for examples outside of the concept. We can measure how representative each unit of the SOM is of a concept by using the typical F-Measure from information retrieval. The measure used here is therefore, for a given SOM and a given concept, the maximum F-Measure, i.e. the F-Measure of the SOM unit that is most representative of the concept.

Measures comparing concept SOMs and base SOMs.

The frequency matrices for the base SOMs of each layer of the model can be considered as the average patterns throughout the dataset. It is therefore meaningful to consider measures that assess how specific concept SOMs diverge from the corresponding base SOMs to assess how activation vectors in the neural network are specific to the concept.

Distance: This measure corresponds, essentially, to the cosine distance between a concept SOM (the frequency matrix) and the corresponding base SOM.

Results

Figure 3 shows the evolution of the values of each measure as applied to the concept SOMs corresponding to the target classes of the model (museum or not museum). The values of the measures were standardized so as to bring them to the same scale (i.e. the z-score for each value is shown).

As can be seen, in the majority of cases, the expected behavior is shown: The value of the measure increases as we get closer to the output layer of the network. However, for all other measures than relative entropy, the values evolve very differently for the two concepts under consideration (C0 and C1), therefore failing to meet our consistency requirement. Relative entropy, on the other hand, is consistent and clearly increases from the embedding layer to the FC layer (named 'relu' in the figure). In addition, it ranges in this case between 0.039 and 0.52, showing that it is suitable as a measure for comparison.

It can also be noticed that there seems to be a correlation between the entropy and the max FM measure, although it is unlikely to be significant.

USE CASE: ANALYZING THE MOST REPRESENTED ARTISTIC MOVEMENTS AND CATEGORIES OF PAINTERS

To check whether the analysis of the activation SOMs as computed following the process presented above could be useful, we consider two use cases, the first of which relies on the model and dataset previously introduced to illustrate the process and test measures.

As mentioned earlier, we collected additional information about the painters from DBpedia, which was not used when training the model, including the artistic movement to which they belong and their category in the DBpedia category taxonomy. As a first test, we calculated the relative entropy of the SOMs for each layer for the artistic movements related in the dataset to more than 20 painters. The results are presented in Figure 4 in order of relative entropy for the fully connected (relu) layer. As can be seen, some concepts such as the one of 'painters of the abstract impressionism movement' appear to be significantly more represented in the relu layer than others, while there representation appears to be average in the other layers. This could possibly indicate a connection between those concepts and the decision. Indeed, as we have seen, the relu layer being the last one before the output layer, it is the one where the target classes should be highly represented. Therefore, the fact that a particular concept is highly represented in that layer might mean that this concept overlaps strongly, at least at a conceptual level, with the target classes. On the other hand, we can also here identify concepts, such as the one of 'painters in the abstract art movement' that are strongly represented in the layers closer to the input, which would indicate that the language and vocabulary used to describe those painters might be significantly different from those for the others, without this necessarily having a strong impact on the prediction (since the same concept has a relatively low representation in the relu layer). This demonstrates how looking at the level of representation of concepts in the layers of the neural network can help to better understand how the model uses (and abstracts from) the dataset to make its predictions.

Applying the same process on the categories of painters (taking those related to at least 30 painters) instead of the artistic movements leads to much more varied concepts, as seen in Figure 5. Here too we can observe that some concepts, such as 'painters from Florence' are better represented in the relu layer, potentially showing either a rule in the domain, or a bias if that rule turned out not to reflect reality (e.g. that painters from Florence tend not to have their paintings in museums). Similarly as with the artistic movement, other concepts are more presented in the initial layers, which are more focused on extracting features from the data (in this case, language features). The concepts of 'painters from Antswerp' and 'painters from New York City' for example are relatively strongly represented in the embdedding layer, indicating that the biographies of painters belonging to those concepts are likely to use a significantly different vocabulary from others. Interestingly, many of the highly represented concepts here relate to locations, as well as other personal attributes of the painters (e.g. 'American Women Painters' or 'Jewish Painters') rather than to the characteristics of their works (e.g., 'Landscape Artists'). Considering that this is a small model trained on data of rather poor quality and not reaching a particularly high accuracy, it seems consistent that predictions from that model would rely on the representation of crude concepts providing indirect and potentially biased predictors for the target classes (confirming some of the results obtained previously on a similar dataset in [START_REF] Nikolov | Uncovering semantic bias in neural network models using a knowledge graph[END_REF]).

LARGER-SCALE USE CASE: THE REPRESENTATION OF GENDER AND ETHNICITY IN PREDICTING A PERSON'S AGE FROM A PHOTO

As discussed above, our first use case is an interesting demonstration of the potential of the method presented here, as it allows us to get a better understanding of the way the prediction can be based on exploiting shortcuts and biases in the data. To test this method on a model with a different architecture and of a larger scale, we apply it on one trained to predict the age of a person from a photo. For this, we reused the ResNet18 vision model [START_REF] He | Deep residual learning for image recognition[END_REF] pretained on the ImageNet dataset, with an additional fully connected layer, and trained it on the UTKface dataset 5 . We obtained a model that achieved a 𝑅 2 score of 80%, corresponding to a mean absolute error of 6.44 years. An advantage of the UTKface dataset is that each image is also annotated with the gender and ethnicity of the person. While we do not use those annotations in training it, they allow us to test how the corresponding concepts are represented in the model. All the code to train the model, extract the SOMs and test them is available in the provided online repository.

To check the validity of our choice of the relative entropy as a measure of the level of representation of a concept, we discretized the target values into three groups (young, medium, and old) using the k-means method and tested the evolution of our four measures across the layers of the model (net.layer1, net.layer2, net.layer3, net.layer4 and net.avgpool being the successive layers of ResNet18 and relu being the added fully connected layer). As can be seen in Having verified that relative entropy remains a valid measure here, our goal is to use the proposed method to check how much the concepts corresponding to the different genders are represented and where in the network those representations are stronger. Figure 7 therefore shows the values of relative entropy for those concepts in the different layers of the network. As can be seen, the representation of the different genders and ethnicity appear to evolve consistently across the layers with some exceptions. In particular, both genders are clearly less represented in layer 2 of the ResNet18 component of the network, but the gender labeled 1 has a slightly higher representation generally in all layers, and is significantly more represented in the last (relu) layer. Similarly, the concepts corresponding to the different ethnicities have different levels of representation but follow roughly a similar trend. Ethnicity 2 is however more clearly represented in the relu layer (and therefore could represent an important concept for the prediction), while ethnicities 1.0 and 4.0 are more represented in the lower level layer net.layer1, indicating that they might be visually more distinguishable than others in the data. It is useful here also to notice that, on average, the concepts corresponding to ethnicities have a higher relative entropy across layers than the ones corresponding to gender, which seems to indicate that ethnicity is both visually and conceptually more important to the model when predicting age than gender. We, of course, cannot rule out that this might be due to biases in the datasets (e.g., different age distributions for the different ethnicities).

CONCLUSION

In this paper, we explored how building and comparing SOMs on the activation vectors of layers of neural networks can help analyze the relative level of representation of concepts such as those associated with artistic movements, location, gender or ethnicity in a neural network model. We found in particular that the relative entropy of the probability distribution of unit activation in a SOM for a given concept, compared to that distribution for the whole dataset, provides a useful measure to check how much a concept is represented within a layer. Through two use cases, we showed that this method and measure can help better understand the basis for predictions in a network and potential biases on which it might rely. In other words, in addition to providing a method to assess the presence of a neural representation of concepts at different levels of a network, our test validated the idea of using SOMs to abstract patterns of activation into structures that are computationally and visually processable. Beyond what is presented here, we therefore expect to be able to use those abstractions, the activation SOMs, as a way to explore and manipulate conceptual knowledge as represented in neural network. We are in particular looking at how the comparison of multiple SOMs for multiple concepts can help identify relations between those concepts. Another exciting way in which this work opens up new avenues for research is by using activation SOMs to identify the presence of unknown concepts in the network. Indeed, we have shown here that we can 'query' the network for the presence and importance of the representation of existing concepts. However, a key question is whether, in training to make a particular prediction, the network could have identified particular properties of the considered objects that group them into meaningful but not yet identified concepts. Being able to recognize such a phenomenon could have obvious benefits in knowledge acquisition and extraction, especially in areas where understanding the mechanisms by which a prediction is made, the concepts that the network might have 'discovered', is just as important as making the prediction itself (e.g., when applying machine learning to scientific research).

Figure 1 :

 1 Figure1: Overview of the process: Starting from a neural network trained on a given dataset, we first build SOMs based on the activation vectors of each layer of the network on that dataset. We then populate those SOMs using the activation vectors in the neural network from subsets of the dataset that correspond to specific concepts. The SOM activation matrices obtained can then be assessed and compared.

Figure 2 :

 2 Figure 2: Base SOMs, concept SOMs for the target classes and concept SOMs for painters of French nationality for each layer of the model to classify painters.

Figure 3 :

 3 Figure 3: Evolution of measures of the level of representation of concepts applied to target classes across layers of the network. 'relu' corresponds to the fully connected layer.

Figure 4 :

 4 Figure 4: Relative entropy of concepts related to the artistic movement of painters.

Figure 5 :

 5 Figure 5: Relative entropy of concepts related to the DBpedia categories of painters.

Figure 6 :

 6 Figure 6: Evolution of measures of levels of representation of concepts applied to age groups across layers of the network.

Figure 6 ,

 6 Figure 6, relative entropy continues to broadly meet the requirements expressed earlier. Interestingly, in this case, all the measures are significantly more consistent, even though they do not all evolve in the way expected (except for distance) as we move up the different layers.

Figure 7 :

 7 Figure 7: Relative entropy of concepts related to gender and ethnicity in the age prediction use case.

https://pypi.org/project/MiniSom/

https://dbpedia.org

The original dataset contained 23.8K painters, of which only 2,416 had paintings in major museums. For undersampling, we therefore randomly selected 2,416 painters who did not.

We also tested those measures on matrices corresponding to the average activation values of the SOM, i.e., the average distance of each tested example to the weight vector of the units. However, the results were either similar to the ones when using the frequency maps, or provided values ranging in very small intervals, rendering them difficult to interpret. Therefore, we omit the results for those here.

https://susanqq.github.io/UTKFace/

ACKNOWLEDGMENTS

Initial ideas and methods that led to the ones presented in this paper were explored during the internship of Maxime Haurel in the author's research team (see https://github.com/MHaurel/nnconcept-interpreter).


Xavier Mariette
email: xavier.mariette@bct.ap-hop-paris.fr

Florence Tubach

Haleh Bagheri

Michel Bardet

Jean-Marie Berthelot

Philippe Gaudin

Denis Heresbach

Antoine Martin

Thierry Schaeverbeke

Dominique Salmon

Bardet

Berthelot Jm

Lemann

Xavier Pr

Mariette

Lymphoma in patients treated with anti-TNF: results of the 3

Keywords: Anti-TNF, Lymphoma, Safety, Rheumatoid arthritis, Spondylarthropathies Lymphoma complicating anti-TNF therapy

teaching and research institutions in France or abroad, or from public or private research centers.

INTRODUCTION

The risk of lymphoma is increased in several systemic autoimmune diseases, mainly Sjögren's syndrome, systemic lupus erythematosus and rheumatoid arthritis (RA) [START_REF] Zintzaras | The risk of lymphoma development in autoimmune diseases: a meta-analysis[END_REF] . In RA, the risk of non-Hodgkin's lymphoma (NHL) is increased by twofold [START_REF] Smedby | Autoimmune and chronic inflammatory disorders and risk of non-Hodgkin lymphoma by subtype[END_REF] and that of Hodgkin's lymphoma (HL) by threefold. [START_REF] Landgren | Autoimmunity and susceptibility to Hodgkin lymphoma: a population-based case-control study in Scandinavia[END_REF] Long-lasting inflammatory activity of RA is considered the main risk factor of lymphoma by its continuous stimulation of B-cells. [START_REF] Baecklund | Association of chronic inflammation, not its treatment, with increased lymphoma risk in rheumatoid arthritis[END_REF] The effect of immunosuppressive drugs on the risk of lymphoma remains a matter of debate. To date, only the deleterious role of azathioprine has been demonstrated for both RA] [START_REF] Baecklund | Association of chronic inflammation, not its treatment, with increased lymphoma risk in rheumatoid arthritis[END_REF] and Crohn's disease (CD). [START_REF] Kandiel | Increased risk of lymphoma among inflammatory bowel disease patients treated with azathioprine and 6-mercaptopurine[END_REF] Although withdrawal of methotrexate (MTX) treatment can rarely induce regression of Epstein-Barr virus (EBV)-associated lymphoproliferation, [START_REF] Kamel | Brief report: reversible lymphomas associated with Epstein-Barr virus occurring during methotrexate therapy for rheumatoid arthritis and dermatomyositis[END_REF] most recent reports did not find any increased risk of NHL in RA patients treated with MTX. 7 8 Recent concerns about lymphoma have focused on therapy with anti-TNF drugs because of their profound immunoregulatory effect. However, anti-TNF therapy could reduce the inflammatory activity of the underlying disease, which is the main risk factor for lymphoma in RA.

In some cohorts of RA patients receiving anti-TNF therapy, the risk of lymphoma was not different than that for RA patients not receiving the therapy. 7 9-11 However, these cohort studies were underpowered to investigate a difference between anti-TNF agents in terms of risk of lymphoma.

We aimed to examine whether patients receiving anti-TNF agents have an increased lymphoma risk and to compare risks for different anti-TNF agents, described the cases of lymphoma and their outcome, and identified the risk factors of lymphoma in patients receiving anti-TNF therapy.

PATIENTS AND METHODS

The French RATIO (Research Axed on Tolerance of bIOtherapies) registry was designed by a multidisciplinary group to prospectively collect all cases of lymphoma occurring in France from February 1, 2004, to January 31, 2007, in patients who were receiving anti-TNF therapy, for whatever the indication. The design has been described elsewhere. [START_REF] Tubach | Risk of tuberculosis higher with monoclonal-antibody than with soluble-receptor anti-TNF therapy in the 3-year prospective French RATIO registry[END_REF] [START_REF] Tubach | The RATIO observatory: French registry of opportunistic infections, severe bacterial infections, and lymphomas complicating anti-TnFalpha therapy[END_REF] The reporting of this study conforms to the STROBE statement. [START_REF] Von Elm | The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies[END_REF]

Identification and validation of lymphoma cases

All cases reported to the 31 French pharmaco-vigilance regional centers of Agence Française de Sécurité Sanitaire des Produits de Santé (AFSSAPS), or pointed out directly to the companies commercializing anti-TNF were collected. In addition, physicians from all the different French hospital centers implied either in the prescription of TNF blockers (i.e. rheumatology, internal medicine, gastroenterology and dermatology departments) and/or in the management of lymphomas (i.e. hematology or oncology centers), were directly required to report each newly diagnosed case). A direct mail reminder 4 times a year and several communications at congresses or in specialized press encouraged them to report cases.

Validation of cases:

Included in the RATIO registry were all cases (from all sources) with a validated diagnosis of lymphoma according to the International Classification of Diseases for Oncology (categories 9590-9599, 9650-9660, 9670-9680, 9690-9699, 9700-9709, 9710-9719). An expert committee involving 3 experts of lymphoma (XM, OH, MR) validated cases by consensus on the basis of the detailed standardized case report form and additional documents (hospitalization summary, histological results or others). The biopsy specimens of all validated cases were reviewed by the same hematopathologist (MR), to validate the diagnosis obtained by histopathology.

In addition, this hematopathologist assessed all biopsy specimens for presence of EBV, detected by Eber in situ hybridization.

Risk of lymphoma for patients receiving anti-TNF therapy

A case-control study was performed..

Cases:

Cases were all validated cases of lymphoma in the RATIO registry with a labeling indication for use of anti-TNF treatment (i.e. RA, spondylarthropathy [SpA;

AS or psoriatic arthritis], UC or CD, or psoriasis).

Controls: Lymphoma-free patients receiving anti-TNF treatment in a labeling indication were included from centers participating in the RATIO registry (thus from the same population source) in a global pool of controls. From that pool, we randomly selected patients for a database of controls reflecting the proportion of patients receiving each of the three anti-TNF drug in France. Two controls per case were randomly matched by sex, age (within 5 years) and underlying inflammatory disease from this database of controls. We also used a second sample of controls randomly selected from the same database of controls, with the same matching criteria (second matching).

Incidence study

Incidence of lymphoma

We estimated the annual incidence rate of lymphoma in patients treated with anti-TNF therapy, adjusted for age and sex, with the French population as a reference (see supplementary file for details).

Statistical analysis

The number of cases of lymphoma in France during the study period determined the sample size. A descriptive analysis was performed for the whole sample. We identified the risk factors of lymphoma by both univariate and multivariate analysis (conditional logistic regression model). The SIR was calculated for anti-TNF agents use as a whole and for agents used individually. We performed subgroup and sensitivity analyses. (see supplementary file for details).

Compliance with research ethics standards

This study was authorized by the ethic committee of AP-HP, GHU Nord (Institutional Review Board of Paris North Hospitals, Paris 7 University, AP-HP; authorization number 162-08). The registry was reported at clinicaltrials.gov (ClinicalTrials.gov Identifier: NCT00224562).

RESULTS

Description of the cases

We collected data on 41 cases of lymphomas, and 38 cases were validated. Among them, 31 were NHL (26 B-cell and 5 T-cell), 5 HL and 2 Hodgkin's-like lymphoma.

The characteristics of the cases are in Table 1.

Biopsy specimens were reviewed for 36 cases (29 NHL, 5 HL and 2 Hodgkin's-like lymphomas). EBV was detected in 2 of 2 Hodgkin's-like lymphoma, 3 of 5 HL and one B-cell lymphoproliferation.

The underlying disease was RA in 27 cases, SpA in 7 cases (AS in 4 cases and psoriatic arthritis in 3 cases), CD in 3 cases and primary Sjögren's syndrome in one case. Secondary Sjögren's syndrome was present in 3 of 27 patients with RA and lymphoma. All patients were HIV negative. Most patients (31/38) had received only one anti-TNF agent. The 3 patients with CD and lymphoma had previously received azathioprine.

Outcome

Three patients with low-grade NHL received no lymphoma-specific treatment, and anti-TNF therapy was stopped. They remained with stable disease, without any progression or regression of the lymphoma (follow-up 19.8-37.0 months). Among the other patients, 29 received chemotherapy, 3 rituximab alone, and 2 radiotherapy, and 1 died before receiving chemotherapy. At last follow-up (median follow-up 18.2 months), 16 cases were in remission, in 3 disease was stable without specific treatment of the lymphoma, in 3 disease relapsed, 7 were still being treated and 9 patients died (24%); 4 of 27 with B-cell NHL, 2 of 5 with T-cell NHL and 3 of 7 with HL and Hodgkin's-like lymphoma.

Time occurrence of lymphoma with anti-TNF therapy

The median time from onset of anti-TNF treatment and the first symptoms of lymphoma was 23.6 months. In 5 patients, lymphoma occurred, but anti-TNF therapy had been discontinued 6.1 to 44.1 months before. For these 5 patients, the last anti-TNF agent received was infliximab for 3 and adalimumab for 2. As indicated in Figure 1, the relation between cumulative frequency of lymphoma and time from onset of anti-TNF therapy appeared to be approximately linear for the first or last anti-TNF agent received and did not differ by drug received.

Risk factors of lymphoma for patients receiving anti-TNF therapy

The case-control study involved 37 cases and 74 controls (as described in the methods, the patient treated for Sjögren's syndrome was not included). The repartition of the 3 anti-TNF in the control group corresponded to that found in the country during the 2004-2006 period, with 18% receiving adalimumab, 51% etanercept and 31% infliximab. The results of univariate analysis are given in Table 2.

On the multivariate analysis (Table 3), 2 factors were independently associated with occurrence of lymphoma in patients receiving anti-TNF therapy: anti-TNF treatment duration less than 2 years (OR=3.30 [1.17-9.30]) and treatment with infliximab or adalimumab versus etanercept (OR=4.12 [1.36-12.49] and OR=4.73 [1.27-17.65], respectively). In the case-control study restricted to RA cases, only one factor was associated with occurrence of lymphoma: treatment with infliximab or adalimumab versus etanercept (OR=6.68 [1.90-23.54]). The ORs of adalimumab or infliximab versus etanercept were also very similar with the second matching and in subgroup and sensitivity analyses (Figure 2).

Incidence and risk of lymphoma for patients receiving anti-TNF therapy compared with the general population

The main analysis relied on a total number of 57,711 patient-years of use of anti-TNF therapy during the 2004-2006 period, as the denominator of the incidence rate. The annual incidence rate of lymphoma adjusted for age and sex among patients receiving anti-TNF therapy, with the French population as a reference, was 42.1 (95% CI 6.9-77.2 per 100,000 person-years). The SIR was 2.4 (95% CI 1.7-3.2; p<0.0001) (Figure 3). For RA and SpA, the SIR was 2.3 (1.6-3.3; p<0.0001) and 1.9 (0.9-4.0; p=0.09), respectively.

Like for the case-control study, the incidence of lymphoma for patients receiving anti-TNF therapy differed depending on the agent received. The incidence rates for patients receiving etanercept, adalimumab, and infliximab were 15.3 (95% CI 0.0-45.6) per 100,000 person-years, 65.1 (95% CI 0.0-160.0) per 100,000 personyears and 69.1 (95% CI 0.0-150.4) per 100,000 person-years, respectively. The SIRs were 0.9 (0.4-1.8; p=0.72), 4.1 (2.3-7.1; p<0.0001), and 3.6 (2.3-5.6; p<0.0001), respectively. We found a difference between etanercept and monoclonal-antibody therapy in the main analysis and in the sensitivity analyses (Figure 3), even when we separately used the different estimates from independent sources, which gave very consistent adjusted incidence rates and SIRs (supplementary Figure 1).

DISCUSSION

This 3-year study is the first national prospective study recording all cases of lymphoma in patients receiving anti-TNF agents, whatever the underlying disease.

This study allowed us to collect enough cases to differentiate between lymphoma risk by use of anti-TNF agent. We found higher incidence of lymphoma with use of the two monoclonal-antibody agents (adalimumab and infliximab) than with the solublereceptor agent (etanercept). Three cohorts of RA patients have been used to compare treatment with anti-TNF agents and with classical disease-modifying anti-rheumatic drugs in terms of risk of lymphoma; 7 9-11 and did not found an increased risk with anti-TNF agents (relative risk of 1.0 [0.6-1.8] 7 , 1.35 [0.82-2.11] 9 and 1.11 [0.51-2.37] 10 ) These studies failed to demonstrate a difference between the treatments in risk of lymphoma due to insufficient power. Although the design of the RATIO study has some limitations, it is probably the only way (or at least the most powerful way) to investigate difference in risk with use of anti-TNF agents.

Our study may have some limitations:

The denominator of the incidence rate was estimated only. However, because each firm evaluated the number of patient-years in the period for each anti-TNF agent, the difference in risk between agents we observed cannot be explained by different methodologies used for the different agents. Furthermore, in the sensitivity analyses, the different estimates from independent sources gave very consistent adjusted incidence rates and SIRs (supplementary Figure 1).

Despite the different strategies used to identify all the cases in the whole country, we cannot exclude that some cases were missed. We make the assumption that reporting was equal with each biologic.. Actually, reporting of adverse events could be lower in patients treated sub-cutaneously (SC; i.e. etanercept and adalimumab) outside the hospital. But, in France, the SC-treated patients are mandatory seen by hospital physicians initially and yearly for renewal. Moreover, the lack of AE reporting is a main issue for minor side effects but not for life-threatening side effects, particularly lymphomas that are a major concern for physicians and patients regarding anti-TNF agents. Furthermore, the patients treated with anti TNF agents that have lymphomas could be notified to RATIO by the anti TNF agent prescribor (rheumatologist, gastro-enterologist, internist or others), by the onco-haematologist, or by the pharmacovigilance regional center. Finally, we found that the risk of lymphoma was similar for adalimumab and infliximab, that share the same mechanism of action (different from the one of etanercep), but adalimumab is a S.C. anti TNF agent and infliximab I. V. anti TNF agent.

Finally, the cumulative activity of the disease, known as a risk factor of lymphoma at least for RA patients, 4 could be different among patients receiving the different anti-TNF agents. Indeed, disease in patients receiving therapy at the beginning of anti-TNF availability (before 2002), was probably more severe, and such patients received exclusively infliximab (the only anti-TNF available in France at that time). However, patients with anti-TNF onset before 2002 did not have a higher risk of lymphoma than others (anti-TNF onset before 2002: OR=1.3 [0.5-3.7]; p=0.60).

Furthermore, the comparison between the type of drug used and risk was adjusted on the time from onset of anti-TNF treatment (Table 3). In addition, indirect markers of disease activity (median duration of the inflammatory underlying disease, percentage of patients treated with steroids, frequency of positive rheumatoid factor and anti-CCP in RA patients) were not greater in patients treated with infliximab or adalimumab than in those treated with etanercept (data not shown). Lastly, the impact of a putative difference in duration of exposure and in disease activity depending on year of introduction of the anti-TNF agent probably cannot explain the difference in incidence of lymphoma depending on the type of anti-TNF agent used because we observed exactly the same increased risk of lymphoma for patients receiving infliximab, which was introduced in 1999, and adalimumab, which has been available since 2004, whereas the risk was lower with use of etanercept, available in France from early 2003.

The strengths of this study are that our population of focus was the whole French population receiving anti-TNF therapy, whatever the indication for use, rather than a limited and selected population included in a specific cohort study. Furthermore, all the cases were validated by an expert committee, and the biopsy specimens were centralized, reviewed by the same hematopathologist and tested for EBV.

Even though the overall risk of lymphoma in RA patients treated with anti-TNF therapy does not appear to differ greatly from what is expected in a population of patients with inflammatory diseases, [START_REF] Zintzaras | The risk of lymphoma development in autoimmune diseases: a meta-analysis[END_REF][START_REF] Smedby | Autoimmune and chronic inflammatory disorders and risk of non-Hodgkin lymphoma by subtype[END_REF][START_REF] Landgren | Autoimmunity and susceptibility to Hodgkin lymphoma: a population-based case-control study in Scandinavia[END_REF] the risk differs depending on the anti-TNF drug used (higher risk with monoclonal anti-TNF therapy, adalimumab and infliximab). This difference in risk depending on agent was found in the case-control study and confirmed in the comparison of incidence with the general population, which supports the robustness of this finding. A meta-analysis assessing cancers in randomized controlled trials using monoclonal anti-TNF therapy, adalimumab and infliximab in RA patients revealed 10 cases of lymphoma (4 in randomized phases of trials and 6 in extension phases) in the treated groups (3,493 patient-years) and 0 in placebo groups (1,512 patient-years). [START_REF] Bongartz | Anti-TNF antibody therapy in rheumatoid arthritis and the risk of serious infections and malignancies: systematic review and meta-analysis of rare harmful effects in randomized controlled trials[END_REF] The same analysis of randomized controlled trials with etanercept of 2,484 patient-years with etanercept treatment and 1,072 patient-years with control therapy revealed 2 cases of lymphoma with etanercept therapy and none with control therapy. [START_REF] Bongartz | Etanercept therapy in rheumatoid arthritis and the risk of malignancies[END_REF] Finally, cases of hepatosplenic T-cell lymphoma have been described in adolescents and young adults with CD treated with monoclonal antibodies and azathioprine combined (10 cases with infliximab [START_REF] Mackey | Hepatosplenic T cell lymphoma associated with infliximab use in young patients treated for inflammatory bowel disease[END_REF] and 3 cases with adalimumab. [START_REF] Agency | Committee for medicinal products for human use[END_REF] )

The absence of intrinsic increased risk of lymphoma in SpA patients makes this population an ideal model for assessing the anti-TNF-related risk of lymphoma. [START_REF] Park | Spontaneous regression of EBV-associated diffuse lymphoproliferative disease in a patient with rheumatoid arthritis after discontinuation of etanercept treatment[END_REF] In our study, no significant increase in risk of lymphoma was observed in patients receiving anti-TNF therapy for SpA. However, no definitive conclusion may be drawn from our data based on very few cases of lymphoma in SpA patients. Some of the cases we observed reinforce the likelihood of a causal role of anti-TNF therapy in risk of lymphoma. In one patient with AS who never received other immunosuppressors, including MTX, EBV-associated Hodgkin's-like lymphoma developed after treatment with infliximab. We observed 3 cases of EBV-induced lymphoproliferation: 2 cases of EBV-associated Hodgkin's-like lymphomas with infliximab treatment (one with RA, one with AS described above) and 1 case of EBV-associated B-cell NHL in a patient with RA treated with adalimumab. These 3 cases demonstrate that lymphomas similar to post-transplant lymphoproliferative disease may occur, even rarely, with anti-TNF treatment. In the literature, one case of EBV-associated lymphoproliferation in a patient with RA treated with etanercept regressed after withdrawal of the drug. [START_REF] Thonhofer | Spontaneous remission of marginal zone B cell lymphoma in a patient with seropositive rheumatoid arthritis after discontinuation of infliximab-methotrexate treatment[END_REF] Another case of hypopharynx MALT lymphoma not associated with EBV regressed spontaneously after withdrawal of infliximab. [START_REF] Baran-Marszak | Effect of tumor necrosis factor alpha and infliximab on apoptosis of B lymphocytes infected or not with Epstein-Barr virus[END_REF] The pathophysiological mechanism inducing a higher risk of lymphoma in patients receiving anti-TNF therapy remains unclear. A direct action of TNF or anti-TNF on B cells was hypothesized, but no increase in survival or apoptosis with TNF or infliximab treatment was found. 23 Actually, in inflammatory diseases and especially RA, the 3 anti-TNF agents may have opposite effects: a beneficial effect due to the decrease in activity of the disease and a deleterious effect due to immunomodulatory activity, which may concern EBV-associated lymphoma but also more classical lymphoma. The mechanism of action of this deleterious effect is still unknown but could be related to T-cell control of viruses such as EBV or of other mechanisms of lymphomagenesis.

This T-cell control may require integrity of membrane TNF, which is upregulated in activated T cells. Some studies suggest a higher efficacy of anti-TNF monoclonalantibody treatment than TNF soluble receptor therapy for inhibiting membrane TNF signaling, [START_REF] Scallon | Binding and functional comparisons of two types of tumor necrosis factor antagonists[END_REF] which could lead to a decreased immune surveillance of different mechanisms of lymphomagenesis.

In conclusion, some lymphomas associated with immunosuppression may occur in such patients. The incidence of lymphoma is higher with monoclonal-antibody agents than with the soluble receptor. This may be due to a difference of targeting membrane TNF, leading also to difference of effectiveness in some diseases such as Crohn's disease or granulomatous diseases.

The authors' contributions were as follows: -Conception and design: Mariette, Tubach, Salmon, Lemann, Ravaud Acquisition of data: Mariette, Bagheri, Bardet, Berthelot, Gaudin, Heresbach, Martin, Schaeverbeke, Salmon, Lemann, Hermine, Raphael -Analysis and interpretation of data: Mariette, Tubach, Ravaud -Drafting the article and revising it critically for important intellectual content: Mariette, Tubach, Ravaud -Final approval of the version to be published: Mariette, Tubach, Bagheri, Bardet, Berthelot, Gaudin, Heresbach, Martin, Schaeverbeke, Salmon, Lemann, Hermine, Raphael, Ravaud Mariette, Tubach and Ravaud had full access to the data in the and take responsibility for the integrity of the data and the accuracy of the data analysis. The main analyses were independently performed by two academic statisticians.

Mariette received consulting and/or talk honorarium from Abbott, Schering Plough, UCB and Wyeth, Schaeverbeke received consulting and/or talk honorarium from Abbott, Schering Plough and Wyeth, Lemann received consulting and/or talk honorarium from Abbott, Schering Plough and UCB. Risk is higher with etanercept Risk is higher with adalimumab or infliximab n is the number of cases involved in the calculation the plot size relates to the number of patients treated involved in the calculation

Figure 1 :

 1 Figure 1: Time from onset of first and last anti-TNF treatment and first symptoms of lymphoma (months)

Figure 2 :Figure 3 :

 23 Figure 2: Estimation of the standardized incidence ratio (SIR) for risk of lymphoma according to underlying disease, and the histological subtype of lymphoma

Table 1 .

 1 Characteristics of the 38 lymphoma cases

		Lymphoma
	All lymphoma	in RA patients
	(n=38)	(n=27)

Table 2 :

 2 Risk factors of lymphoma for patients receiving anti-TNF agents (univariate analysis: main analysis and analysis restricted to rheumatoid arthritis [RA] patients)

			All lymphoma			Lymphoma in RA patients
			(37 cases and 74 controls)			(27 cases and 54 controls)
		Cases	Controls	OR	p	Cases	Controls	OR	p
		(n=37)	(n=74)	[95% CI]	value	(n=27)	(n=54)	[95% CI]	value
	Duration of the underlying
	inflammatory disease (years)	11.07 (8.75)	16.5 ± 12.4	0.95 [0.91 -1.00]	0.047	11.26 (8.97)	17.46 (12.12)	0.94 [0.88 -1.00]	0.06
	Activity of the underlying
	inflammatory disease at the
	time of diagnosis of lymphoma
	Null, fair, or moderate	23 ( 69.7%)	42 (85.7%)	1	0.22	17 ( 70.8%)	33 ( 89.2%)	1	0.15
	High	10 ( 30.3%)	7 (14.3%)	2.04 [0.65 -6.44]		7 ( 29.2%)	4 ( 10.8%)	2.84 [0.69 -11.67]
	Number of anti-TNF agents
	received
	2 or 3 1	7 ( 18.9%) 30 ( 81.1%)	18 ( 24.3%) 56 ( 75.7%)	0.71 [0.25 -1.97] 1	0.51	7 ( 25.9%) 20 ( 74.1%)	14 ( 25.9%) 40 ( 74.1%)	1.00 [0.33 -3.04] 1	1.00
	First anti-TNF agent received
	Etanercept Infliximab Adalimumab	11 ( 29.7%) 18 ( 48.6%) 8 ( 21.6%)	30 ( 40.5%) 35 ( 47.3%) 9 ( 12.2%)	1 1.38 [0.55 -3.45] 2.64 [0.74 -9.41]	0.33
	First anti-TNF agent received
	Etanercept Adalimumab or infliximab	11 ( 29.7%) 26 ( 70.3%)	30 ( 40.5%) 44 ( 59.5%)	1 1.61 [0.69 -3.78]	0.27	8 ( 29.6%) 19 ( 70.4%)	26 ( 48.1%) 28 ( 51.9%)	1 2.05 [0.79 -5.27]	0.14
	Last anti-TNF agent received
	Etanercept Infliximab Adalimumab	7 ( 18.9%) 18 ( 48.6%) 12 ( 32.4%)	33 ( 44.6%) 28 ( 37.8%) 13 ( 17.6%)	1 3.00 [1.10 -8.15] 4.52 [1.39 -14.71]	0.03
	Last anti-TNF agent received
	Etanercept Adalimumab or infliximab	7 ( 18.9%) 30 ( 81.1%)	33 ( 44.6%) 41 ( 55.4%)	1 3.40 [1.31 -8.80]	0.012	4 ( 14.8%) 23 ( 85.2%)	29 ( 53.7%) 25 ( 46.3%)	1 6.68 [1.90 -23.54]	0.003
	Time from onset of first anti-

Table 3 :

 3 Risk factors of lymphoma for patients receiving anti-TNF agents (multivariate analysis: main analysis and analysis restricted to rheumatoid arthritis [RA] patients)

	All lymphoma
	(37 cases and 74 controls)
	OR	p value
	[95% CI]

Acknowledgements

The authors thank all the clinicians who actively participated in the RATIO Registry: Abitbol (Paris), Allanore (Paris), André (Clermont-Ferrand), Ardizzone (Mulhouse), Bergman (Paris), Azais (Poitiers), Bachelez (Paris), Bardet (Orléans), Beau (Poitiers), Bergman (Paris), Belmatoug (Clichy), Berthelot (Nantes), Blasco

, Bonnet (Limoges), Bouhnik (Clichy), Bourgarit (Paris), Bouvard (Angers), Bressot (Chalon sur Saone), Briançon (Aix les bains), Brocq (Nice), Cadiot (Reims), Castela (Nice), Visanica (Metz), Combert (La Rochelle), Couret (Valence), Cuillerier (Dreux), Dalle (Lyon), Dasilva (Elbeuf), Debandt (Aulnay sous-bois), Debourdeau (Lyon), Depernet (Langers), Dereure (Montpellier), Descamps (Paris), Duriez (Saint-Brieuc), Fach (Bergerac), Fain (Bondy), Fautrel (Paris), Filippi (Nice), Fior (Bondy), Flourie (Lyon), Fulpin (Marseille), Gaborit (Orange), Gaudin (Grenoble), Gendre (Paris), Ghringhelli (Bordeaux), Gillet (Nancy), Goupille (Tours), Grados (Amiens), Grosclaude (Uriage), Gueit (Rouen), Guillaume (Colmar), Guyot (Roubaix), Heresbach (Rennes), Hoen (Besançon), Houvennagel (Lomme), Beguinot (Reims), Jang-Guyro (Briançon), Jardin (Privas), Justum (Caen), Laharie (Bordeaux), Lambotte (Le kremlin-Bicêtre), Lecompte (Nancy), Leparc (Boulogne-Billancourt), Lequen (Pau), L'hirondel (Caen), Liné (Soissons), Lioté (Paris), Lucht (Saint Etienne), Maillefert (Dijon), Marguerie (Berck), Maqub (Arles), Marteau (Paris), Martin (Saint-Brieuc), Mehadaoui (Evreux), Melac-Ducamp (Nevers), Meyer (Paris), Miceli (Le kremlin-Bicêtre), Michelet (Rennes), Morel (Montpellier), Nocent (Bayonne), Novel (Dijon), Pallot Prades (Saint-Etienne), Pham (Marseille), Piroth (Dijon), Perdriger (Rennes), Pertuiset (Pontoise), Petitou (Bigorre), Pouplin (Rouen), Puechal (Le Mans), Pujol (Clermont Ferrand), Rouidi (Dreux), Sacchi (Mantes-La Jolie), Saindenberg (Clichy), Salmon (Paris), Schaeverbeke (Bordeaux) Solau-Gervais (Lille), Sordet (Strasbourg), Sprunk (Bourg en Bresse), Taillan (Monaco), Thevenot (Laon), Thorel (Lorient), Ulmann (Marseille), Vernhes (Libourne), Wendling (Besançon), Zarnitsky (Le Havre), Zabraniecki (Toulouse), Zeller (Paris).

The authors are also grateful to G.R. Auleley, J Deligne and C Blum-Boisgard from the RSI for providing data to validate the denominator estimate of the incidence rate we used, P Grosclaude from the Francim for providing the French annual incidence rate of lymphoma by 5-year age and sex class in 2005, the AFSSAPS and the regional pharmacovigilance centres for their contribution to the exhaustiveness of the RATIO registry.

The authors thank C Roy and G Baron for the statistical analysis, and N Nicolas, S Makhlouf and A Djemoui for their help in collecting and preparing the validation of the cases. The pharmaceutical companies (Abbott, Schering Plough and Wyeth) had no role in the design and conduct of the study, collection, management, analysis, and interpretation of the data, preparation, review, or approval of the manuscript. Risk is lower than in the French population Risk is higher than in the French population


Christophe Sira

Solène Malerba

Lucie Schirr

Véronique Bertrand

Marc Schaming

The web interface for consulting documents via the TGIR Huma-Num data portal is currently under development (https://huma-num.fr). Pending completion of this project, the documents already digitised are available to scientists in raw form from the BCSF-Rénass (bcsf-renass@unistra.fr).

Lorem ipsum

Nearly 400,000 documents to be digitised Its archives are all the documents resulting from macroseismic surveys carried out since that date (forms, letters, maps, newspaper articles, macroseismic reports, etc.). Between 1921 and 1996, 960 earthquakes were surveyed to estimate macroseismic intensities (severity of ground shaking) by sending paper forms to the municipalities (Sira et al. 2021a). This represents 29 linear meters of archival documents (photo opposite). The aim of the MacrosisData project is to safeguard, disseminate and promote this collection, unique in France, through an online database.

Over the last ten years or so, the French seismological, geodetic and gravity communities have come together around of Résif-Epos (Réseau sismologique et géodésique français -European Plate Observing System). The aim of the Résif-Epos "Seismicity Transverse Action" is to coordinate all work on French seismicity within a single, transversal structure, with the aim of increasing the efficiency and visibility of the work carried out.

Axis 3 of this action concerns the collection and analysis of macroseismic data (historical and contemporary). It is in this area that the desire to safeguard a unique legacy was born, through the implementation of a documentary digitization project. This project is scheduled to run from 2020 to 2025.

Background

Since 2020, 86% of the files in the 1921 -1996 collection have been digitized. This digitization phase concerns almost 200,000 pages in PDF format, or almost 150 GB of data.

A test phase for the database and its consultation interface is scheduled for late 2023, to validate the final product design.

At the end of the project, this online database will make it possible to download all the macro-seismic documents available on a particular department, town or earthquake, using text or geographical queries. This irreplaceable collection, the only one of its kind in France, will be preserved and will give scientists from a wide range of specialities (architects, town planners, civil engineers, historians, sociologists) access to this information for the development of new and innovative research.

The project aims to be operational by 2025.

Results

The main objective is to bring together all digitized macroseismic survey documents in a dedicated database, to enable access to information in line with the so-called FAIR principles (Findable, Accessible, Interoperable, Reusable).

Over and above the aspect of preserving and safeguarding archival macroseismic data (Sira et al. 2021b), this project will provide data for numerous innovative scientific studies (data mining, macroseismic revisions, building studies, communal vulnerabilities, human behavior, updated macroseismic scales, etc.).

Objective

This multi-year project began in 2020 with the creation of a Data Plan Management (on a project scale), data description, documentation and associated data quality, legal (General Data Protection Regulation -GDPR) and ethical requirements through to ongoing data storage and backup. Documents are digitized by a specialized company that provides Acrobat PDF format documents. After digitization, the quality of the scanned documents is checked. The "survey" files for each event are segmented by document type (letter, press, forms, cards, photos, etc.). The data is then associated with a metadata file describing its characteristics and limits of use. Municipal geographical indexation is associated with the data, enabling geographic selections to be made.

A database and its consultation interface are being created (2023)(2024)(2025) to store and enable attribute and geographic queries on the database. This phase is based on the Huma-Num Research Infrastructure (https://huma-num.fr) of the French Ministry of Higher Education and Research, which implements this digital infrastructure enabling scientific communities to develop, implement and preserve research data and tools over the long term -in an open science and shared data approach.

MacrosisData ( 1 )

 1 Université de Strasbourg, CNRS, EOST UAR 830, F-67000 Strasbourg, France (2) Université de Strasbourg, Service des archives, F-67000 Strasbourg, France (3) Université de Strasbourg, CNRS, ITES UMR 7063, F-67000 Strasbourg, France Abstract for more information : www.franceseisme.fr contact : christophe.sira@unistra.fr Safeguarding and promoting the heritage of french macroseismic surveys carried out between 1921 and 1996 BCSF-Rénass, the French Central Seismological Bureau and National Seismic Monitoring Network, has been responsible for estimating macroseismic intensities in France since 1921 (fig. 1).

  Fig. 1 : Maximum communal intensities in mainland France 1921-2021. doi:10.25577/6w78-nn28