S Picart

M Crozet

D Roudil

R Brennetot

F Pilon

INTER-LABORATORY COMPARISON FOR TRACE ANALYSIS: EQRAIN TRACES PROFICIENCY TESTING ORGANIZED BY CETAMA

The Commission d'Établissement des Méthodes d'Analyse (CETAMA) organises Inter-Laboratory Comparison round robins entitled "Evaluation of the Quality of Analysis Results in the Nuclear Industry"," (EQRAIN, "Evaluation de la Qualité du Résultat d'Analyse dans l'Industrie Nucléaire", in French). The "EQRAIN Trace Elements n° 21" inter-laboratory comparison (ILC), which was launched in the first half of 2016, concerned the elemental chemical analysis of 15 elements whose concentrations were between 0.2 and 15 mg/L. The main objective of this proficiency test was to check the quality of measurement results for the concentrations of metallic elements in a solution. It enabled the participating laboratories to evaluate their performances in carrying out this type of analysis and if they wish, to compare different analytical techniques. After the measurement results had been returned, the reference values were sent to the laboratories which were then able to estimate the bias of their measurements and to check for the absence of drift, implementing corrective actions if necessary. Inductively coupled plasma mass spectrometry (ICP-MS) and inductively coupled plasma atomic emission spectrometry (ICP-AES) are the best-adapted analytical techniques for this kind of exercise, leaving atomic absorption a complementary role and the use of colorimetry for certain special elements.

1.

INTRODUCTION

One of the missions of the Analysis Method Establishment Commission (CETAMA) is to supervise the organization of Inter-Laboratory Comparisons (ILC) for the nuclear industry and for environmental or safeguards public organizations. These ILC programs are organized every year or every two years on uranium, plutonium and anions or cations; at a trace level in solution for the latter.

Notably, the EQRAIN traces ILC concerns laboratories implementing inductively coupled plasma atomic emission spectroscopy (ICP-AES) or mass spectrometry (ICP-MS) and willing to estimate the accuracy of their measurements for trace elements at concentrations between 0.1 and 15 mg/L. The challenge for the laboratories is to evaluate the performance of their methods and equipment for the determination of these elements in solution, and to meet the requirements of quality management and accreditation body.

The "target" elements are chosen by partners during working group (WG) meetings dedicated to Atomic Emission Spectroscopy (WG6) and Mass Spectrometry (WG12). The results of the ILC are presented every year during joint WG6-WG12 meetings, in which the performance scores are discussed anonymously. It also provides the opportunity to run a variance analysis on both techniques, ICP-AES and ICP-MS, for the 15 elements and to highlight evolution in their performance.

This paper will then focus on the implementation of "EQRAIN Traces N°21" proficiency testing and on the statistical treatment of the results 1 .

OBJECTIVES

Since 1987, the CETAMA has been organizing EQRAIN Traces ILC which concern the determination of The ILC which is presented hereafter took place between May and September 2016, and it enabled the 30 participating laboratories to evaluate their performance in elemental chemical analysis for all or part of the 15 elements available, using the technique of their choice.

The main objective was to check the quality of elemental concentration measurement results for a solution, and to thus enable the participating laboratories to check their conformity to their quality system and to respect the requirements imposed by the related standards.

The ILC was also intended to situate the general level of performances in the main techniques and associated analytical methods currently used for multi-elemental analyses, i.e. inductively-coupled plasma mass spectrometry (ICP-MS) and inductively-coupled plasma atomic emission spectrometry (ICP-AES).

Result processing was based on the normative system in force, i.e. the ISO NF 13528 standard for the evaluation of laboratory performances and of deviations from trueness, and the ISO NF 5725 standard for the determination of method precision. Through its analytical approach, it permits not only the evaluation of the measurement techniques but also the highlighting of the difficulty of testing for certain elements.

TARGET ELEMENTS

The material to be analyzed was a multi-elemental solution including the following elements: Al, B, Be, Co, Fe, Gd, Li, Mo, Ni, Ru, Si, Sn, Ta, U, and Zr.

The material suppliers was SPEX-CertiPrep and the certified concentrations are given in Table 1 with the associated expanded uncertainties expressed in mg/L (for k=2). The reference values were certified with a relative uncertainty of 2% for a period of 6 months. The 15 elements to be analyzed were decided on after discussion among the members of the Working Groups concerned, i.e. WG 06 "Atomic Absorption and Emission Spectrometry" and WG 12 "Mass Spectrometry", early in 2016.

They were selected depending on the applications implemented in the main laboratories. Several families can be listed, including those inevitable for spectrometry (Fe, Mo, Ni), the alkalines and alkalino-earths (Li, Be), the elements of interest in fuel cycle processes (Al, Mo, B, Si, Zr, U, Gd) and those which are important for environmental monitoring (U, Be). This EQRAIN Traces 21 ILC also offered the analysis of 3 new impurities which are rarely proposed: Sn, Co and Ru. Thus 4 elements were excluded -As, Pd, Se, and Zn -and 4 newcomers were introduced, Co, Ru, Sn, and Ta. Uranium, an important tracer in nuclear activities, has been present in all the round robins, with a concentration range of from 0.2 to 10 mg/L. The elements selected for the EQRAIN 21 ILC are shown in the Mendeleyev Periodic Table in Figure 1. The samples were distributed to the 30 laboratories participating, among AREVA, CEA, CNRS, EDF, EVANS ANALYTICAL, HUNGARIAN ACADEMY OF SCIENCES, IRSN, JRC KARLSRUHE, KAERI, MELOX, VALAIS. Depending on its choice of technique(s) to implement and thus the number of additional vials requested, each laboratory could send back several series of results if different techniques or different operators were involved in the tests. Thus one laboratory could be allocated several laboratory codes, and there were 64 laboratory codes in total.

FIG. 1. Chart showing the position of chosen elements for the EQRAIN Traces ILC in the Mendeleyev periodic

The samples were sent out to the laboratories in June 2016, and the round robin was closed at the end of September that year.

Table 2 reports the number of measurements carried out for each element and for each technique. These indicate the levels of interest which the laboratories had in testing for certain elements, or may point to difficulties or specific issues involved with some techniques when testing for these elements. The results were presented in Pau during the plenary meetings of joint meeting WG 06 "Atomic Absorption and Emission Spectrometry" and WG 12 "Mass Spectrometry", held 29-30 November 2016.

The result reporting formalism was imposed, but the participants were left a free choice as to the techniques they would implement.

Together with a written acknowledgement of the reception of its results, each laboratory received a number known only to itself and to the CETAMA and which served as the laboratory reference for the statistical processing and the anonymous return of the performance scores.

STATISTICAL PROCESSING

The results were processed following the recommendations given in the following standards NF ISO 13528: 2015 2 and -NF ISO 5725-5: 1998 [START_REF]method for the determination of repeatability and reproducibility of a standard measurement method[END_REF] .

These standards foresee the use of calculations called "robust" for data analyses. Such calculations integrate all of the results, including those considered as outliers by the criteria of a normal law. They are based on an iterative procedure which enables the influence of extreme values to be minimized.

Measurement model

The mathematical model for a given level of concentration considers that any result from an individual measurement for a given method without method bias (true method) is the sum of 3 components m: expected value for Y; m is estimated by the reference value, b: bias from the laboratory, with a variance (interlaboratory variance) for which the estimation on a sample is , e: random error of the sample measurement (individual), with a repeatability variance (intralaboratory variance) for which the estimation on a sample is From the model of equation 1, the total variance of y for a sample, the reproducibility variance, , can be written as being equal to: Equation 2 During an interlaboratory comparison, there are several laboratories carrying out their measurement replicates. Thus hereafter in this document the nomenclature applied is that given in Table 3. TABLE 3 Nomenclature used for the laboratory tests. -Equation 4

Proficiency testing

In a proficiency test, each laboratory i is required to supply a result y lab_i and its associated uncertainty, using the analysis method of its choice (Participants were recommended to carry out n = 5 independent tests). Most of the time, the result y lab_i returned is the result of the mean of n i individual measurements carried out by laboratory i. This result therefore corresponds to the mean of n i values (replicates) of measurement y lab_i = (Equation 5 and Figure 2).

FIG. 2. Matrix representation of the data (analysis results

) for an identical number of replicates for all the laboratories.

-With or y lab_i the mean of the n i replicates from laboratory i. The general approach followed for the statistical processing of the results from a proficiency test ("proficiency testing", with the abbreviation "pt"), according to the NF ISO 13528 standard, is structured in 3 broad steps:

determination of the mean (robust) of the laboratory results y lab_i and of the standard deviation for the proficiency evaluation s pt , checking for the existence of outlier values, determination of the performance indicators (z score, zeta score…) for each laboratory. The main performance indicators are (equation 6), z (equation 7) and zeta (equation 9).

The difference D i between the result y lab_i and the assigned value y Pt expressed in %, estimation of the measurement bias for the laboratory concerned i, is: % Equation 6where -y lab_i is the measurement result supplied by laboratory i -y pt is the assigned value; in the case of this ILC, it is the certified reference value.

The z score, a performance indicator for the laboratory compared to all the participants without the evaluation of the estimation of its own reported uncertainty: Equation 7where s pt is the standard deviation for the proficiency test, here the robust standard deviation ("robust" is symbolized by *) for the laboratory measurements:

Equation 8 The zeta score, the evaluation of a participant's proficiency in obtaining values close to the reference value with the uncertainty estimated by that laboratory: zeta Equation 9where is the standard uncertainty associated with result y lab_i supplied by laboratory i, is the standard uncertainty associated with the assigned value for the ILC, here the standard uncertainty for the certified reference value. This statistical processing was carried out using the commercial software SAS JMP and used a"robust" calculation algorithm called "A" described in the standards NF ISO 13528 and NF ISO 57025.

The evaluation of a laboratory's proficiency is carried out based on the values of the z score and of the zeta score:  between -1 and + 1 being very satisfactory;  between -2 and -1 or between +1 and +2 being satisfactory;  between + 2 and +3 or -2 et -3 being debatable;  greater than +3 or lower than -3 being unsatisfactory.

Normalized deviation

The compatibility between the robust mean and the reference value y ref , which is equal here to y pt as given in the SPEX certificate, is represented by the normalized deviation , calculated with the following equation:

Where:

-: robust mean of all y lab-i results -: reference value -: robust mean uncertainty : reference value uncertainty

Interpretation of the normalized deviation: 

: satisfactory results When the normalized deviation is between -2.0 and +2.0, there is no discernible bias in the methods implemented by the laboratories.  ou : debatable results When the normalized deviation is between -3.0 and -2.0 or +2.0 and +3.0, there is either a potential bias in the methods implemented by the laboratories, or the reference value is erroneous. The presence of a bias should be confirmed in past and future comparisons.



: unsatisfactory results When the normalized deviation is less than -3.0 or higher than +3.0, there is a significant difference between the reference value and the mean of the laboratory means. This difference comes either from a bias in the methods implemented by the laboratories, or from an erroneous reference value. In this case, potential causes for this difference should be analyzed.

Method evaluation

The precision of the method can be appreciated by respecting the characteristic conditions described below:

-Conditions of repeatability exist when the measurements are made by the same operator, on the same instrument, using a single method and within a short period of time, in order to obtain measurements under conditions which are as similar as possible.

-Conditions of reproducibility exist when one of the following conditions of execution changes:

operator, instrument, variation in the application of the method, period of execution, or any other cause which adds sources of variability. In this type of approach, the laboratory is asked to repeat each individual measurement a certain number of times n (5 in our case), and to report each of these results y ij .. The planning of such a round robin enables the method to be validated via the criteria of trueness and of precision depending on the analysis techniques used and supposing that the methods applied are equivalent. It consists in using a variance analysis (ANOVA) to determine the standard deviations of repeatability and of reproducibility characterizing the method. Prior to these variance analysis, outliers were excluded using the Cochran and Grubs tests because classical statistics (not robust) are used and are sensitive to outliers. Then going back to Equations 3 and 4, model of the measurement and associated variance, and provided that the measurement results were obtained using the same method, the mean per laboratory y lab_i = can be obtained (equation 10):

= m + b i + e ij Equation 10as well as its associated variance (equation 11), provided that all the laboratories made the same number of replicates n i = n: Equation 11In the ANOVA approach, the sum of the squares of the deviations (noted SS total hereafter) from the general mean consists of 2 terms (Equation 12): one corresponding to the inter-laboratory variance  (noted SS inter ), and the other corresponding to a residual or repeatability or intra-laboratory variance  (noted SS intra ) (This is only valid for n i = n).

Equation 12  =  +  SS total = SS inter + SS intra By dividing by their respective numbers of degrees of freedom np-1 for term , p-1 for term  and p(n-1) for term , these terms correspond respectively to the total variance s 2 total , the inter-laboratory variance s [START_REF]Statistical methods for use in proficiency testing by interlaboratory comparison[END_REF] inter and the repeatability variance (also called intra-laboratory or residual) s r 2 = s intra 2 . The repeatability variance for the method s r 2 is therefore (equation 13):

Equation 13

The interlaboratory variance can be expressed depending on the standard deviations of the laboratory results (means) by equation 14: Equation 14Based on equation 12, i.e. , it is then possible to write:

Equation 15 Whence it can be taken that the inter-laboratory variance s [START_REF]Statistical methods for use in proficiency testing by interlaboratory comparison[END_REF] inter is composed of the variance due to the laboratory factor and to the intra-laboratory variance by the relationship: Equation 16Variance due to the laboratory factor is then :

Équation 17 The reproducibility variance of the method s R 2 equals (cf. Equation 4) : Equation 18 which writes as : Équation 19

RESULTS PER ELEMENT

A first graph recapitulates the means of the laboratory results by increasing order of the relative deviations D, identified by the laboratory codes and technique (Figure 3 in the case of Al). The information includes:

the reference value y ref = y pt together with its expanded uncertainty in the form of a green line, and a confidence interval also in green, the robust mean value together with twice the standard deviation for the robust mean u( ) in the form of a red line, and a confidence interval also in red.

FIG. 3. Al content analysis results for EQRAIN Traces 21 certified reference material..

Two more graphs show the relative deviations in increasing order, depending on the laboratory codes and the z and zeta scores arranged in increasing z score order, with the same laboratory codes (Fig. 4).

IAEA-ID-260

[Right hand page running head is the paper number in Times New Roman 8 point bold capitals, centred]

FIG. 4. Graphs of relative deviation Di% and of z / zeta scores for Al content analysis ( EQRAIN Traces 21 ILC).

Next, a table summarizes per element (Table 4) the reference value y ref together with its expanded standard uncertainty U(y ref ), the number of laboratories for the test, all techniques together and per technique; the robust mean and the robust standard deviation for the test in its totality s* (k=2) and for ICP-AES and ICP-MS results. A final part discusses the ANOVA variance analysis at 1 factor (laboratory) with graphs showing the result y i (outliers omitted) compared to the arithmetical mean for each technique (Fig. 5).

z (a) (b) FIG. 5.Graph of ANOVA results for ICP-AES (a) and ICP-MS (b) techniques for Al content analysis.

PICART et al.

[Left hand page running head is author's name in Times New Roman 8 point bold capitals, centred. For more than two authors, write

AUTHOR et al.]

9

Two final tables give the calculation of the standard deviations for repeatability and for reproducibility for the ICP-AES (Table 5) and ICP-MS methods.

SYNTHESIS OF RESULTS AND DISCUSSION

Bias analysis

En, the normalized deviations, which evaluate the compatibility between the robust means (consensual value for the proficiency test) and the reference values, are given in the following tables and figures for all the elements and for the two main techniques implemented, i.e. ICP-AES and ICP-MS.

FIG. 6. Graph of normalized deviation for the ICP-AES technique and EQRAIN Traces 21 ILC.

For the ICP-AES technique, only uranium had a borderline significant negative bias.

FIG. 7. Graph of normalized deviation for the ICP-MS technique and EQRAIN Traces 21 ILC.

For the ICP-MS method, no bias was detected. Both ICP-AES and ICP-MS results from ILC are then consistent with the certified values of the standard and indicates that the laboratories are generally good for the implementation of both techniques and trace analysis.

Performance indicators,

The z score enables an evaluation of the laboratory proficiency in giving a true result, compared to the expected uncertainties for the community taking part in the proficiency test.

The zeta score enables an estimation of the laboratory proficiency in giving a true result, compared to the uncertainties given by the laboratory combined with those of the reference.

In order to make reading the score indications easier, a common evaluation grid was adopted, as was the case for previous EQRAIN Traces ILC summaries.

The tables drawn up for each technique include a ranking by laboratory (in lines by increasing code number) and a ranking by element (in columns by alphabetical order): the value of the score is given in the box and a colour is attributed according to the key below:

 green indicates an absolute value for the z or zeta score between 0 and 1: very satisfactory result,  light green indicates an absolute value for the z or zeta score between 1 and 2: satisfactory result,  orange indicates an absolute value for the z or zeta score between 2 and 3: debatable result, .

 red indicates an absolute value for the z or zeta score higher than 3: unsatisfactory result,  white means "not determined".

FIG. 8. Z Score performance grid for ICP-AES technique (EQRAIN Traces 21 ILC).

A laboratory which tested the elements on its list very satisfactorily or just satisfactorily (score between 0 and 2 in absolute value) is considered to have met the test criteria, and is judged to be "satisfactory" in terms of either the z score or the zeta score.

From the score grid, it can be seen that 35% of the laboratories are "satisfactory to very satisfactory" for the ICP-AES method on the basis of the z criterion (13 laboratories out of 37), compared to 24% for the ICP-MS method (6 laboratories out of 24).

For the zeta criterion, the number of laboratories judged to be "satisfactory to very satisfactory" doubles for the ICP-MS method, going from 6 to 12, which shows that the laboratory uncertainties are well managed and enables the biases observed to be relativized. This is unlike what can be observed for the ICP-AES method, where the number of laboratories decreases from 13 to 8, indicating that the laboratory uncertainties are sometimes underestimated for this technique.

Global analysis by element

Studying the result of the proficiency test overall, it can be seen that the relative deviations (difference between the robust mean and the reference value, in relative value) are well-managed for all of the elements and are less than or equal to 1.5%. The result dispersion is also low, less than 1%.

The only exception is the element Si, for which the bias and the dispersion are 3.4 % and 2.7% respectively. To this we can add the element Ta, whose dispersion at 1.3% is greater than 1%, as is the case for Si.

This analysis is confirmed by the graphs in Figure 9 where it can be clearly seen that there is a significant number of unsatisfactory results for silicon in terms of zeta score (9 laboratories out of 26, i.e. one laboratory out of three) with the ICP-AES method showing a strong tendency to overestimate the value. As concerns the ICP-MS method, very few analyses were reported (4), which indicates the difficulty of analyzing Si routinely by ICP-MS. The same difficulty is found for Ta, but only when using ICP-AES.

FIG. 9. Graph of satisfactory zeta score by element for ICP-AES and ICP-MS technique.

In terms of the normalized deviation, U stands out for its significant negative deviation of -2.0, as the dispersion is low (0.4%) compared to the non-negligible relative deviation (1.0%). The same remark can be made in the case of Al, with a high negative normalized deviation of -1.8 (0.6% dispersion for a deviation of -1.2%) or for Zr, with a positive normalized deviation of 1.5 (1.0% dispersion for a deviation of 1.5%).

Continuing with Figure 9, a significant number of laboratories had "debatable" and "unsatisfactory" zeta scores for the measurement of Al using ICP-AES (8 laboratories did not meet the "satisfactory" criterion), thus confirming the difficulty of measuring the Al concentration with trueness for certain laboratories.

Analysis of repeatability and reproducibility

Concerning method repeatability, the repeatability standard deviations are mainly between 0.7 and 2% for ICP-AES and between 1.5 and 3% for ICP-MS when the elements do not represent any special problems for testing. Beyond this, the parameter indicates any special technical issues, for example for Si where the repeatability standard deviation rose to 2.7% with ICP-AES, or for Al using ICP-MS, with a value of 3.7%.

For method reproducibility, the standard deviation varies between 1.6 and 5.0% for ICP-AES and between 2.6 and 7.0% for ICP-MS, meaning a value more than double that for the repeatability standard deviation. If it is outside this range, the parameter confirms the existence of an analysis issue which may be caused by spectral or mass problems (interferences) or have a chemical origin (pollution, adsorption, concentration level...): this is the case for Si using ICP-AES, where the reproducibility standard deviation reaches 12.4%. It should also be remembered that these deviations are sensitive to the concentration level, as is IAEA-ID-260 [Right hand page running head is the paper number in Times New Roman 8 point bold capitals, centred] the case for Zr with a reproducibility standard deviation of 3.6% for ICP-AES. This is certainly high, but it corresponds to a low Zr concentration (200 µg/L).

CONCLUSION AND PERSPECTIVES

The EQRAIN Traces 21 ILC enabled the 30 participating laboratories to evaluate their performances in elemental chemical analysis for a solution of 15 trace elements. There was a free choice as to the technique to implement, and some laboratories used both ICP AES and ICP MS to carry out the tests Although mainly working in the nuclear-related field, the ILC participants are laboratories whose objectives (expertise or monitoring) as well as their fields of application (industry, research or instrumentation) are different, sometimes even opposite. The concentrations were chosen in order to to meet most of the needs in terms of trace measurement proficiency assessment. The annual frequency of EQRAIN ILCs is an opportunity to validate laboratory proficiency and to test new instruments or methods.

The statistical processing for all the results returned by the laboratories is based on the NF ISO 13528 standard and prioritizes robust methods for the calculation of trueness and for the expression of laboratory performance scores. It is also based on the ISO 5725 standard for the determination of the precision of the ICP-AES and ICP-MS methods via the evaluation of the standard deviations for repeatability and reproducibility.

These performance scores enable the laboratories to evaluate their proficiency in testing for elements at a level described as "trace", which are representative of those found in nuclear activities. They enable result trueness to be judged, as well as the realism of the reported laboratory uncertainties.

This round robin was also the opportunity to evaluate the performance of the ICP-AES and ICP-MS methods in terms of repeatability and reproducibility, for all the ILC participants. It thus enabled the laboratories to compare their individual performance with that of the group, and to examine changes to practices and methods if necessary.

The summary of this exercise shows very high quality analysis results, as attested to by the satisfactory z scores (|z|<2) for a large number of elements and laboratories. Moreover, a good number of laboratories returned excellent results, which confirms the reference values and the quality of the standards involved.

The number of laboratories registered and the distribution by technique were as follows:  30 participating laboratories, (64 laboratory codes)  37 series of results obtained with the ICP-AES technique,  25 series of results obtained with the ICP-MS technique.

Atomic absorption was used once in this ILC, as was colorimetry.

The data processing involved 64 series of results 64. This shows laboratory interest in such an exercise as an evaluation tool for their know-how. The data processed concerned 656 concentration measurement results, representing more than 3000 unitary values and the associated uncertainties.

Participation in type of round robin is recommended by the ISO 17025 standard related to the competence of laboratories in carrying out tests [ISO17025]. This is particularly true for laboratories which have undertaken steps towards certification or accreditation.

Finally, during our last meeting, the idea came to organize a future ILC on trace elements contained in a uranium matrix composed of a 10 g/L uranium solution in acidic medium. The implementation of such an ILC is currently subject to a feasibility study and would be a follow-up to the proficiency test run in 2012 by CETAMA at the request of the IAEA. A total of 15 elements, between 1-100 ppm relative to uranium, would have to be analyzed, and will be chosen between alkaline, alkaline earth, transition metal, platinum metal and lanthanide groups.

  (equation 1): where y = m + b + e Equation 1 y: result of the measurement (individual), IAEA-ID-260 [Right hand page running head is the paper number in Times New Roman 8 point bold capitals, centred]

  Index j : j = 1 to n Relative to the replicates Index i : i = 1 to p Relative to the laboratories Thus each individual measurement or replicate can be written ij (equation 3): y ij = m + b i + e ij Equation 3 where y ij: result of the j th replicate for laboratory i; m: expected value for Y; estimated by the reference value; b i : bias due to laboratory i; e ij : random error of the individual measurement for the j th replicate of laboratory i. For each measurement result y ij , variance is written as:

  Co Fe Gd Li Mo Ni Ru Si Sn Ta U Zr En IAEA-ID-260 [Right hand page running head is the paper number in Times New Roman 8 point bold capitals, centred]

  IAEA-ID-260[Right hand page running head is the paper number in Times New Roman 8 point bold capitals, centred]

TABLE 1 .

 1 EQRAIN Traces 21 metal concentration and their expanded uncertainties. EQRAIN21

		Al	B	Be	Co	Fe	Gd	Li	Mo	Ni	Ru	Si	Sn	Ta	U	Zr
	Xref (mg/L)	11.000	3.010	0.900	0.500	0.300	0.400	4.030	7.010	1.590	1.510	2.470	0.600	0.810	5.030	0.198
	U(Xref) (mg/L)	0.060	0.020	0.005	0.003	0.002	0.002	0.020	0.040	0.008	0.008	0.010	0.003	0.004	0.030	0.001

TABLE 2 .

 2 Counts of number of analysis per element and per element & technique.

	Element	Measur. ICP-AES	ICP-MS	Autre
	Al	50	36	13	1 (AA)
	B	49	34	15	0
	Be	43	24	18	1 (AA)
	Co	51	32	18	1 (AA)
	Fe	47	36	10	1 (AA)
	Gd	37	24	13	0
	Li	50	31	18	1 (AA)
	Mo	51	32	18	1(AA)
	Ni	57	37	19	1(AA)
	Ru	31	18	13	0
	Si	32	28	4	0
	Sn	38	21	17	0
	Ta	31	18	13	0
	U	40	20	19	1 (Color.)
	Zr	44	28	16	0

  PICART et al.[Left hand page running head is author's name in Times New Roman 8 point bold capitals, centred. For more than two authors, write AUTHOR et al.]

	Equation 5

  IAEA-ID-260[Right hand page running head is the paper number in Times New Roman 8 point bold capitals, centred]

TABLE 4

 4

	Al Eqrain Traces 21	Method	n Lab.	Mean/ robust mean uncertainty (k=2)	Normalized bias E n
		All techniques	50	(10.87 ± 0.13)mg/L	-1.81
	X ref = (11.00 ± 0.06) mg/L (k=2)	ICP-AES	36	(10.89 ± 0.15)mg/L	-1.34
		ICP-MS	13	(10.78 ± 0.34)mg/L	-1.25

Summary of the mean values obtained for Al content analysis and of the normalized deviation.

TABLE 5 .

 5 ANOVA calculation for results concerning Al content and ICP-AES measurements.

	ANOVA [Al](mg/L)	total	inter	intra
	ICP-AES	N = 170	p = 34
	SS	28.416	25.971	2.445
	deg. of freedom	169	33	136
	V	0.168	0.787	0.0180
	variance	V R 0.172	V L 0.154	V r 0.0180
	Standard deviation	s R	s L	s r
	estimation	0.414	0.392	0.134
		3.8%	3.6%	1.2%

[Left hand page running head is author's name in Times New Roman 8 point bold capitals, centred. For more than two authors, write AUTHOR et al.]


Christian Wolf
email: christian.wolf@insa-lyon.fr

Corentin Kervadec
email: corentin.kervadec@orange.com

Grigory Antipov
email: grigory.antipov@orange.com

Moez Baccouche
email: moez.baccouche@orange.com

Weak Supervision helps Emergence of Word-Object Alignment and improves Vision-Language Tasks

Weak Supervision helps Emergence of Word-Object

Alignment and improves Vision-Language Tasks

Introduction

High-capacity deep neural networks trained on large amount of data currently dominate methods addressing problems involving either vision or language, or both of these modalities jointly. Examples for vision-language tasks are image retrieval task [START_REF] Karpathy | Deep visual-semantic alignments for generating image descriptions[END_REF] (retrieve an image given a query sentence); image captioning [START_REF] Lin | Microsoft coco: Common objects in context[END_REF] (describe the content of an input image in one or more sentences), and Visual Question Answering [START_REF] Antol | Vqa: Visual question answering[END_REF] (VQA: textually answer a question on an input image) etc. These tasks require different forms of reasoning, among which we find the capacity to analyze instructions -e.g. the question in VQA -, or the ability to fuse modalities or to translate one modality into another one -e.g. in image captioning. Additionally, they often require different levels of understanding, from a global image-text comparison to fine-grained object-word matchings.

In this context, a large panoply of high-performing models adopt self-attention architectures [START_REF] Vaswani | Attention is all you need[END_REF] and BERT-like [START_REF] Devlin | Bert: Pre-training of deep bidirectional transformers for language understanding[END_REF] training objectives, which complement the main task-related loss with other auxiliairy losses correlated to the task. The common point of this large body of work is the large-scale training of unified vision-language encoders on image-sentence pairs. However, despite their ability to model interactions unique to one modality (i.e. intra-relationships), we observe that these State-Of-The-Art (SOTA) approaches tend to struggle to identify fine-grained object-word relationships (interrelationships, or cross-modality relationships). These relationships are important, which can be illustrated in the example of VQA: answering a question given an input image requires the detection of certain objects in the image, which correspond to words in the question, and eventually the detection of more fine-grained relationships between visual objects, which are related to entities in the sentence.

In the literature, the alignment or matching of words to visual objects is generally assumed to be implicitly learned from applicationspecific losses -mostly cross-entropy for classification -thanks to the inductive biases provided by the encoder's architecture, i.e. the possibility of the model to represent this kind of matching. In this work we show that (1) modality alignment (cf. Figure 1) does not necessarily emerge automatically and (2) that adding weak supervision for alignment between visual objects and words improves the quality of the learned models on tasks requiring visual reasoning.

Our contributions are as follows:

• We enhance vision-language encoder approaches by adding explicit weak supervision of object-word alignment, taking into account the uncertainty present in the detection result of the vision module. • We improve the accuracy of SOTA vision-language models on the VQA task GQA [START_REF] Hudson | Gqa: A new dataset for real-world visual reasoning and compositional question answering[END_REF] dataset) without the need of finetuning, to achieve SOTA-level results. In other words, with our new objective, pre-training is sufficient for SOTA results.

Related Works

Vision-language tasks -Vision and language understanding is a broad area and can take several forms at many different levels of granularity. Some tasks focus on matching problems, as for instance Image Retrieval, which requires finding the most relevant image given a query sentence [START_REF] Karpathy | Deep visual-semantic alignments for generating image descriptions[END_REF], [START_REF] Lee | Stacked cross attention for image-text matching[END_REF]. The inverse problem -namely Sentence Retrieval -has also been explored [START_REF] Karpathy | Deep visual-semantic alignments for generating image descriptions[END_REF]. A similar task with finer granularity is Visual Grounding, where the model must associate image regions to words or sentences [START_REF] Kazemzadeh | Referitgame: Referring to objects in photographs of natural scenes[END_REF], [START_REF] Bryan A Plummer | Flickr30k entities: Collecting region-to-phrase correspondences for richer image-to-sentence models[END_REF].

Other tasks require more high-level reasoning over images and sentences, which, in general, requires multi-modal interactions but also the ability to compare, count or find relations between objects in the image. In Visual Question Answering (VQA) [START_REF] Antol | Vqa: Visual question answering[END_REF] [START_REF] Hudson | Gqa: A new dataset for real-world visual reasoning and compositional question answering[END_REF] we ask questions (given as input) about an input image and the model must predict the answer. Answering the questions requires a variety of skills: finding relations, counting, comparing colors or other visual features, materials, sizes, shapes, etc. The binary task of Language-driven Comparison of Images takes as input triplets (img1, img2, sentence) and requires predicting whether the sentence truly describes the image pair [START_REF] Suhr | A corpus for reasoning about natural language grounded in photographs[END_REF].

Finally, some tasks involve the generation of one modality from the other. Image captioning consists in translating an image into text [START_REF] Lin | Microsoft coco: Common objects in context[END_REF]. Similarly, some tasks aim to generate questions about an image [START_REF] Li | Visual question generation as dual task of visual question answering[END_REF]. Inversely, it is also possible to generate an image from a caption [START_REF] Mansimov | Generating images from captions with attention[END_REF]. However, such multimodal generation is out of the scope of our work.

Vision-language multi-modal fusion -Early work in vision and language understanding focused on separate models for each modality followed by multi-modal fusion [START_REF] Ren | Exploring models and data for image question answering[END_REF]. In this context, bi-linear fusion is an expressive family of models, which, however, suffers from overparametrization and therefore overfitting. Subsequent work addressed this by creating low-rank decompositions of the fusion tensors, either through Tucker tensor compositions as in MUTAN [START_REF] Ben-Younes | Mutan: Multimodal tucker fusion for visual question answering[END_REF], or block tensor decompositions like in BLOCK [START_REF] Ben-Younes | Block: Bilinear superdiagonal fusion for visual question answering and visual relationship detection[END_REF].

However the general tendency is to move towards holistic architectures, modeling all the interactions between modalities, and also between different objects in the visual modality. Object level reasoning, i.e. the analysis of visual data in the form of a collection of previously detected local entities/objects, has become a general tendency in computer vision beyond VQA, also seen in video analysis [START_REF] Baradel | Object level visual reasoning in videos[END_REF] etc. In this context, the Relation Network [START_REF] Santoro | A simple neural network module for relational reasoning[END_REF] considers all the pairwise interactions between visual objects. [START_REF] Teney | Graphstructured representations for visual question answering[END_REF], [START_REF] Norcliffe-Brown | Learning conditioned graph structures for interpretable visual question answering[END_REF] and [START_REF] Lu | Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-andlanguage tasks[END_REF] apply variants of Graph Convolutional Network [START_REF] Thomas | Semi-supervised classification with graph convolutional networks[END_REF] on visual objects and question words for VQA. [START_REF] Yu | Deep modular co-attention networks for visual question answering[END_REF] and [START_REF] Gao | Dynamic fusion with intra-and inter-modality attention flow for visual question answering[END_REF] go a step further by modeling multimodal interactions via adapting transformer [START_REF] Vaswani | Attention is all you need[END_REF] principles to vision and language. We call them holistic because they consider both intra-modality (inside a modality) and inter-modality (fusion between modalities) relationships.

Multi-task pretraining -A second tendency is the evolution of training from task-specific supervision signals to a set of different losses, which are related to general vision-language understanding, and whose supervision signal can successfully be transferred to different downstream tasks.

This use of auxiliary tasks and knowledge transfer can be performed on both modalities: on language, the use of word embeddings such as GloVe [START_REF] Pennington | Glove: Global vectors for word representation[END_REF] or BERT [START_REF] Devlin | Bert: Pre-training of deep bidirectional transformers for language understanding[END_REF] is frequent. On vision, usual objectives include the use of pre-trained object detectors such as Faster RCNN [START_REF] Shaoqing Ren | Faster r-cnn: Towards real-time object detection with region proposal networks[END_REF], BUTD [START_REF] Anderson | Bottom-up and top-down attention for image captioning and visual question answering[END_REF] for VQA and image captioning, and SCAN [START_REF] Lee | Stacked cross attention for image-text matching[END_REF] for image retrieval.

More recent work shows that a joint pre-training over both modalities can benefit downstream vision-language tasks. This is achieved by setting up strategies to learn a vision-language representation in a multi-task fashion similar to BERT [START_REF] Devlin | Bert: Pre-training of deep bidirectional transformers for language understanding[END_REF] in Natural Language Processing (NLP). Thereby, LXMERT [START_REF] Tan | Lxmert: Learning cross-modality encoder representations from transformers[END_REF] and VilBERT [START_REF] Lu | Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-andlanguage tasks[END_REF] use holistic architectures to learn a vision-language encoder trained on a largescale amount of images-sentences pairs. Their encoder is then transferred to specific vision-language tasks, where they generally achieve SOTA results.

Symbolic representation for visual reasoning -Aside from these approaches, others address the visual reasoning problem by constructing a symbolic view of vision, language and of the reasoning process. Thus, [START_REF] Yi | Neural-symbolic vqa: Disentangling reasoning from vision and language understanding[END_REF] uses reinforcement learning to learn a program generator predicting a functional program from a given question in order to model the reasoning. NSM [START_REF] Hudson | Learning by abstraction: The neural state machine[END_REF] predicts a probabilistic graph from the image to obtain an abstract latent space which is then processed as a state machine.

Our work follows tendencies in SOTA and is based on a holistic joint vision-language encoder with multiple training objectives. We improve on the SOTA by adding an additional objective, which (weakly) supervises the model to align objects and words referring to the same entity. This new objective complements the inductive bias inherent in current models, which allows learning of cross-modality relationships.

Vision-Language Encoder

In this Section, we present our vision-language encoder which is used for learning multimodal embeddings. The encoder is built upon the recent work, and in particular [START_REF] Yu | Deep modular co-attention networks for visual question answering[END_REF]. The overall architecure of our model is presented in Figure 2. Below, we firstly present the embeddings extraction part of our encoder (Section 3.1), and then focus on its central part (Section 3.2).

Input Embeddings

Vision Input -On the vision side, we use an object detector -Faster-RCNN [START_REF] Shaoqing Ren | Faster r-cnn: Towards real-time object detection with region proposal networks[END_REF] -to extract object level-visual features from the input image as in [START_REF] Anderson | Bottom-up and top-down attention for image captioning and visual question answering[END_REF]. Similar to hard attention mechanisms, this enforces the system to reason on object level rather than on the pixel

Word index

Inter-Modality Transformer Alignment matrix between words and visual objects Vision masking [START_REF] Tan | Lxmert: Learning cross-modality encoder representations from transformers[END_REF] Language masking [START_REF] Devlin | Bert: Pre-training of deep bidirectional transformers for language understanding[END_REF] Word Piece level or global level. Thus, for each image we extract No=36 bounding boxes and associated 2048-dimensional visual features:

(f, b) = RCNN(I), (1)
where Language Input -On the language side, sentences are tokenized using the WordPiece tokenizer [START_REF] Wu | Google's neural machine translation system: Bridging the gap between human and machine translation[END_REF]. As common in language processing, a special token [CLS] is added at the beginning of the tokenized sentence, which encodes the multimodal information of the image and sentence. The transformation of this token, performed during the forward pass through the network, corresponds to the prediction of the answer to the task. Tokens are embedded into d-dimensional vectors using a look-up table learned during a pre-training phase which concentrates on language only. The index position of the word is added to the dense vector as a positional encoding in order to obtain index-aware word level embeddings S = [s0, . . . , si, . . . , sN s-1 ].

I

Vision-Language Transformer

The neural model encodes the independent vision and language embeddings (O, S) described above and transforms them as they pass through the network:

O , S = Encoder(O, S), (2)
where (O , S ) are the updated output embeddings. We resort to the widely-used transformer architecture [START_REF] Vaswani | Attention is all you need[END_REF] adapted to visionlanguage problems as in [START_REF] Gao | Dynamic fusion with intra-and inter-modality attention flow for visual question answering[END_REF] and [START_REF] Yu | Deep modular co-attention networks for visual question answering[END_REF].

The vision-language transformer is composed of two self-attention modules of type intra-modality transformer and inter-modality transformer as defined in [START_REF] Gao | Dynamic fusion with intra-and inter-modality attention flow for visual question answering[END_REF]. They take as input one input sequence (in case of intra-modality) or two input sequences (in case of intermodality) and calculate an output sequence:

xi = T (qi, kj, vj) = j αijvj, (3)
where qi, kj and vj are, respectively the query, key and value vectors in R d [START_REF] Vaswani | Attention is all you need[END_REF], which are calculated as linear mappings from the input sequences. Their exact definition depends on the type (inter vs. intra) and is given further below.

The αij represent an attention map, which predicts how the different elements of the input sequences attend to each other:

αij = sof tmaxi( q T i kj √ d ), (4)
where d is the number of dimensions of the embedding space. As in [START_REF] Vaswani | Attention is all you need[END_REF], we use multi-head attention, where the embeddings are split into H parts, transformations are calculated in parallel, and predictions concatenated. Each transformer layer is followed by a residual connection, a layer normalization [START_REF] Lei Ba | Layer normalization[END_REF], and a feed-forward layer.

Intra-Modality Transformer blocks -allow to model the interactions inside one modality. Thus, their query, key and value vectors come from the same modality. They are defined as follows:

s = Ts(s q , s k , s v ) (5) o = To(o q , o k , o v ), (6)
where

x q = W q x, x k = W k x and x v = W v x.
Inter-Modality Transformer blocks -model information flowing between both modalities vision and language. They are defined similarly to the intra-modality transformer but the key and value vectors are crossed between the modalities:

s = Ts←o(s q , o k , o v ) (7) s = Ts(s q , s k , s v ) (8) o = To←s(o q , s k , s v ) (9) o = To(o q , o k , o v ) (10)
Stacked Architecture -The vision-language transformer is built by stacking the previously defined modules as shown in Figure 2: the image/sentence input data is passed through detectors/tokenizers, embedding extractors, several intra-modality transformer blocks and finally several cross-modality transformer blocks. Summarizing, the model contains No←o and Ns←s stacked intra-modality transformer followed by No↔s inter-modality transformers.

The vision-language transformer provides inductive biases for modeling intra-modality relationships (e.g. sentence dependency graph for language, scene graphs for vision) and inter-modality relationships (e.g. vision-language alignment). However, as shown below, inductive biases are not sufficient for learning inter-modal interactions, it is therefore necessary to define adequate supervision signals.

Supervised Objectives

We train the vision-language encoder defined in Section 3 following the recently widely-adapted strategy of combining BERT-like [START_REF] Devlin | Bert: Pre-training of deep bidirectional transformers for language understanding[END_REF] self-supervised signals with task-specific supervision signals, which has been applied to various problems in vision and languagee.g. [33] [23]. We select four supervision signals: vision masking [START_REF] Tan | Lxmert: Learning cross-modality encoder representations from transformers[END_REF], language masking [START_REF] Devlin | Bert: Pre-training of deep bidirectional transformers for language understanding[END_REF], image-sentence matching [START_REF] Lu | Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-andlanguage tasks[END_REF] and visual question answering [START_REF] Antol | Vqa: Visual question answering[END_REF], which are briefly described below.

Vision/Language Masking -This signal aims to supervise the encoder's ability to reconstruct missing information in language and vision. More precisely, we randomly mask each language token (resp. visual object) with a probability of 0.15 and ask the model to predict the missing words (resp. objects). Therefore we add two classifiers -for vision masking 5 and language masking -on top of the vision language encoder and supervise via a cross-entropy loss. [START_REF] Tan | Lxmert: Learning cross-modality encoder representations from transformers[END_REF] proposes to take the object detector prediction as ground truth in order to get over the disparity of visual annotation. Additionally, we also supervise the model to regress the masked objects' Faster-RCNN features via L2 loss.

Image-Sentence Matching -BERT [START_REF] Devlin | Bert: Pre-training of deep bidirectional transformers for language understanding[END_REF] proposes next sentence prediction supervision by asking to predict if two sentences are consecutive in a given text, or randomly sampled from a corpus. Its vision-language equivalent is image-sentence matching [START_REF] Tan | Lxmert: Learning cross-modality encoder representations from transformers[END_REF] [START_REF] Lu | Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-andlanguage tasks[END_REF], where the model has to predict whether a given sentence matches a given image or not. Thus, we randomly replace the image in each sentence-image pair with a probability of 0.5. We add a feed-forward layer on top of the [CLS] output embedding to predict whether the pair matches or not. This global matching is supervised using a binary cross-entropy loss.

Visual Question Answering -Our model is applicable to a wide range of vision-language problems (in Section 6 we evaluate it to two different tasks, namely VQA and Language-driven Comparison of Images). At the same time, independently of the target visionlanguage task, pretraining on helps reasoning as shown in [START_REF] Tan | Lxmert: Learning cross-modality encoder representations from transformers[END_REF]. The VQA task is defined as a classification problem over a set of most frequent answers. In this work, we perform this classification from a prediction head attached to the [CLS] token and supervise it using a cross-entropy loss. 5 It is worth noticing that the vision masking requires to predict both the object classes and its attributes (e.g. color, materials, etc.) F F stands for feed-forward layers.

Weak Supervision of Object-Word Alignment

The presented SOTA vision-language supervision signals -i.e., vision/language masking, image-sentence matching and VQA -have proved their efficiency at encoding rich vision-language embeddings [33] [23]. However, none of them explicitly supervises the objectword alignment. At the same time, matching words and visual objects referring to a same high-level entity is a natural prerequisite for visual reasoning.

The reason why such supervision has not been proposed before is probably that this fine-grained matching property is frequently assumed to be implicitly learned via inter-modality attention modules training. In this work, we claim that the word-object alignment does not necessarily emerge automatically but rather requires explicit supervision.

Vision-Language Alignment Decoder -We propose to add a vision-language alignment decoder on top of the encoder. The whole model is then supervised to predict the object-word alignment matrix A from the encoder's outputs (O , S ). First, (O , S ) are projected into a joint space using a feed-forward layer with layer normalization [START_REF] Lei Ba | Layer normalization[END_REF] and residual connection. We obtain ( Ô, Ŝ) from which we compute A:

A = Ŝ ⊗ Ô √ d , (11)
where ⊗ is the outer product. In other words, the alignment scalar Aij is computed as the scaled-dot-product between each object-word pair (oi, sj), as shown in Figure 3:

Aij = ŝi.ô T j √ d (12)
For each word si we only keep the top-k highest predictions and apply a softmax:

Ai = sof tmaxj(top k (Aij)) (13)
In this work, we empirically set k = 3. This way, we compute from each word a probability distribution Ai over the set of visual objects detected by Faster-RCNN. A high probability Aij means word si and object oj refer to the same high-level entity. The dedicated loss L align is defined using Kullback-Leibler (KL) divergence:

L align = KL(A * , A), (14)
Soft Alignment Score: approximating A * -Let's suppose we have the ground truth object-word pair (si, b * s i ) (cf. Section 6.1). This pair is composed of a word or group of words si taken from the input sentence and a bounding box b * s i indicating the position of the respective object in the image. However we cannot directly use this supervision because both ground truth object-word annotations and the object detector are imperfect. More precisely, (1) the ground truth visual-object annotation is often misaligned with the object detection's bounding box prediction, or (2) the annotated object can simply be not detected at all.

To address this issue we set up a soft-alignment score taking into account both the detection-annotation misalignment and the object detector imperfection. To this end, we consider two criteria: the position one and the semantic one.

Position Criterion -For each ground truth object-word pair (si, b * s i ), we compute Intersection over Union (IoU) between object detector's predicted bounding box bo j and the ground truth object's bounding box b * s i :

P A * ij = IoU (b * s i , bo j ), (15)
A high IoU leads to a high criterion value. Therefore, this criterion permits to give more importance to objects detected in the same image region as the ground-truth object. Semantic Criterion -At the same time, we cannot only rely on positional information. Indeed, we also have to take into account the semantics of the object detector's prediction. This would avoid to align a word with a well-localized but a semantically-different object (according to the detector). Therefore we define the semantic criterion which computes the semantic similarity between a word si and the object's class co j -and attribute ao j -predicted by the detector:

SA * ij = 3 4 S(si, co j ) + 1 4 S(si, ao j ), (16)
where S(•, •) compute the cosine similarity between the GloVe embeddings of the class/attribute names. We bias the similarity toward object class as we empirically found it more relevant than the attribute prediction.

Finally, we combine the two criteria in order to obtain a soft alignment score for each object-word pair in the annotation:

A * ij = normj(P A * ij ) + normj(SA * ij ) 2 (17)
The resulting soft-alignment scores are normalized over the objects such as:

n objects j A * ij = 1 (18)
Hence the ground truth soft alignment score A * i of word si is a probability distribution over the set of visual objects detected by the object detector.

The soft alignment score defined in this Section is by construction approximate. Therefore, we refer to the designed supervision signal as weak.

Experiments

In this section we evaluate the learned vision-language encoder on two tasks, namely VQA [START_REF] Antol | Vqa: Visual question answering[END_REF] and the Language-driven Comparison of Images [START_REF] Suhr | A corpus for reasoning about natural language grounded in photographs[END_REF].

Training Data

Following [START_REF] Tan | Lxmert: Learning cross-modality encoder representations from transformers[END_REF], we construct our dataset as the concatenation of the two public ones, namely: MSCOCO [START_REF] Lin | Microsoft coco: Common objects in context[END_REF] and Visual Genome [START_REF] Krishna | Visual genome: Connecting language and vision using crowdsourced dense image annotations[END_REF]. These datasets provide images annotated with captions. The VQA annotations are taken from three datasets (based on images from either MSCOCO or Visual Genome): VQA 2.0 [START_REF] Goyal | Making the V in VQA matter: Elevating the role of image understanding in Visual Question Answering[END_REF], GQA [START_REF] Hudson | Gqa: A new dataset for real-world visual reasoning and compositional question answering[END_REF] and VG-QA [START_REF] Krishna | Visual genome: Connecting language and vision using crowdsourced dense image annotations[END_REF]. Consequently, our dataset is composed of 9.18M image-sentence pairs (a sentence can be either a caption or a question).

The object-word alignment scores, which are defined in Section 5, are calculated based on the annotations extracted from GQA and Visual Genome. In GQA dataset, salient question words and answers are annotated with visual pointers. A visual pointer consists in a bounding box corresponding to the visual region described by the words composing the question or the answer. Nevertheless, as GQA represents only 12% of the dataset, the use of the GQA pointers would have been insufficient.

To alleviate this issue, we augment the pointer annotation with visual grounded annotations from Visual Genome. Every Visual Genome image is accompanied with visual region descriptions forming (description, bounding box) pairs. Unlike in GQA, descriptions are full descriptive sentences and not small groups of words. Therefore, the so obtained pointer is less discriminative towards the language part. Therefore, we choose to combine these descriptions in order to obtain sentences with one, two or three pointers. For instance the two descriptions "the cat playing near the tree" and "the yellow bird" become "the cat playing near the tree and the yellow bird", with the associated bounding boxes.

All in all, by combining annotations from GQA and Visual Genome, we gather roughly 6M image-sentence pairs annotated with pointers. In other words, about 70% of the total number of the imagesentence pairs in the dataset have fine-grained object-word alignment annotations.

Evaluation Tasks

To evaluate the reasoning quality of our vision-language encoder supervised with the object-word alignment, we evaluate it on two tasks requiring to reason over image and text.

The first one is the VQA task. It consists in predicting the answer asked about an image. This task is challenging as it requires a highlevel understanding of vision and language. For evaluation, we select the most recent and, arguably, the most challenging VQA-dataset today, namely GQA. As GQA is already used during the visionlanguage encoder training, we do not find it necessary to finetune our model on the datatset.

The second is the Language-driven Comparison of Images. We choose the Natural Language for Visual Reasoning (NLVR2) dataset [START_REF] Suhr | A corpus for reasoning about natural language grounded in photographs[END_REF]. NLVR2 is composed of triplets (img1, img2, sentence) where img1 and img2 are two images and sentence is a sentence describing one or both images. The goal is to predict if the sentence is true. It is worth noticing that NLVR2 data is not viewed during the encoder training, therefore it truly evaluates the generalization capacity of our method. We use the same finetuning strategy as in [START_REF] Tan | Lxmert: Learning cross-modality encoder representations from transformers[END_REF]. Thus we concatenate the two encoder's output [CLS] embeddingsobtained with (img1, sentence) and (img2, sentence) pairs -and pass them through a feed-forward layer. We then use a binary crossentropy loss.

Table 1. Evaluation of the proposed object-word alignment weak supervision on the GQA [START_REF] Hudson | Gqa: A new dataset for real-world visual reasoning and compositional question answering[END_REF] dataset. The presented results are calculated on the dataset's test-std split. The GQA's accuracy is presented in the last column. The exact definitions of all other (auxiliary) metrics can be found in [START_REF] Hudson | Gqa: A new dataset for real-world visual reasoning and compositional question answering[END_REF].

Models

Results

Training Details -We train our vision language encoder using Adam optimizer [START_REF] Diederik | Adam: A method for stochastic optimization[END_REF] during 20 epochs. However, the VQA supervision is only added after 10 epochs, following [START_REF] Tan | Lxmert: Learning cross-modality encoder representations from transformers[END_REF]. We set the learning to 10 -4 with warm starting and learning rate decay. The batch size is 512. On the architecure side, the number of stacked inter-and intra-modality transformers is No↔s = 5, No←o = 5 and Ns←s = 9. The encoder hidden's vectors are of dimension d = 768 and each attention layer has H=12 heads. Moreover, to reduce computation, we set the maximum sentence length to Ns = 20 tokens and the number of visual objects to No = 36. Training is done on four V100 GPUs.

For NLVR2 [START_REF] Suhr | A corpus for reasoning about natural language grounded in photographs[END_REF], we finetune during 4 epochs using Adam optimizer [START_REF] Diederik | Adam: A method for stochastic optimization[END_REF]. The learning rate is set to 5 * 10 -5 and the batch size is 32. We only supervise with the task-specific binary objective, i.e., we drop all the supervision signals used for encoder training. Visual Question Answering -Table 1 compares the results of applying our vision-language encoder on the VQA task versus the recent published works. As one may observe, our model obtains the 2nd-best SOTA-result, just after the NSM model [START_REF] Hudson | Learning by abstraction: The neural state machine[END_REF]. The latter is fundamentally different from our approach (contrary to NSM, our model operates on raw images rather than on the scene graphs). Moreover, it is important to highlight that, unlike previous work, our model has not been finetuned on the target dataset making the obtained result even more significant.

In order to quantify the impact of the our object-word alignment weak supervision on the VQA task, we evaluate the two versions of our model, with and without the proposed loss, on the GQA dataset. The results are reported in Table 2. One may observe that the proposed weak supervision boosts the accuracy with +5.6 points. Moreover, when we focus on the metric which explicitly measures the reasoning capacity of the model, namely the consistency, our weaklysupervised alignment allows to gain more than +10 points. This demonstrates that, by enforcing the model to explicitly align words with visual objects, we obtained a finer multimodal representation.

Natural Language for Visual Reasoning (NLVR2) -As shown Table 3. Evaluation of the proposed object-word alignment weak supervision on the NLVR2 evaluation splits. Models marked with * have been ran by the authors of [START_REF] Suhr | A corpus for reasoning about natural language grounded in photographs[END_REF].

Models

Dev. Test-P MAC* [START_REF] Drew | Compositional attention networks for machine reasoning[END_REF] 50.8 51.4 FiLM* [START_REF] Perez | Film: Visual reasoning with a general conditioning layer[END_REF] 51.0 52.1 CNN+RNN* [START_REF] Suhr | A corpus for reasoning about natural language grounded in photographs[END_REF] 53.4 52.4 MaxEnt [START_REF] Suhr | A corpus for reasoning about natural language grounded in photographs[END_REF] 54.1 54.8 LXMERT [START_REF] Tan | Lxmert: Learning cross-modality encoder representations from transformers[END_REF] 74.9 74.5 ours 75.8 75.5

in Table 3, our method outperforms the published 6 SOTA accuracy on NLVR2 with a gain of +1 point. Furthermore, we have performed the same ablation analysis as for the VQA task (i.e. with and without the object-word alignment weak supervision), and the obtained results are summarized in Table 4. These results are coherent with those calculated on the VQA task confirming the advantage of the proposed supervision. Note that the scores in Table 4 are reported both for unbalanced and balanced subsets of the NLVR2 dataset. This split takes into account the visual biases present in the dataset. The benefit of our fine-grained alignment supervision method is constant between both subsets, showing that the gain is not bias-related. biases in the data) can be difficult. The visualizations in this Section are obtained from the dev-test split of the GQA [START_REF] Hudson | Gqa: A new dataset for real-world visual reasoning and compositional question answering[END_REF] dataset. We inspect the attention maps inside the inter-modality transformers, which illustrates the information flow between the two modalities (vision and language). Generally, attention maps convey information on the importance that a neural map poses on local areas in input or activations. In the particular case of our model, the intermodalilty attention map visualize how modalities are fused by the model, as they give weight to outputs for a given word as a function of a given object (or vice-versa).

Following equation (4), we visualize the values αij, showing the attention given to the pair (si, oj). We visualize the map of the 4 th inter-modality transformer and sum the maps over the 12 parallel attention heads, comparing the maps of our proposed model with and without the proposed object-word alignment supervision in Figure 4.

The effectiveness of the new object-word alignment objective is corroborated by attention units which are higher for object-word pairs referring to the same entity in our model. We observe a radically different behavior in the baseline's attention maps, where attention is less-fine grained: roughly uniform attention distributions indicate that the layer outputs of all words attend to roughly the same objects.

Caveat: we do not want to imply, that the exact word-object alignment in the inter-modality layer is indispensable for a given model to solve a reasoning task, as a complex neural network can model relationships in the data in various different layers. However, we do argue, that some form of word-object alignment is essential for solving vision-language tasks, as the model is required to query whether concepts from the question are present in the image, and eventually query their relationships to other concepts. Inductive bias has been added to model for this type of reasoning in the form of intermodality layers, and it is therefore natural to inspect whether this cross-attention emerges at this exact place. We would also like to point out that we do not force or favor word-object alignment at a specific layer, as our proposed objective is injected through a new module attached to the inter-modality layer (see Figure 2). The attention maps show that the objective is successfully propagated from the new alignment head to the inter-modality layer.

Conclusion

In this work, we design a vision-language encoder and train it with a novel object-word alignment weak supervision. To this end, we carefully design a soft alignment signal taking into account both spatial and semantic alignment between the words and the detected visual objects. We empirically show the benefit of this new supervision on two tasks requiring to reason over images, namely the VQA and the Language-driven Comparison of Images on which we obtain the SOTA-level accuracies. We also provide a qualitative visualization of the attention distributions of our model, showing that attention units are higher for object-word pairs referring to the same entity, and that the proposed object-word alignment does not emerge naturally without supervision. Future work will explore the application of this weak supervision signal to other vision-language tasks, including image retrieval.

Figure 1 .

 1 Figure 1. In the context of vision+language problems, we show that the alignment of visual objects to words in an input sentence does not naturally emerge from task oriented losses and propose a new auxiliary training objective addressing this problem. The presented image and question are taken from GQA [14] dataset.

Figure 2 .

 2 Figure2. Architecture of our vision-language encoder (Section 3) and the respective supervision tasks (Sections 4 and 5). From left to right: (left), image and sentence are embedded into input embeddings (O, S) (Section 3.1); (middle), they are then processed by the vision-language transformer (Section 3.2) modeling both intra-and inter-modality interactions in order to obtain multimodal embeddings (O , S ); (right), the encoder is supervised using objectives introduced in Section 4 and our soft object-word alignment proposed in Section 5.

  is the input image and f ={f0, ..., fN o -1} and b={b0, ..., bN o -1} are, respectively, the dense visual features and the bounding boxes detected for objects. Box and dense vectors are fused to obtain position-aware object level embeddings O = [o0, . . . , oi, . . . , oN o -1].

Figure 3 .

 3 Figure3. The proposed vision-language alignment decoder and the respective weakly-supervised loss. In this illustration, we present the alignment prediction A i between one word S i and the visual objects O .F F stands for feed-forward layers.

Figure 4 .

 4 Figure 4. Visualization of the attention maps of the penultimate (=4th) inter-modality transformer. Word-object alignment does not emerge naturally for the baseline (without object-word alignment supervision), whereas our model with the proposed weakly-supervised objective learns to pay strong cross-attention on co-occurring combinations of words and objects in the scene. In the attention maps, rows represent words and columns represent visual objects. For the sake of visibility, we display the bounding box of the detected object with the highest activation regarding to the selected word. The predicted answer (underlined) is written after the question. Its corresponding language token is [CLS], i.e., the first row in attention maps.

  • On the task of Language-driven Comparison of Images, requiring to reason over two images and one sentence, our proposed model outperforms the current SOTA model on the challenging NLVR2 dataset.• We show visualizations of attention maps, which corroborate the claim that word-object alignment does not naturally emerge from task losses, while it is discovered by our weak supervision signal.

  Binary Open Validity Plausibility Consistency Distribution Acc.

	Human [14]	91.2	87.4	98.9	97.2	98.4	-	89.3
	BUTD [1]	66.6	34.8	96.2	84.6	78.7	6.0	49.7
	MAC [13]	71.23	38.9	96.2	84.5	81.6	5.3	54.1
	LCGN [11]	73.7	42.3	96.5	84.8	84.7	4.7	57.0
	LXMERT [33]	77.2	45.5	96.4	84.5	89.6	5.7	60.3
	NSM [12]	78.9	49.3	96.4	84.3	93.3	3.7	63.2
	ours	76.9	46.1	96.3	84.7	89.7	5.3	60.5

Table 2 .

 2 Impact of the proposed object-word alignment weak supervision on the VQA task. The presented results are calculated on the GQA[START_REF] Hudson | Gqa: A new dataset for real-world visual reasoning and compositional question answering[END_REF] test-std split.

	Models	Consistency Accuracy
	ours (w/o alignment supervision)	79.5	54.9
	ours (with alignment supervision)	89.7	60.5

Table 4 .

 4 Impact of the proposed object-word alignment weak supervision on the Visual Reasoning grounded by Language task. The presented results are calculated on the Test-P part of the NLVR2 dataset.

	Models	Test-P	Unbalanced Balanced
	ours (w/o alignment supervision)	74.5%	76.0%	73.1%
	ours (with alignment supervision) 75.5%	77.2%	74.5%
	7 Visualizing Reasoning
	The reasoning capabilities of high-capacity deep neural networks are
	notoriously difficult to interpret, as inputs and intermediate activa-
	tions are embedded in high-dimensional spaces in non trivial appli-
	cations. Vision-language models are no exceptions, therefore we pro-
	pose visualizations of some of the key activations of the proposed
	model. Such visualizations -when wisely chosen -can be a step
	toward more interpretable models, especially in the field of visual
	reasoning, where distinguishing real reasoning (i.e. which follows
	causal chains) from educated guesses (i.e. exploiting subtle statistic

What material is the crosswalk in front of the stores? concrete What material is the crosswalk in front of the stores? concrete What material is the crosswalk in front of the stores? concrete What material is the crosswalk in front of the stores? brick What material is the crosswalk in front of the stores? brick What material is the crosswalk in front of the stores? brick Ours Baseline What is the vehicle that is in front of the flag? van Ours Baseline What is the vehicle that is in front of the flag? van What is the vehicle that is in front of the flag? van What is the vehicle that is in front of the flag? car What is the vehicle that is in front of the flag? car What is the vehicle that is in front of the flag? car

At the time of the writing, an unpublished work, called UNITER[START_REF] Chen | Uniter: Learning universal image-text representations[END_REF], reported a better result on NLVR2.