Rennan Cabral

Angelo Jamil Maia

Ygor Jacques

Fábio Agra Bezerra Da Silva

Clístenes Farias Amorim

Williams Araújo

Do Nascimento

Tales Tiecher

Olivier Evrard

Adrian L Collins

Caroline Miranda Biondi

Yuri Jacques

Ygor Jacques

Agra Bezerra

Rennan Cabral Nascimento

Angelo Jamil Maia

Ygor Jacques

Williams Araújo Do Nascimento

Yuri Jacques

Sediment source apportionment using geochemical composite signatures in a large and polluted river system with a semiarid-coastal interface, Brazil

Keywords: soil erosion, sediment source fingerprinting, multiple sources framework, Bayesian un-mixing model

Introduction

In Brazil, high erosion rates have often been associated with changes in land cover, due to the expansion of agriculture, and with the implementation of inappropriate practices of soil management, such as up and down slope farming without terraces and use of heavy agricultural machinery [START_REF] Didoné | Quantifying soil erosion and sediment yield in a catchment in southern Brazil and implications for land conservation[END_REF]. Events with high rainfall intensities occur frequently in this part of the world [START_REF] Anache | Runoff and soil erosion plot-scale studies under natural rainfall: A meta-analysis of the Brazilian experience[END_REF]. Combined, these conditions can generate elevated volumes of surface runoff and high sediment loads in river systems across the country [START_REF] Molisani | Water discharge and sediment load to Sepetiba Bay from an anthropogenically-altered drainage basin, SE Brazil[END_REF][START_REF] Strauch | The impact of Best Management Practices on simulated streamflow and sediment load in a Central Brazilian catchment[END_REF]. In this context, it is fundamental to obtain reliable information on the origin of sediment in order to improve our understanding of key controlling processes and to support targeted mitigation plans.

The sediment fingerprinting approach is increasingly used worldwide [START_REF] Haddadchi | Sediment fingerprinting in fluvial systems: review of tracers, sediment sources and mixing models[END_REF][START_REF] Collins | Sediment source fingerprinting: benchmarking recent outputs, remaining challenges and emerging themes[END_REF]. In brief, this approach involves the comparison of conservative bio-physico-chemical properties of sediment source and target sediment samples to determine the relative contributions of individual sources through the application of statistical tests and numerical mass balance modelling [START_REF] Walling | Fingerprinting suspended sediment sources in the catchment of the River Ouse, Yorkshire, UK[END_REF][START_REF] Collins | Apportioning catchment scale sediment sources using a modified composite fingerprinting technique incorporating property weightings and prior information[END_REF]. The critical requirements for robust applications of the fingerprinting approach have been discussed in [START_REF] Collins | Sediment source fingerprinting as an aid to catchment management: a review of the current state of knowledge and a methodological decision-tree for end-users[END_REF].

In large river catchments, the heterogeneous environmental features (i.e., geology, pedology, geomorphology and climate) can challenge the robust geochemical characterization of potential individual sediment sources. This, in turn, can undermine robust land use-based source discrimination and consequently generate more substantive errors for sediment source apportionment [START_REF] Pulley | The impact of catchment source group classification on the accuracy of sediment fingerprinting outputs[END_REF]. To avoid these difficulties, sediment source apportionment based on soil type may provide a pragmatic alternative, since soils express geochemical signatures inherited from the parent material and pedogenetic processes. For instance, soils with elevated weathering rates tend to exhibit higher contents of low mobility chemical elements, such as Al, Ti, Si, Th, Zr, and Fe, and lower contents of high mobility chemical elements, such as K, Na, Cl, Mg, and Ca, compared to less weathered soils [START_REF] Silva | Concentrations of major and trace elements in soil profiles developed over granites across a climosequence in northeastern Brazil[END_REF]. Nevertheless, the geochemical potential for source classification according to soil type has received little attention in the literature so far in comparison with land use [START_REF] Evrard | Tracing sediment sources in a tropical highland catchment of central Mexico by using conventional and alternative fingerprinting methods[END_REF][START_REF] Lepage | Investigating the source of radiocesium contaminated sediment in two Fukushima coastal catchments with sediment tracing techniques[END_REF][START_REF] Gall | Quantifying sediment sources in a lowland agricultural catchment pond using 137Cs activities and radiogenic 87Sr/86Sr ratios[END_REF]Silva et al., 2018a). Another alternative is the recent work of [START_REF] Batista | Using pedological knowledge to improve sediment source apportionment in tropical environments[END_REF], who developed a regional source classification based on the interpretation of lithological and soil maps, and the a priori knowledge of erosion processes in a large river basin of Minas Gerais, Southeastern Brazil. They divided the catchment into different parts (upper, middle and lower), revealing more robust discriminations (90% of source samples classified correctly) using the geochemical composition of particles <63 μm.

Most importantly, few studies have assessed different strategies to classify and quantify different sediment sources in the same river catchment -a relevant step to providing welldesigned policies and control strategies for protecting soil and water resources, especially in large complex catchments [START_REF] Pulley | The impact of catchment source group classification on the accuracy of sediment fingerprinting outputs[END_REF].

The Ipojuca River catchment is considered to be one of the most polluted river systems and the third worst for water quality indices in Brazil (IBGE, 2015). Surveys have linked high loads of metals and natural radionuclides transported in suspended sediment to natural hotspots and to battery factories, textile industries and some municipalities that directly discharge non-treated wastewater into the upstream river [START_REF] Lima Barros | Modelling of point and diffuse pollution: application of the Moneris model in the Ipojuca river basin, Pernambuco State, Brazil[END_REF][START_REF] Silva | Trace element fluxes in sediments of an environmentally impacted river from a coastal zone of Brazil[END_REF]Silva et al., 2018b;[START_REF] Nascimento | Thorium content in soil, water and sediment samples and fluvial sediment-associated transport in a catchment system with a semiarid-coastal interface, Brazil[END_REF].

The construction of a Harbour Complex and dams near the estuary in the 1980s have decreased the hydrological connectivity at the estuary-sea interface. Accordingly, the control of erosion and sediment delivery in these fragile ecosystems requires a better understanding of sediment transfer processes in these areas. To this end, the current research provides one of the first sediment fingerprinting studies conducted in the Northeastern region of Brazil. In this work, we assessed the apportionment patterns of different sediment source typologies (soil type, land use, and catchment zone) in the Ipojuca River catchment using a suite of geochemical tracers. The implications of these novel results for supporting the environmental recovery of this polluted river basin and its coastal environments are then discussed.

Materials and methods

Study area

The Ipojuca River catchment (08 ° 09 ′ 50 ″ -08 ° 40 ′ 20 ″ S and 34 ° 57 ′ 52 ″ S and 34 ° 57 ′ 52 ″ -37 ° 02 ′ 48 ″ W; Fig. 1) extends from the western semiarid zone to the humid coast of Pernambuco State, Northeastern Brazil. The total area of the catchment is ~3435 km² and the main watercourse is 324 km long (CONDEPE / FIDEM, 2005). The depth and width of downstream cross-sections of the main watercourse vary from 0.8 to 2.4 m and 21.8 to 30.3 m, respectively. Flow rates and suspended sediment discharge for the same section vary from 1.2 to 25 m³ s -1 and 7.6 to 669 Mg day -1 in periods of low and high flow rate, respectively [START_REF] Silva | Trace element fluxes in sediments of an environmentally impacted river from a coastal zone of Brazil[END_REF].

The mean annual rainfall ranges from 600 to 800 mm in the semiarid upstream part with high spatial and temporal variability, and from 1800 to 2400 mm in the downstream coastal zone where the rain is distributed more evenly throughout the autumn-winter months. These different patterns of rain distribution promote an intermittent fluvial regime in the upstream reaches, and a perennial regime in the middle to downstream reaches. The local average air temperature varies from 25 to 28 °C; typical of Brazilian tropical conditions (CONDEPE / FIDEM 2005). Undulating or very undulating slopes are found in zones covering ~31.5% and ~12.4% of the total catchment surface area, respectively (CPRM, 2015). A large transition zone separates the upper and lower regions of the study basin, characterized by the sloping eastern escarpments of the Borborema granite-gneiss massif. The soil parent materials are mainly metaluminous granites (56%), orthogneisses (31%) and biotite-muscovite gneisses (8%) [START_REF] Silva | Trace element fluxes in sediments of an environmentally impacted river from a coastal zone of Brazil[END_REF] (Fig. 1a). The main soil classes found in the study basin are Entisols (~37%), Alfisols (~18%), Ultisols (~32%) and Oxisols (~9%) (Fig. 1b). The areas under vegetation (natural and semi-natural) and cropland cover 18% and 19% of the Ipojuca River catchment area, respectively (CONDEPE / FIDEM 2005).

The Caatinga vegetation is found in the preserved and semi-preserved areas of the upstream region, with typical endemic plant species resistant to arid conditions (Supplementary Figure 1). The natural vegetation in the lower catchment is the Atlantic Forest, which has been extensively degraded due to the expansion of agriculture.

Nowadays, this region is characterized by small and isolated fragments of vegetation, often distant from the river network as a result of the limited preservation of riparian vegetation as well as the expansion of sugarcane (Saccharum officinarum) monoculture, which covers almost the entire agricultural landscape in this portion of the study basin.

Collection of source and sediment samples and sediment source classification

Soil samples (n = 207; see Fig. 1c for distribution) were collected from the main potential sources of sediment. These collection points were defined by a preliminary assessment of sediment delivery pathways using satellite images, soil, geology and slope maps, as well as field observations. Each sample (Fig. 2) was composed of 15 subsamples collected at 0-5 cm depth for superficial sources and from the lower parts of channel bank profiles. Three source classification schemes were generated from the same sample set, based, respectively, on the distribution of land uses (1), soil classes (2), and the regional and environmental contrasts between the upper and lower portions of the study basin (3):

1. The first typology was based on land uses and covers found in the study basin. The land uses selected and the corresponding numbers of samples (Fig. 2 Target suspended sediment samples (n = 4; Fig. 1b) were collected in a crosssection located at the outlet of the main river system, by means of time-integrated samplers [START_REF] Phillips | Time-integrated sampling of fluvial suspended sediment: a simple methodology for small catchments[END_REF] installed between April 2019 and February 2020, in order to capture the full range of hydrological responses (Table 1). Target bed sediment samples (n = 6) were also collected in October 2019 at the overall outlet in order to permit comparison of the source apportionment results for two target sediment types. Recovery of this material was performed by scraping the accumulated sediment, taking care not to exceed a depth of 5 cm.

Sample preparation and particle size characterization

The source and target sediment samples were respectively air-dried and dried in a forced-circulation stove at 50°C. All samples were then gently disaggregated and sieved through a 2-mm mesh. Particle size distribution analysis was conducted to define the target fractions for sediment fingerprinting. Furthermore, organic matter in 2 g of each sample was burned using 20 ml H2O2 (25%) in a stove at 50°C for 24 to 48 h, and the dispersion of the particles was achieved by the addition of 10 ml NaOH (6%) and agitation at 130 rpm for 12 h. The absolute size distribution of particles was determined after dispersion in a liquid analyzer (Microtrac S3500), suitable for the size range 20-nm to 2mm. In addition to the samples collected specifically for sediment fingerprinting, 25 composite samples from the superficial layer (0-20 cm) of reference soils in the study basin collected for previous work [START_REF] Silva | Trace element fluxes in sediments of an environmentally impacted river from a coastal zone of Brazil[END_REF] were used to support the particle size characterization of the main soil classes in the area. The particle size distribution was obtained according to [START_REF] Gee | Topp GC (4 ed) Methods of soil analysis: Physical methods[END_REF].

The average diameter and D90 of the target suspended sediment samples from the outlet of the Ipojuca River (Supplementary Figure 2) were 12.2 μm and 24.3 μm, respectively. The silt fraction represented approximately 85% of the particle size composition of these samples. Therefore, all geochemical analyses were conducted on the <32 μm fraction, with the aim to optimize the direct comparison of the properties of the sources and target sediment.

Geochemical analysis

The contents of 20 metals were used as potential fingerprint properties: Al, Ba, Ce, Cr, Fe, La, Nd, Ni, Pb, Pr, Sc, Sm, Sn, Sr, Th, Ti, V, Y, Zn and Zr. The extraction of these metals followed the method proposed by Estévez [START_REF] Alvarez | Nuclear and related analytical methods applied to the determination of Cr, Ni, Cu, Zn, Cd and Pb in a red ferralitic soil and Sorghum samples[END_REF]. First, 0.5 g of each source and target sediment sample was pre-digested in 10 ml of HF in a rest system for 12 h. Then, the samples were put in vessels with 5 ml of HNO3 and 3 ml of The recovery rates of the analyzed metals ranged from 48% (Zn) to 107% (Ni). More than 80% of the chemical elements showed recovery rates ranging from 80% to 100%.

Selection of tracers and apportionment of sediment sources

The successive steps for tracer selection were: 3; chain length = 3,000,000; burn = 2,700,000; thin = 300. The outputs from the models were rejected if a variable was above 1.01 for the Gelman-Rubin diagnosis. The averages, standard deviations, confidence intervals, and posterior correlations of source contributions were also estimated. The accuracy of the source estimates was evaluated using virtual mixtures, which were generated to compare known and predicted source contributions, using the composite signatures selected by LDA [START_REF] Phillips | Source partitioning using stable isotopes: coping with too many sources[END_REF]:

𝑦 = ∑ 𝑘 𝑖 𝑓 𝑖 𝑛 𝑖=1 (1)
in which y is the virtual mixture, k is the result of the individual source using mixtures of target sediments, n is the number of tracers, i is the tracer used, and f is the individual source. The accuracy of the predicted source estimates was evaluated using the root mean square error (RMSE) and mean absolute error (MAE), viz.:

𝑅𝑀𝑆𝐸 = √ ∑ (𝑌 𝑝𝑟𝑒𝑑𝑖𝑐𝑡𝑒𝑑 -𝑌 𝑘𝑛𝑜𝑤𝑛 ) 2 𝑛 𝑖=1 𝑛 (2) 𝑀𝐴𝐸 = ∑ |𝑌 𝑝𝑟𝑒𝑑𝑖𝑐𝑡𝑒𝑑 -𝑌 𝑘𝑛𝑜𝑤𝑛 | 𝑛 𝑖=1 𝑛 (3)
in which Ypredicted is the relative contribution of the sediment source predicted by the model, Yknown is the known relative contribution of the source to the virtual mixture and n represents the total number of sediment sources. All statistical procedures were undertaken using R software (version 3.6.1, R Core team, 2021).

Results

Exploratory apportionment of sediment sources

The discrimination potential of regional sources (upstream and downstream), soil classes (Ultisol, Entisol (Aquent), Oxisol, Entisol and Alfisol) and land uses (Caatinga, channel banks, sugarcane and unpaved roads) were evaluated using all geochemical tracers by forward stepwise LDA (p < 0.1) (Fig. 3). This analysis showed significant overlaps between the ellipses of Ultisols and Entisols (Aquents) (Fig. 3a) as well as between those of channel banks and unpaved roads (Fig. 3c). Thus, the final sources were reclassified by merging these groups, resulting in the Ultisols + Entisols (Aquents) (Fig. 3b) and channel banks + unpaved roads (Fig. 3d). Some previous sediment fingerprinting studies have clearly pointed to improved discrimination and apportionment modelling from reclassifying initial source categories [START_REF] Barthod | Selecting color-based tracers and classifying sediment sources in the assessment of sediment dynamics using sediment source fingerprinting[END_REF][START_REF] Lizaga | Legacy of historic land cover changes on sediment provenance tracked with isotopic tracers in a Mediterranean agroforestry catchment[END_REF].

Here, in general, this merging ensured increased discriminatory power (Fig. 3b and3d), exemplified by the rates of correctly classified samples (CCS) on the basis of soil classes improving from 66% to 90% and in the case of land uses, from 59% to 77%, respectively.

The first and second linear discriminant functions accounted for much of the variance in these source groups, explaining 97% and 100% of the total variance in the soil classes (LD1 = 78%, LD2 = 19%; Fig. 3b) and land uses (LD1 = 63%, LD2 = 37%; Fig. 3d),

respectively. LDA revealed high potential for differentiation between the two regional sources (Wilks' Lambda = 8; CCS = 99%).

Tracer selection

All analyzed elements in the < 32 μm fraction passed the range test (Table 2) and were therefore considered conservative during erosion and sediment transport to, and through, the river network (Supplementary Figures 3 and4). Four and six elements (Table 3) failed to differentiate the samples classified on the basis of soil classes (Ce, Nd, Sm, and Zn) and regional sources (Ce, Nd, Pr, Sm, Th, and Zn). In contrast, the concentrations of all the tracers were significantly different among the sources classified on the basis of land use. The LDA forward stepwise analysis selected sets of eight tracers for the regional sources (Al, Sr, Y, Ti, Pb, La, Fe, and Zr), ten for the soil classes (Al, Ba, Ni, Ti, La, Pr, Sr, Zr, Th, and Sc) and six for the land use sources (Al, Ce, Ti, V, Pb, and Sr) (Table 4).

These sets explained 75% of the differences between regional sources, 81% in the case of the soil classes, and 45% for the land uses, as per Wilks' Lambda cumulative (LW), and were able to correctly classify 97% of the regional samples, 86% of the soil class samples and 72% of the land use samples. Individual discrimination error rates ranged from 4% to 17% (regional sources), 20% to 37% (soil classes), and 37% to 42% (land uses) (Supplementary Figures 5 and6). Only Al, Ti and Sr were selected in all three approaches, highlighting mainly the high individual discriminatory power of Al and Ti for soil classes and regional sources. The source estimates predicted by MixSIAR were similar to the known proportions used to generate virtual mixtures, suggesting acceptable accuracy (Table 5).

Estimated sediment source contributions

The RMSEs and MAEs ranged from 0.1-6.0% and 0.1-4.9%, respectively, and their respective overall means were 3.6% and 3.3%. Correlations among the estimated posterior contributions were generally weak; -0.30 for soil classes and -0.42 for land uses (Supplementary Figures 7, 8, and9). However, strong negative correlations were observed between the contributions of Ultisol + Entisol (Aquent) and Oxisol (-0.71) and sugarcane, channel banks + unpaved roads (-0.95) and upstream and downstream (-1.00).

(a) (b) (c)

Discussion

Tracer conservation and sediment source discrimination

All tracers passed the range test using the <32 μm fraction of source and target sediment samples collected in the Ipojuca River catchment. Nevertheless, it is important to bear in mind the limitations of such tests [START_REF] Batista | Using pedological knowledge to improve sediment source apportionment in tropical environments[END_REF][START_REF] Collins | Sediment source fingerprinting: benchmarking recent outputs, remaining challenges and emerging themes[END_REF][START_REF] Ramon | Combining spectroscopy and magnetism with geochemical tracers to improve the discrimination of sediment sources in a homogeneous subtropical catchment[END_REF]. Elements with higher enrichment potential linked to anthropogenic activities in the study basin, such as Ca, Mg, Mn, P, and K, were disregarded in this study.

A high degree of anthropogenic enrichment of Pb, Ni, and Zn in downstream river sediments has been reported for the study river by previous work [START_REF] Silva | Bedload as an indicator of heavy metal contamination in a Brazilian anthropized watershed[END_REF]. In general, however, different trace elements exhibit limited enrichment in sediments at the downstream sites in the study area [START_REF] Silva | Bedload as an indicator of heavy metal contamination in a Brazilian anthropized watershed[END_REF]Silva et al., 2018b;[START_REF] Nascimento | Thorium content in soil, water and sediment samples and fluvial sediment-associated transport in a catchment system with a semiarid-coastal interface, Brazil[END_REF].

Although the Ultisols and Entisols (Aquents) generally show distinct geochemical signatures, the surface samples of these soil classes expressed strong similarity in the Ipojuca River catchment (Fig. 3b). Ultisols and Entisols (Aquents) are distributed in adjacent areas of the east-central portion of the study basin (Fig. 1b), indicating that these soils may be derived from geochemically similar parent materials. In addition, based on field observations, distribution maps (Fig. 1), and the particle size of these river basin sources (Supplementary Table 1), the Entisols (Aquents), located on the lower elevations of the local geomorphological interfluves, may be the receptors of material eroded in the steeper neighbouring areas represented by Ultisols (Fig. 1b). As a result, it is possible that some samples collected in areas represented by the Entisols (Aquents) may represent mixtures. Furthermore, the similarity between channel banks and unpaved roads highlights the lack of substantial differences in the subsurface geochemical signatures of the soils where these sources are found. Overall, unpaved roads represent subsurface sections that have been exposed at the foot of many steeper slope reliefs in the study basin, whilst channel banks have been reported to exhibit similar signatures to other nearby sources in some river catchments [START_REF] Vale | Storm event sediment fingerprinting for temporal and spatial sediment source tracing[END_REF].

Only Al, Ti, and Sr were selected in the final composite signatures for discriminating the catchment sediment sources using all three source classifications, reflecting the high KW H-values for source discrimination by these individual tracers.

Aluminum and Ti can express key differences in association with the development stage of soils. Al, for example, is widely used in indices of chemical weathering, such as the CIA method [START_REF] Nesbitt | Early Proterozoic climates and plate motions inferred from major element chemistry of lutites[END_REF]. The tendency here is that higher Al and Ti contents are observed in more weathered and developed superficial horizons due to the high strength and low mobility of these elements [START_REF] Koiter | Investigating the role of connectivity and scale in assessing the sources of sediment in an agricultural watershed in the Canadian prairies using sediment source fingerprinting[END_REF][START_REF] Smith | Evaluating tracer selection for catchment sediment fingerprinting[END_REF].

The mean Al and Ti concentrations measured in our samples (Table 6) distinguished upstream (73.5 g kg -1 and 3550.5 mg kg -1 ) from downstream sources (126.5 g kg -1 and 6567.9 mg kg -1 ) and Alfisols (72. 7 g kg -1 and 3427.4 mg kg -1 ) and Entisols (99.7 g kg -1

and 4901.3 mg kg -1 ) from Ultisol-Entisols (Aquent) (127.2 g kg -1 and 6662.6 mg kg -1 )

and Oxisols (119.2 g kg -1 and 6035.3 mg kg -1 ). However, this was not the case for the land use sources when considering sugarcane and Caatinga as surface sources and channel banks + unpaved roads as subsurface sources: sugarcane (118.6 g kg -1 and 5759.1 mg kg - 1 ), Caatinga (71.1 g kg -1 and 3130.9 mg kg -1 ) and channel banks + unpaved roads (119.0 g kg -1 and 5268.7 mg kg -1 ). With respect to Al, this is because channel banks + unpaved roads represent both the more developed soils (Ultisol-Entisol (Aquent) and Oxisol)

downstream and the less developed soils (Entisol and Alfisol) upstream. Another factor reducing source discrimination is the disturbance and incorporation of the surface and subsurface soil layers during sugarcane management in the study catchment. Iron, selected only in the composite signature for discriminating regional sources, was expected to exhibit similar discriminant patterns as Al and Ti for the soil classes because of the potential for oxide accumulation in the more developed soils found downstream. The average concentrations of Ba, Sr and Zr ensured discrimination between Ultisol + Entisols (Aquent) (525.6 mg kg -1 , 108.6 mg kg -1 and 630.3 mg kg -1 ) and Oxisols (277.8 mg kg -1 , 71.3 mg kg -1 and 390.5 mg kg -1 ); Sr and Ce improved discrimination between sugarcane (118.0 mg kg -1 and 105.3 mg kg -1 ) and channel banks + unpaved roads (148.1 mg kg -1 and 124.9 mg kg -1 ), although this benefit for source discrimination was not so pronounced in the case of soil classes (Supplementary Tables 3 and4).

Table 6. Mean concentrations and coefficients of variation (CV) of the tracers selected in the final composite signatures for discriminating the sediment sources in the study basin using all three classification schemes.

Sources/Sediment Parameters Al (g kg -1 ) Sr (mg kg -1 ) Ti (mg kg -1 ) Additional data for the mean concentrations and coefficients of variation (CV) of the LDA-selected elements can be found in supplementary Tables 1, 2, 3, 4 and 5.

The regional and soil class sources had the lowest discrimination errors reflecting the influence of pedogenetic processes on tracer signatures. Other studies have reported successful discrimination of soil classes in large catchments in Brazil (Le [START_REF] Gall | Tracing sediment sources in a subtropical agricultural catchment of Southern Brazil cultivated with conventional and conservation farming practices[END_REF][START_REF] Batista | Using pedological knowledge to improve sediment source apportionment in tropical environments[END_REF]. The higher discrimination errors obtained when the sources were classified on the basis of land use reconfirmed the difficulty of geochemically differentiating these sources in large heterogeneous basins [START_REF] Pulley | The uncertainties associated with sediment fingerprinting suspended and recently deposited fluvial sediment in the Nene river basin[END_REF]Pulley et al., 2017). For example, the overall average coefficient of variation (CV) for individual tracers for the sources classified on the basis of land use (42%) was slightly higher than for regional sources (40%) and soil classes (36%). This same trend was also followed by the average CVs of the common tracers selected in the composite signatures for discriminating sources using all three classification schemes (Al, Ti, and Sr): land use (34%) > regional sources (31%) > classes (30%). Caatinga (mean CV = 52%) and channel banks + unpaved roads (mean CV = 39%) contributed strongly to the variations in the case of sources classified by land use, with the highest value (137%) measured for Pb concentrations in Caatinga. The distribution of the channel banks + unpaved roads samples in the basin (Fig. 1c) may explain this scenario, since about 30% and 70% of these samples, respectively, were collected along the upper and central parts of the study basin (i.e., representing very different lithological, pedological and climatic contexts).

Despite this pattern of intra-source variability and the corresponding higher LDA errors, all the RMSE and MAE estimates for the predicted source proportions using the land use classification were acceptably low and, in fact, similar to the corresponding errors calculated when the sources were classified on the basis of regional sources or soil classes, indicating acceptable confidence in all of the predicted source proportions.

Contribution patterns of different source types

Most (SS = 81% and BS = 87%) of the sampled sediments transported in the Ipojuca River at the outlet were predicted to originate from sources located in the downstream region of the study catchment. This distribution pattern reflects several contrasting hydro-climatic and environmental conditions. The downstream areas comprise environments characterized by a humid climate (average annual rainfall of about 2400 mm year -1 ), whereas the upstream portion is typically represented by a semiarid climate (i.e., average rainfall of about 600 mm year -1 ). Accordingly, rainfall erosivity is threefold higher downstream than in the upstream region of the study basin [START_REF] Cantalice | Linhas isoerosivas do estado de Pernambuco-1ª aproximação[END_REF]. The small contribution of the upstream sources (SS and BS < 20%) is likely to have been reduced by the dams built upstream of the main channel.

These structures can significantly decrease longitudinal connectivity, which means longer sediment trapping and residence times in that portion of the study catchment and a lower transfer rate to the downstream portion [START_REF] Kitamura | Predicting sediment and cesium-137 discharge from catchments in eastern Fukushima[END_REF][START_REF] Silva | Trace element fluxes in sediments of an environmentally impacted river from a coastal zone of Brazil[END_REF][START_REF] Batista | Using pedological knowledge to improve sediment source apportionment in tropical environments[END_REF]. Our results indicate, however, that the dams did not result in total sedimentary disconnection during the observation period. As [START_REF] Cucchiaro | Multitemporal analysis of the role of check dams in a debris-flow channel: Linking structural and functional connectivity[END_REF] underlined, dams can be ineffective in the case of large flows, continuing to release trapped sediments.

Soils located in the downstream part of the study catchment (Oxisol, Ultisol + Entisol (Aquent)) were the main sediment sources, contributing an average of 80% and 58% of the target suspended and bed sediment samples, respectively. Entisols and Alfisols in the western (upstream) portion of the study basin were predicted to supply low contributions to the sampled target sediments. Our sediment fingerprinting estimates suggested high contributions (60%) from Oxisols to target suspended sediment samples; three times higher than the corresponding contributions from the other soil classes.

Although they are deep soils with greater infiltration capacity and cover only about 9% of the study basin, the Oxisols are mainly distributed in the downstream region and in areas closer to the main river channels (represented in yellow in Fig. 1b). This distribution enhances hydrological and sedimentological connectivity with the overall outlet sampling site for the target sediment samples. In contrast, the Ultisols comprise 32% of the study catchment but are often found in areas with lower slope-to-channel connectivity than the Oxisols. The low contribution of the Ultisol + Entisol (Aquent) soil combination may be associated with the fine texture in the surface layer of the Ultisols (Supplementary Table 1) and the limited spatial coverage of the study area by the Entisols (Aquents) (<1.5%).

Sediment source apportionment based on the land use classification of potential sources also underlined the importance of the downstream region of the study area.

Sugarcane contributed 62% and 64% of the target suspended and bed sediment sampled.

In contrast, Caatinga contributed only 6% to both types of target sediment.

The sugarcane contribution to sediment samples shows the importance of the surface erosion processes in the study catchment. Its protective vegetation cover during the more advanced development stages buffers the soil against erosion processes [START_REF] Bezerra | Erosão entre sulcos em diferentes condições de cobertura do solo, sob cultivo da cana-de-açúcar[END_REF][START_REF] Amorim | Sediment source apportionment using optical property composite signatures in a rural catchment, Brazil[END_REF], but the post-harvest period and the initial development stages are likely associated with the high soil erosion in this land use. Our results suggested that channel banks plus unpaved roads are the second most important sediment source classified by land use. This reflects the lack of riparian vegetation along the Ipojuca River in the case of the channel banks. High unpaved road contributions have been observed in other watersheds in Brazil, but these are typically relatively low compared to other sources, mainly due to the small surface areas of these sources in the catchments investigated [START_REF] Tiecher | Fingerprinting sediment sources in a large agricultural catchment under no-tillage in Southern Brazil (Conceição River)[END_REF][START_REF] Ramon | Combining spectroscopy and magnetism with geochemical tracers to improve the discrimination of sediment sources in a homogeneous subtropical catchment[END_REF][START_REF] Amorim | Sediment source apportionment using optical property composite signatures in a rural catchment, Brazil[END_REF].

Sediment source apportionment based on soil classes alone led to contrasting results for SS and BS. The Oxisols and Entisols are mainly responsible for this pattern.

These results may reflect a legacy from accumulated bed sediment-associated metal pollution in the river in in the downstream region, since the average metal concentrations were higher than those observed in SS, except for Pr and Sc (Supplementary Figures 2 and3).

Overall, the Oxisols under sugarcane represent the most important sediment source. However, the source apportionment estimates are scale-dependent [START_REF] Batista | Using pedological knowledge to improve sediment source apportionment in tropical environments[END_REF][START_REF] Koiter | Investigating the role of connectivity and scale in assessing the sources of sediment in an agricultural watershed in the Canadian prairies using sediment source fingerprinting[END_REF], meaning that the predicted source contributions would most likely differ for target sediment sampling sites located further upstream. Minimizing errors associated with the classification of sediment sources on the basis of land use is a challenge for future studies in the large river catchments typical of Brazil. Here, there is a need to explore the utility of additional and novel tracers. In this regard, previous work in Brazil has illustrated the utility of optical property composite signatures [START_REF] Amorim | Sediment source apportionment using optical property composite signatures in a rural catchment, Brazil[END_REF] and total P concentrations and their fractionation [START_REF] Tiecher | Potential of phosphorus fractions to trace sediment sources in a rural catchment of Southern Brazil: comparison with the conventional approach based on elemental geochemistry[END_REF]. The latter could be useful for our study basin, since in the downstream part of the catchment, topsoil layers under tillage tend to have higher P contents, compared with channel banks and unpaved roads, due to the use of phosphate fertilizers. This potential to use P-based tracers is enhanced by the ability of clay minerals, typical of the more developed soils in the downstream region of our study basin, to fix P in their structures, generating potentially useful signatures to assist sediment source discrimination. Other possibilities are to test environmental DNA to characterize the dominant plant communities found in soils under Caatinga and sugarcane [START_REF] Evrard | Environmental DNA provides information on sediment sources: a study in catchments affected by Fukushima radioactive fallout[END_REF]Foucher et al., 2020a), the composition of organic matter in soils from different land uses [START_REF] Laceby | Do forests represent a long-term source of contaminated particulate matter in the Fukushima Prefecture?[END_REF]Foucher et al., 2020b) and the compound specific stable isotopes [START_REF] Upadhayay | Methodological perspectives on the application of compoundspecific stable isotope fingerprinting for sediment source apportionment[END_REF].

Conclusions

Application of different classifications of sediment sources has enhanced the level of detail on sediment source patterns in the study basin. The more detailed information generated using more than one source area classification scheme better supports the targeting of sediment management.

Future research could use sedimentary deposits for the temporal reconstruction of sediment source patterns in the semiarid region of the basin. Erosion control in downstream landscapes with Oxisols under sugarcane cultivation should be a priority for reducing the delivery of sediments and associated contaminants towards the estuarine and mangrove environments. Our study herein supports future sediment source fingerprinting studies in areas crossing semiarid and coastal environments under threat from excessive erosion and sediment delivery.

Fig. 1 .

 1 Fig. 1. Map of the Ipojuca River catchment, Northeastern Brazil, showing the distribution of the main geological (a), pedological and sampling locations for target sediment (b) and land use zones and sampling locations for potential sources (c). Dams are shown in (b).

  ) were: Caatinganatural vegetation (n = 17); channel banks (n = 69); unpaved roads (n = 56), and; sugarcane cropland (n = 65). Caatinga samples represent the preserved and semi-preserved conditions found in upper semiarid areas, comprising samples collected under riparian vegetation and in areas connected to the main channel of the Ipojuca River. 2. The classification of sources based on soil classes was executed using the database of the Agroecological Zoning of Pernambuco (ZAPE) and the Brazilian System of Soil Classification (SiBCS) and by overlaying sample locations on these maps. The following soil classes were included in this classification scheme: Entisols (n = 11), Alfisols (n = 37), Ultisols (n = 76), Entisols (Aquents) (n = 57) and Oxisols (n = 26). Entisols occur mainly in the uppermost region of the study basin. The Alfisols in the study basin usually manifest a pattern of acidity from light to neutral, sodium saturation lower than 8% at the Bt horizon, and low content of superficial organic matter. The Ultisols, mainly distributed in the downstream region, are deep, with a low natural fertility, with strong to moderate acidity, and low contents of exchangeable calcium and magnesium. The Ultisols of the upstream region are less deep, although they exhibit small chemical differences compared to the former. The Entisols (Aquents) present varied chemical characteristics as a consequence of their spatial distribution. The Oxisols are deep, with strong acidity, natural cohesion, high content of Al, and are developed mainly in the wetter areas of the basin and the upper sections of plains in the downstream region. 3. In the third classification scheme for potential sources, the same sampling set was divided into two environmentally contrasting regions: upper and lower catchment. Samples from the upstream region (n = 45) represent the combination of the dry climate with a highly spatially-and temporally-variable hydrological regime, shallow soils, intermittent and semi-intermittent fluvial regime, and the sparse occurrence of cropland. The samples (n = 162) from the downstream zone represented wet landscapes, a perennial fluvial regime, deep and well-developed soils, and extensive areas of monoculture cropland.

Fig. 2 .

 2 Fig. 2. Representative landscapes for potential sediment sources and sampling sites in the Ipojuca River catchment: (A) and (B) cross-sections selected downstream in the Ipojuca River for the installation of the sediment samplers during the period of higher water discharge; (C) example of

  HClO4 and placed on a hot plate at 180 °C. The latter step was repeated to ensure the total dissolution of the samples. Each extract was then dissolved in 5 ml HCl and diluted in deionized water to fill a 25 ml volumetric flask. Calibration and recalibration of curves, high purity acids, reagent blanks and standard reference materials (SRM 2709 Montana Soil, National Institute of Standards and Technology,NIST, 2002) were used to ensure quality control of the analyses. The concentrations of metals in the final extracts were determined by means of inductively coupled plasma optical emission spectrometry (ICP-OES/Optima DV7000, Perkin Elmer) with a coupled cyclonic chamber system to enhance precision of the measurements. Only the total concentrations of Zr were determined by means of X-Ray fluorescence spectrometry (S8 TIGER ECO -WDXRF-1KW model).

  (1) assessment of conservative behaviour (range test); (2) comparison of the individual sources (Kruskal-Wallis H-test), and; (3) linear discriminant analysis (forward stepwise tracer selection). Tracer conservation was assessed using a conventional range test based on the individual comparison of the minimum and maximum elemental concentrations in source material and target sediment samples. The Kruskal-Wallis H-test was used to explore and confirm individual tracer capacity for distinguishing the sediment sources (p < 0.1). Linear discriminant analysis (p < 0.1) used the minimization of Wilk's lambda to select three final composite signatures for discriminating the sediment sources on the basis of the three classification schemes. To minimize problems associated with characterization of sources and effects of point or non-point pollution, we removed any extreme or outlier values from this fingerprinting modelling. MixSIAR (Stock and Semmens, 2016) was applied to estimate the average relative contribution of the individual sediment sources, using Markov Chain Monte Carlo (MCMC) maximum parameters: number of chains =

Fig. 3 .

 3 Fig. 3. Two-dimensional plots of the LDA (p < 0.1) of the initial (a and c) and final (b and d) source groups after reclassification of the Ipojuca River catchment sources for the samples defined as soil classes (a and b) and land use (c and d), considering all geochemical tracers. UL = Ultisol,

Fig. 4 Fig. 4 .

 44 Fig. 4 presents source type contributions to SS and BS samples estimated by

  UP = Upstream and DO = Downstream; AL = Alfisol, EN = Entisol, OX = Oxisol and UE = Ultisol + Entisol (Aquent); CA = Caatinga, CR = channel bank + unpaved road, SC = sugarcane; standard deviation (SD); root mean squared error (RMSE) and mean absolute error (MAE).

Table 1 .

 1 Sampling periods for target suspended sediment and the corresponding rainfall totals in the upstream and downstream portions of the Ipojuca River catchment.

	Observation period*	Sample dry mass (g)	Amount of rainfall (mm) (upstream -downstream) **
	04/15/19 -05/22/19	20.4	165 -249
	05/22/19 -06/10/19	38.9	121 -473
	09/27/19 -12/03/19	90.0	32 -147
	12/11/19 -02/10/20	17.2	28 -105

* channel cross-section location: 8°24'20.90"S -35°03'37.22"W; ** (IPA, 2019).

Table 2 .

 2 Minimum and maximum values of geochemical tracer concentrations in source material and target sediment samples (suspended and bed sediments) used to assess tracer conservation.

			Sources		Suspended sediments	Bed sediments
	Tracers (mg kg -1 )		(n = 207)			(n = 4)			(n = 6)
		Min. Max.	Mean	Min.	Max.	Mean	Min.	Max.	Mean
	Al	9240 162850 115011 103650 131200 116250 106150 137850 127200
	Ba	4.1 1713.0 595.7	294.5	570.5	422.6	445.8 1040.5 773.3
	Ce	44.1 329.9	137.8	129.7	138.9	134.0	88.8	138.9	121.6
	Cr	5.9	123.8	38.6	7.2	48.8	37.5	35.8	55.3	41.2
	Fe	4556 86600 38228 13510 45070 31932 35845 64200 46318
	La	18.4 140.2	59.6	57.5	60.6	58.6	34.4	65.2	53.4
	Nd	8.9	85.0	33.0	31.6	33.4	32.5	18.9	34.8	28.5
	Ni	2.9	52.6	16.6	4.8	17.4	13.2	12.8	15.9	14.5
	Pb	4.8	301.1	25.8	7.1	24.8	18.2	14.8	34.3	26.0
	Pr	3.0	43.1	14.4	14.3	16.0	14.8	4.4	14.8	10.4
	Sc	2.1	15.6	8.1	8.5	9.8	9.3	6.2	9.0	7.4
	Sm	2.4	21.7	8.7	8.4	9.3	8.8	4.8	8.6	7.3
	Sn	1.6	22.4	6.4	2.2	7.2	5.4	6.4	8.2	7.0
	Sr	1.1	714.0	127.4	31.7	127.9	90.8	91.7	187.1	146.7
	Th	11.9 111.6	34.2	26.6	29.9	28.3	20.8	35.8	29.5
	Ti	266 12725	5912	1627	6225	4606	5655	7245	6657
	V	6.0	181.1	73.6	18.9	87.4	63.0	59.1	80.7	71.9
	Y	3.4	55.9	12.3	12.7	13.9	13.4	7.5	14.7	11.7
	Zn	0.0	158.6	49.0	11.2	72.0	53.9	53.9	76.4	64.4
	Zr	112.0 3441.0 643.6	135.0	376.5	215.0	270.0	961.0	536.3

Table 3 .

 3 Kruskal-Wallis H-test results for the sources classified using the geochemical tracers that passed the conservation test.

	Tracers	Regions H-value p-value H-value p-value H-value p-value Soil classes Land use
	Al	97.81	<0.01	88.29	<0.01	35.52	<0.01
	Ba	52.43	<0.01	76.15	<0.01	23.38	<0.01
	Ce	1.70	0.19	4.26	0.24	16.14	<0.01
	Cr	11.31	<0.01	18.88	<0.01	6.41	0.04
	Fe	74.69	<0.01	67.92	<0.01	34.11	<0.01
	La	7.08	0.01	11.78	0.01	17.42	<0.01
	Nd	0.10	0.75	0.40	0.94	12.63	<0.01
	Ni	4.32	0.04	13.80	<0.01	6.31	0.04
	Pb	26.82	<0.01	35.71	<0.01	14.93	<0.01
	Pr	0.61	0.44	7.62	0.05	10.39	0.01
	Sc	67.38	<0.01	63.89	<0.01	26.49	<0.01
	Sm	1.38	0.24	1.66	0.65	12.05	<0.01
	Sn	12.85	<0.01	8.43	0.04	7.67	0.02
	Sr	60.94	<0.01	78.40	<0.01	24.74	<0.01
	Th	0.41	0.52	18.23	<0.01	6.93	0.03
	Ti	84.62	<0.01	82.95	<0.01	27.36	<0.01
	V	74.54	<0.01	76.95	<0.01	23.78	<0.01
	Y	67.38	<0.01	70.44	<0.01	38.06	<0.01
	Zn	0.54	0.46	2.69	0.44	4.69	0.10
	Zr	25.98	<0.01	36.09	<0.01	15.39	<0.01

Table 4 .

 4 Final composite signatures selected by linear discriminant analysis and corresponding parameters for the three classifications of sediment sources in the Ipojuca River catchment.

	Tracers	Wilks' lambda	F-value	p-value	IER (%) CER (%)
			Regions
	Al	0.40	311.82	<0.01	4.3	4.3
	Sr	0.33	208.01	<0.01	8.1	3.3
	Y	0.31	153.50	<0.01	11.8	3.3
	Ti	0.29	125.92	<0.01	15.3	1.9
	Pb	0.28	105.49	0.01	16.3	1.9
	La	0.27	91.29	0.01	16.9	1.9
	Fe	0.26	80.23	0.04	15.9	1.9
	Zr	0.25	72.68	0.02	17.2	2.9
			Soil classes
	Al	0.46	80.64	<0.01	20.7	20.7
	Ba	0.35	46.97	<0.01	33.3	21.7
	Ni	0.30	34.29	<0.01	34.2	20.7
	Ti	0.28	27.00	<0.01	22.7	18.8
	La	0.25	23.83	<0.01	35.7	19.3
	Pr	0.22	21.87	<0.01	35.7	15.4
	Sr	0.21	19.44	0.01	35.7	15.4
	Zr	0.20	17.59	0.02	34.3	16.4
	Th	0.19	15.99	0.06	35.7	15.4
	Sc	0.19	14.68	0.09	37.1	13.5
			Land uses
	Al	0.78	28.31	<0.01	42.0	42.0
	Ce	0.72	18.40	<0.01	37.1	40.1
	Ti	0.66	15.43	<0.01	40.0	35.7
	V	0.61	14.34	<0.01	39.6	31.8
	Pb	0.57	12.84	<0.01	39.6	31.4
	Sr	0.55	11.67	0.01	40.0	28.0

IER = individual error rate; CER = cumulative error rate.

Table 5 .

 5 Comparison of the mean known (MC) and predicted (MP) contributions of the different sources of target suspended sediment (SS) and bed sediment (BS) samples using virtual mixtures and statistical tests for the accuracy of MixSIAR outputs.

		Sources	Target sediment	MC (%)	SD (%)	MP (%)	RMSE MAE
		Upstream		19.5	10.2	24.4
			SS				4.9	4.9
		Downstream		80.5	10.2	75.6
	Regions
		Upstream		13.3	6.3	13.4
			BS				0.1	0.1
		Downstream		86.7	6.3	86.6
		Alfisol		6.0	5.7	11.0
		Entisol		13.5	7.0	11.4
			SS				5.3	4.9
		Oxisol		65.0	9.3	57.2
	Soil	Ultisol-Entisol (Aquent)		15.5	8.6	20.4
	classes	Alfisol		7.8	6.0	16.3
		Entisol		33.7	9.3	25.3
			BS				6.0	4.7
		Oxisol		30.8	9.1	31.8
		Ultisol-Entisol (Aquent)		27.7	11.3	26.6
		Caatinga		6.2	6.0	7.5
		Channel bank + unpaved roads	SS	31.7	17.5	34.0	2.5	2.3
		Sugarcane		62.0	17.7	58.5
	Land use
		Caatinga		6.1	4.3	6.8
		Channel bank + unpaved roads	BS	30.3	13.6	33.4	2.8	2.5
		Sugarcane		63.6	13.4	59.8

Acknowledgements Y.J.A.B. Silva is grateful to the National Council for Scientific and Technological Development -CNPq for research productivity scholarships (Process Number: 303221/2019-4). The contribution to this manuscript by ALC was funded by UKRI-BBSRC (UK Research and Innovation-Biotechnology and Biological Sciences Research Council) grant award BBS/E/C/000I0330. This study was financed in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior -Brasil (CAPES) -Finance Code 001.


Ayyoob Imani

Silvia Severini

Masoud Jalili Sabet
email: masoud@cis.lmu.de

François Yvon
email: francois.yvon@limsi.fr

Hinrich Schütze

Graph-Based Multilingual Label Propagation for Low-Resource Part-of-Speech Tagging

Part-of-Speech (POS) tagging is an important component of the NLP pipeline, but many lowresource languages lack labeled data for training. An established method for training a POS tagger in such a scenario is to create a labeled training set by transferring from high-resource languages. In this paper, we propose a novel method for transferring labels from multiple high-resource source to low-resource target languages. We formalize POS tag projection as graph-based label propagation. Given translations of a sentence in multiple languages, we create a graph with words as nodes and alignment links as edges by aligning words for all language pairs. We then propagate node labels from source to target using a Graph Neural Network augmented with transformer layers. We show that our propagation creates training sets that allow us to train POS taggers for a diverse set of languages. When combined with enhanced contextualized embeddings, our method achieves a new state-ofthe-art for unsupervised POS tagging of low resource languages.

Introduction

In many applications, Part-of-Speech (POS) tagging is an important part of the NLP pipeline. In recent years, high-accuracy POS taggers have been developed owing to advances in machine learning methods that combine pretraining on large unlabeled corpora and supervised fine-tuning on wellcurated annotated datasets. This methodology only applies to a handful of high-resource (HR) languages for which the necessary training data exists, leaving behind the majority of low-resource (LR) languages. When training resources are scarce, an established method for training POS taggers is to automatically generate the training data via crosslingual transfer [START_REF] Yarowsky | Inducing multilingual POS taggers and NP bracketers via robust projection across aligned corpora[END_REF][START_REF] Fossum | Automatically inducing a part-of-speech tagger by projecting from multiple source languages across aligned corpora[END_REF][START_REF] Agić | Multilingual projection for parsing truly low-resource languages[END_REF][START_REF] Eskander | Unsupervised cross-lingual part-ofspeech tagging for truly low-resource scenarios[END_REF] Figure 1: The sentence "Actions speak louder than words" in English and its translations in Persian, German, and Turkish, aligned at the word level. The POS tags for high-resource English and German are known. We use a GNN to exploit this graph structure and compute POS tags for low-resource Persian and Turkish. 2020). Typically, POS annotations are projected through alignment links from the HR source to the LR target of a word aligned parallel corpus.

In this paper, we propose GLP (Graph Label Propagation), a novel method for transferring labels simultaneously from multiple high-resource source languages to multiple low-resource target languages. We formalize POS tag projection as graph-based label propagation. Given translations of a sentence in multiple languages, we create a graph with words as nodes and alignment links as edges by aligning words for all language pairs. We then propagate POS labels associated with source language nodes to target language nodes, using a label propagation model that is formalized as a Graph Neural Network (GNN) [START_REF] Scarselli | The graph neural network model[END_REF]. Nodes are represented by a diverse set of features that describe both linguistic properties and graph structural information. In a second step, we additionally employ self-learning to obtain reliable training instances in the target languages.

Our approach is based on multiparallel corpora, meaning that the translation of each sentence is available in more than two languages. We exploit the Parallel Bible Corpus (PBC) of [START_REF] Mayer | Creating a massively parallel Bible corpus[END_REF],1 a multiparallel corpus that covers more than 1000 languages, many of which are extremely low-resource, by which we mean that only a tiny amount of unlabeled data is available or that no language technologies exist for them at all [START_REF] Joshi | The state and fate of linguistic diversity and inclusion in the NLP world[END_REF].

We evaluate our method on a diverse set of lowresource languages from multiple language families, including four languages not covered by pretrained language models (PLMs). We train POS tagging models for these languages and evaluate them against references from the Universal Dependencies corpus [START_REF] Daniel Zeman | Universal dependencies 2[END_REF]. We compare the results of our method against multiple stateof-the-art (SOTA) cross-lingual unsupervised and semisupervised POS taggers employing different approaches like annotation projection and zero-shot transfer. Our experiments highlight the benefits of our new transfer and self-learning methods; crucially, they show that reasonably accurate POS taggers can be bootstrapped without any annotated data for a diverse set of low-resource languages, establishing a new SOTA for high-resource-to-lowresource cross-lingual POS transfer. We also assess the quality of the projected annotations with respect to "silver" references and perform an ablation study. To summarize, our contributions are:2

• We formalize annotation projection as graphbased label propagation and introduce two new POS annotation projection models, GLP-B (GLP-Base) and GLP-SL (GLP-SelfLearning).

• We evaluate GLP-B and GLP-SL on 17 lowresource languages, including 4 languages not covered by large PLMs.

• By comparing our method with various supervised, semisupervised, and PLM-based approaches for POS tagging of low-resource languages, we establish a new SOTA for unsupervised POS tagging.

Related work

POS tagging Part of Speech tagging aims to assign each word the proper syntactic tag in its context [START_REF] Manning | Foundations of Statistical Natural Language Processing[END_REF]. For highresource languages, for which large labeled training sets are available, high-accuracy POS tagging is achieved through supervised learning [START_REF] Kondratyuk | [END_REF][START_REF] Tsai | Small and practical bert models for sequence labeling[END_REF].

Zero-shot transfer In low-resource settings, one approach is to use cross-lingual transfer thanks to pretrained multilingual representations, thereby enabling zero-shot POS tagging. [START_REF] Kondratyuk | [END_REF] analyze the few-shot and zero-shot performance of mBERT [START_REF] Devlin | BERT: Pre-training of deep bidirectional transformers for language understanding[END_REF] finetuning on POS tagging. We include this approach in our set of baselines below. [START_REF] Ebrahimi | How to adapt your pretrained multilingual model to 1600 languages[END_REF] and Wang et al. (2022a) analyze zeroshot POS tagging performance of XLM-RoBERTa [START_REF] Conneau | Unsupervised cross-lingual representation learning at scale[END_REF] and propose complementary methods such as continued pretraining, vocabulary expansion and adapter modules for better performance. We show that combining GLP with Wang et al. (2022a)'s embeddings further improves our base performance.

Annotation projection is another approach for annotation of low-resource languages. [START_REF] Yarowsky | Inducing multilingual POS taggers and NP bracketers via robust projection across aligned corpora[END_REF] first proposed the idea of projecting annotation labels across languages exploiting parallel corpora and word alignment. To reduce systematic transfer errors, [START_REF] Fossum | Automatically inducing a part-of-speech tagger by projecting from multiple source languages across aligned corpora[END_REF] extended this by projecting from multiple source languages. Agić et al. (2015a) and [START_REF] Agić | Multilingual projection for parsing truly low-resource languages[END_REF] exploit multilingual transfer setups to bootstrap POS taggers for low-resource languages starting from a parallel corpus and taggers and parsers for high-resource languages. Other works project labels by leveraging token and typelevel constraints [START_REF] Täckström | Token and type constraints for cross-lingual part-of-speech tagging[END_REF]Buys and Botha, 2016a;[START_REF] Eskander | Unsupervised cross-lingual part-ofspeech tagging for truly low-resource scenarios[END_REF]. The latter study notably proposes an unsupervised method for selecting training instances via cross-lingual projection and trains POS taggers exploiting contextualized word embeddings, affix embeddings and hierarchical Brown clusters [START_REF] Peter | Class-based n-gram models of natural language[END_REF]. This approach is also used as a baseline below.

Semi-supervised approaches have been proposed to mitigate the noise of projecting between languages. This can be achieved with auxiliary lexical resources [START_REF] Täckström | Token and type constraints for cross-lingual part-of-speech tagging[END_REF][START_REF] Ganchev | Crosslingual discriminative learning of sequence models with posterior regularization[END_REF][START_REF] Wisniewski | Crosslingual part-of-speech tagging through ambiguous learning[END_REF][START_REF] Li | Wiki-ly supervised part-of-speech tagging[END_REF] that guide unsupervised learning or act as an additional training signal [START_REF] Plank | Distant supervision from disparate sources for low-resource partof-speech tagging[END_REF]. Other works combine manual and projected annotations [START_REF] Garrette | Learning a part-of-speech tagger from two hours of annotation[END_REF][START_REF] Fang | Learning when to trust distant supervision: An application to lowresource POS tagging using cross-lingual projection[END_REF]. We outperform prior works without the use of additional resources (such as dictionaries and annotations).

Graph Neural Networks Many natural and reallife structures like physical systems, social networks & interactions, and molecular fingerprints have a graphical structure [START_REF] Liu | Introduction to graph neural networks[END_REF]. Graph neural networks have been successfully used to model them. Applications include social spammer detection [START_REF] Wu | Graph convolutional networks with markov random field reasoning for social spammer detection[END_REF], learning molecular fingerprints [START_REF] David K Duvenaud | Convolutional networks on graphs for learning molecular fingerprints[END_REF] and human motion prediction [START_REF] Li | Dynamic multiscale graph neural networks for 3d skeleton based human motion prediction[END_REF]. Recently, GNNs have been adopted for NLP tasks such as text classification [START_REF] Peng | Large-scale hierarchical text classification with recursively regularized deep graph-cnn[END_REF], sequence labeling [START_REF] Zhang | Sentencestate LSTM for text representation[END_REF][START_REF] Marcheggiani | Encoding sentences with graph convolutional networks for semantic role labeling[END_REF], neural machine translation [START_REF] Bastings | Graph convolutional encoders for syntax-aware neural machine translation[END_REF][START_REF] Beck | Graph-to-sequence learning using gated graph neural networks[END_REF], and alignment link prediction [START_REF] Imani | Graph neural networks for multiparallel word alignment[END_REF]. As far as we know, our work is the first to form the annotation projection problem as graph-based label propagation.

Multiparallel corpora A multiparallel corpus provides the translations of a source text in more than two languages. A few such corpora [START_REF] Agić | JW300: A widecoverage parallel corpus for low-resource languages[END_REF][START_REF] Mayer | Creating a massively parallel Bible corpus[END_REF][START_REF] Tiedemann | Parallel data, tools and interfaces in opus[END_REF] provide sentence-level aligned text for hundreds or thousands of languages; for many of these languages only a tiny amount of digitized content is available [START_REF] Joshi | The state and fate of linguistic diversity and inclusion in the NLP world[END_REF]. Although the amount of text found in existing multiparallel corpora is far less than in monolingual corpora, we believe that they can serve as cross-lingual bridges, with which effective representation for low-resource languages can be derived. Highly multiparallel corpora have been used for expanding pretrained models to more languages [START_REF] Ebrahimi | How to adapt your pretrained multilingual model to 1600 languages[END_REF]Wang et al., 2022b), word alignment improvement and visualization [START_REF] Imanigooghari | ParCourE: A parallel corpus explorer for a massively multilingual corpus[END_REF][START_REF] Imani | Graph neural networks for multiparallel word alignment[END_REF], embedding learning [START_REF] Dufter | Embedding learning through multilingual concept induction[END_REF], and annotation projection (Agić et al., 2015b;[START_REF] Severini | Towards a broad coverage named entity resource: A data-efficient approach for many diverse languages[END_REF].

Method

We now introduce our Graph Label Propagation (GLP) method, which formalizes the problem of annotation projection as graph-based label propagation. We first describe the graph structure, then the features associated with each node, and finally the architecture of our model.

Problem formalization

The multilingual alignment graph (MAG) of a sentence is formalized as follows. Each sentence σ in our multiparallel corpus exists in a set L of languages.3 L contains both high-resource source languages (in L s ) and low-resource target languages (in L t ) with L s ∪ L t = L. Each word in these |L| versions of σ will constitute a node in our graph.

We first automatically annotate the text in all the source languages using pre-existing taggers: these POS tags are node labels; they are only known for languages in L s , unknown otherwise. We then use Eflomal [START_REF] Östling | Efficient word alignment with Markov Chain Monte Carlo[END_REF], an unsupervised word alignment tool to compute alignment links for all |L| * (|L|-1) 2 language pairs: these links define the edges of our MAG. Figure 1 displays an example MAG for four languages, with English and German as sources and Turkish and Persian as targets. Note that both the word alignments and the node labels are noisy since we do not use gold data but statistical methods to generate them.

Features

To train graph neural networks, we represent each node using a set of features [START_REF] Duong | On node features for graph neural networks[END_REF]. In Figure 2 you see a simple illustration of how nodes are represented using a feature vector. The graph in this figure is part of the original graph in Figure 1. Two types of features are considered: features that represent the inherent meaning of a node/word (word representation features) and features that describe the position of a node within the graph (graph structural features). Word representation features consist of: XLM-R [START_REF] Conneau | Unsupervised cross-lingual representation learning at scale[END_REF] embeddings, the node's language and its position within the sentence. Since XLM-R embeddings are not available for all languages, we alternatively experiment with static word embeddings created using [START_REF] Levy | A strong baseline for learning cross-lingual word embeddings from sentence alignments[END_REF]'s sentence-ID method, which we train on PBC. Our graph structural features are similar to [START_REF] Imani | Graph neural networks for multiparallel word alignment[END_REF]'s work on link prediction. They include five centrality features: degree, closeness [START_REF] Linton C Freeman | Centrality in social networks conceptual clarification[END_REF], betweenness [START_REF] Brandes | A faster algorithm for betweenness centrality[END_REF], load (Newman, 2001), and harmonic centrality [START_REF] Boldi | Axioms for centrality[END_REF]. Each of these features describes the node's position within the graph from a different perspective. For example, degree is the number of neighbors of the node and harmonic centrality measures how important/influential a node is. They also include two community features corresponding to the ID of the node's communities computed respectively with the greedy modularity community detection method of [START_REF] Clauset | Finding community structure in very large networks[END_REF] and the label propagation algorithm of [START_REF] Cordasco | Community detection via semi-synchronous label propagation algorithms[END_REF]. These two methods detect communities of nodes such that there are many links within the communities and only a few between them. The encoder has two GATConv layers [START_REF] Veličković | Graph attention networks[END_REF]: given a graph with M nodes represented as x 1 , x 2 , ..., x M , with respective neighborhoods N (1), N (2), ..., N (M ), a GATConv layer computes a new representation x i for each node as:

GLP architecture

x i = j∈N (i)∪{i} α i,j Wx j , (1)
with W a learnable weight matrix. α i,j measures how much node i "attends" to node j as follows:

α i,j = exp g a [Wx i Wx j ] k∈N (i)∪{i} exp (g (a [Wx i Wx k ]))
where stands for concatenation, g is the LeakyReLU (Maas et al.) and a is a weight vector. As neighborhoods only use alignment links, the representation of a node is only influenced by nodes in other languages. Also note that both source and target nodes are fed to the encoder.

We train two GLP models: GLP-Base (GLP-B) and GLP-SelfLearning (GLP-SL). The first one is the basic GNN architecture. It tags a token based on the other languages only, i.e., it makes no use of the sequence information of the token in its own language. The second additionally employs selflearning and is given access to the local context of each token in its own language.

GLP-B uses a multi-layer perceptron as classifier. We feed the node representations to the classifier and train the model end-to-end. We can only do this for source nodes since we have no training data for the target languages.

GLP-SL additionally employs self-learning and a better classifier. Self-learning takes advantage of node labels predicted by GLP-B in the first step: when the prediction confidence exceeds a threshold γ, these labels are deemed correct and the corresponding nodes are considered when training the classifier. GLP-SL uses a Transformer architecture to predict POS tags. The Transformer input consists of all translations of a sentence, where words are represented as GNN node embeddings. Each embedding is the concatenation of input (x i ) and output representations (x i ) of the corresponding node in the GNN. In addition to the information available from neighbor nodes in other languages, the Transformer can attend to other words of the sentence in the same language, some of which may already be (automatically) labeled. This is very different from the training of GLP-B, where the POS of words of the same language were either all known (for source languages) or all unknown (for target languages), and explains why we resorted to a simpler classifier in the first stage.

Similar to [START_REF] Eskander | Unsupervised cross-lingual part-ofspeech tagging for truly low-resource scenarios[END_REF][START_REF] Agić | Multilingual projection for parsing truly low-resource languages[END_REF], GLP-SL uses type-level information: for each word type, we create a tag distribution by accumulating counts of the number of times each tag was assigned. For source words, we use the training data to estimate the distribution. For target words, we use the predictions of GLP-B on PBC.

Neural POS tagger

We use the noisy labeled data, generated by GLP-B or GLP-SL, to train monolingual neural POS taggers. Each model is a Bi-LSTM (Bidirectional Long Short-Term Memory, (Hochreiter and Schmidhuber, 1997)) with XLM-RoBERTa embeddings [START_REF] Conneau | Unsupervised cross-lingual representation learning at scale[END_REF]. The input is a sentence labeled by GLP-B or GLP-SL. A token is assigned the NULL tag in case of missing labels. It is then ignored (i.e., masked) when computing the cross-entropy loss. To avoid predicting NULL, we set the corresponding output cell in the softmax to -∞, similar to [START_REF] Eskander | Unsupervised cross-lingual part-ofspeech tagging for truly low-resource scenarios[END_REF]. Prior work has used older versions of UD . We now compare against each baseline, evaluating on the relevant version of UD in each case.

Experimental setup

Annotation projection-based baselines

In this section, we compare with the unsupervised SOTA in cross-lingual POS tagging via annotation projection: ESKANDER [START_REF] Eskander | Unsupervised cross-lingual part-ofspeech tagging for truly low-resource scenarios[END_REF], AGIC [START_REF] Agić | Multilingual projection for parsing truly low-resource languages[END_REF] and BUYS (Buys and Botha, 2016b) as well as EFLOMAL. We also compare with a semi-supervised SOTA method that uses rapid annotation in addition to cross-lingual projection: CTRL [START_REF] Cotterell | Crosslingual character-level neural morphological tagging[END_REF].

Fully unsupervised baselines

EFLOMAL is a simple projection method using alignment links followed by majority voting, similar to early annotation projection methods (Agić et al., 2015b;[START_REF] Fossum | Automatically inducing a part-of-speech tagger by projecting from multiple source languages across aligned corpora[END_REF]. We first align all target sentences with the corresponding sentences in all training languages with Eflomal [START_REF] Östling | Efficient word alignment with Markov Chain Monte Carlo[END_REF]. Each target word is then tagged with the most common tag in the aligned source words. The annotation projection method ESKANDER [START_REF] Eskander | Unsupervised cross-lingual part-ofspeech tagging for truly low-resource scenarios[END_REF] uses alignment links and token and type constraints to project tags from source to target. The neural POS tagger features include XLM-R embeddings, affix embeddings, and word clusters created on PBC and Wikipedia of the target languages. [START_REF] Agić | Multilingual projection for parsing truly low-resource languages[END_REF] is a crosslingual POS tagger for low-resource languages based on PBC excerpts and translations of the Watchtower.8 BUYS (Buys and Botha, 2016b) extends previous approaches for projecting POS tags using bitexts to infer constraints on the possible tags for a given word type or token.

Table 4 shows that GLP outperforms AGIC and BUYS, except for Portuguese (BUYS), where our results are slightly below. BUYS projects from Spanish, which is closely related to Portuguese. [START_REF] Eskander | Unsupervised cross-lingual part-ofspeech tagging for truly low-resource scenarios[END_REF] showed that it can be advantageous to transfer only from one closely related language as opposed to a mix of close and distant languages. Note that BUYS performance for Portuguese drops down to 84.3 when transferring from English. BUYS also uses Europarl9 with up to 2M tokens which is closer in domain to UD than PBC. Thus, compared to BUYS, the parallel data we use are smaller, and from a more distant domain.

Semisupervised baseline

CTRL [START_REF] Cotterell | Crosslingual character-level neural morphological tagging[END_REF] is a characterlevel recurrent neural network for multi-task crosslingual transfer of morphological taggers. Their experiments include small sets of 100 and 1000 annotated target tokens. The bottom part of Table 4 shows that GLP-SL outperforms CTRL despite being fully unsupervised. [START_REF] Agić | Multilingual projection for parsing truly low-resource languages[END_REF], BUYS (Buys and Botha, 2016b), CTRL (Cotterell and Heigold, 2017) and GLP-SL. We also report the source language or "mul" for multilingual, and for CTRL, the number of the supervision tokens.

Zero-shot baselines

Cross-lingual projection is also possible thanks to multilingual pretrained language models (PLMs).

A PLM is first fine-tuned to POS tagging on source languages and then used to infer tags for target languages. While this approach performs well for some languages without requiring any parallel data, its performance tends to be poor for low-resource languages [START_REF] Hu | Explicit alignment objectives for multilingual bidirectional encoders[END_REF]. [START_REF] Joshi | The state and fate of linguistic diversity and inclusion in the NLP world[END_REF] clusters languages into six groups based on the amount of available unlabeled and labeled data that exists for them. Groups 1 and 2 consist of languages such as Manx and Yoruba with the least amount of available data, while group 5 contains languages like English and Spanish with the largest amount of available monolingual and labeled data. We compare our approach with three baselines using test languages from groups 1 and 2.

mBERT based baselines: [START_REF] Kondratyuk | [END_REF] use the zero-shot approach with multilingual BERT [START_REF] Devlin | BERT: Pre-training of deep bidirectional transformers for language understanding[END_REF] as PLM. We train our POS taggers using mBERT (instead of XLM-R) embeddings for a fair comparison. Table 5 displays the results for the low-resource languages in group 1 and 2 also available in the compared work. GLP-SL outperforms zero-shot in all cases by at least 12 percentage points. This result suggests that annotation projection using GLP is more effective than using multilingual representations for truly low-resource languages (i.e., languages from the first two groups of [START_REF] Joshi | The state and fate of linguistic diversity and inclusion in the NLP world[END_REF]). To create proper representations for a language, PLMs require a huge amount of monolingual data that is not available for many languages. As Table 5 suggests, due to poor representations, zero-shot transfer to these languages is also poor. However, we could successfully exploit the Bible's parallel data in GLP for the benefit of these languages.

XLM-R based baselines: 6 Analysis

Ablation study

We conduct an ablation study to better understand what benefits our model.

"Eng" vs "All" Previous works highlighted the importance of a diverse set of source languages for cross-lingual transfer [START_REF] Lin | Choosing transfer languages for cross-lingual learning[END_REF][START_REF] Turc | Revisiting the primacy of english in zero-shot cross-lingual transfer[END_REF]. The last four lines of Table 3 report GLP-B and GLP-SL results when transferring from English (i.e., using English as the only source), and when transferring from the full set of source languages (see Table 1). The transfer from English has lower performance than from all languages (except for a decrease from 67.5 to 67.4 for Basque/GLP-B). This means that our projection method does benefit from more data and from the rich information present in the diversity of source languages. 2). In order to understand their usefulness in the transfer process, we compare with the performance obtained when static embeddings are used by GLP-SL. Results reported in Appendix B show an average improvement of 3 percentage points when XLM-R embeddings are used. The largest differences (> 5%) are observed for Hindi, Persian, and Marathi. However, for the four languages not supported by XLM-R, the POS tagging accuracy is substantially lower when using contextualized embeddings compared to static embeddings (16.6 points drop on average).

D J A D P A D V A U X C C O N J D E T IN T J N O U N N U M P A R T P R O N P R O P N P U N C T S C O N J S Y M V E R B

GLP-B vs GLP-SL

Quality of artificial training sets

In order to evaluate the quality of the training sets generated by GLP-SL ("GLP sets"), we create a "silver" reference and compute the accuracy of GLP sets with respect to it. To build the silver reference, we annotate the training sets with the Stanza POS tagger for the languages for which it is available (12 out of 17). We obtain an average accuracy of 78.7, with Belarusian being the best and Basque the worst. The best predicted tokens are punctuation marks, coordinating conjunctions, and verbs, while the worst ones are symbols, interjections, and particles (see Figure 4). The high accuracy of 78.7 illustrates the ability of GLP-SL to successfully project annotations from high to low-resource languages.

Conclusion and future work

We presented GLP, a novel method for transferring labels from high-resource source to low-resource target languages, based on a formalization of annotation projection as graph-based label propagation. We exploited the Parallel Bible Corpus and showed that reasonably accurate POS taggers can be bootstrapped from projected labels. Since we do not use PBC-specific or language-specific features, GLP is in principle applicable to the more than 1000 languages of PBC and to any other multiparallel corpus.

One direction for the future is to employ a similar model to transfer higher-level structures such as dependency trees. Since our method works with graphical structures, one might be able to project dependency trees effectively. We could also extend our projection method to other tagging tasks like named entity recognition -although this requires using other parallel corpora to mitigate the domain shift problem of such a task. Another line for future work is to study the best combinations of source languages to transfer to any target language.

Limitations

Our method is evaluated on 17 languages carefully chosen to be from different families and scripts. However, we don't consider the other languages (more than 1000) in PBC due to computational constraints and lack of test sets.

A limitation of the GLP is that training over a MAG (multilingual alignment graph) created for all PBC languages requires a prohibitively large amount of resources, and based on our experiments, if we use a larger number of target languages at the same time, the performance will likely drop. Therefore one has to process languages in smaller batches (in our case, 36 languages). Accordingly, to cover all PBC subcorpora, 1341/36 = 38 GLP models should in principle be trained.

Ethic statement

Our work is based on the Parallel Bible Corpus of [START_REF] Mayer | Creating a massively parallel Bible corpus[END_REF] that consists of Bible verses and is tested on the Universal Dependency treebanks [START_REF] Daniel Zeman | Universal dependencies 2[END_REF], an ensemble of different data sources. We would like to clarify that we treat the data simply as a multiparallel corpus, and the content does not necessarily reflect the opinions of the authors nor of the institutions funding the authors.

Figure 2 :

 2 Figure 2: An example of how we represent nodes of an alignment graph using features for a part of the graph in Figure 1.

Figure 3

 3 Figure 3 displays the architecture of our GLP model; white nodes are for the source (= training) languages and green nodes for the target languages. The model has two parts: the GNN-based encoder turns the alignment graph into node representations and the classifier learns to label nodes based on these representations. The network is trained to reproduce POS tags for each source node; it is then used to predict the unknown tags for target nodes.The encoder has two GATConv layers[START_REF] Veličković | Graph attention networks[END_REF]: given a graph with M nodes represented as x 1 , x 2 , ..., x M , with respective neighborhoods N (1), N (2), ..., N (M ), a GATConv layer computes a new representation x i for each node as:

Figure 3 :

 3 Figure 3: The architecture of GLP (Graph Label Projector). Source nodes are in white, target nodes in green. For training, we first feed the alignment graph of a sentence to the encoder to compute a representation for each node. Next we feed the representations of the source nodes to the classifier. The training objective is cross entropy on prediction of POS tags. Note that we know the POS tags of the source nodes. After training, the model can generalize the POS tag prediction to target nodes.

Figure 4 :

 4 Figure 4: Average per tag accuracy of our GLP sets with respect to the "silver" reference.

Table 1 :

 1 Language family and number of verses in PBC for training, dev, and test languages in our experiments.

				Lang	ISO	Family	# verses
				Arabic	arb	Afro-Asiatic, Semitic
				Chinese	zho	Sino-Tibetan, Sinitic
				Danish	dan	Indo-European, Germanic
				English	eng	Indo-European
				Finnish	fin	Uralic, Finnic
				French	fra	Indo-European, Romance
		Training	languages	German Irish Italian	deu gle ita	Indo-European, Germanic Indo-European, Celtic Indo-European, Romance
				Polish	Pol	Indo-European, Slavic
				Russian	rus	Indo-European, Slavic
				Spanish	spa	Indo-European, Romance
				Swedish	swe	Indo-European, Germanic
				Tamil	tam	Dravidian, Southern Dravidian	7942
				Urdu	urd	Indo-European, Indic	7046
	Table 1 gives our split of languages into training	Dev	languages	Czech Greek Hebrew Hungarian Afrikans	ces ell heb hun afr	Indo-European, Slavic Indo-European, Greek Afro-Asiatic, Semitic Uralic, Ugric Indo-European, Germanic
	(15), development (4) and test (17) sets. The train-			Amharic Basque	amh Afro-Asiatic, Semitic eus Basque, Basque	7942 7958
	ing set contains the source languages used for the			Belarusian bel Bulgarian bul	Indo-European, Slavic Indo-European, Slavic	7958
	transfer, while the development set languages are			Hindi Indonesian ind hin	Indo-European, Indic Austronesian, Malayo-Sumbawan	7952
	used as targets for parameter tuning. Training and test languages represent diverse language families	Test	languages	Lithuanian lit Marathi mar Persian pes Portuguese pos	Indo-European, Baltic Indo-European, Indic Indo-European, Iranian Indo-European, Romance	7947 7931
	and diverse availability. Note that training and dev			Telugu Turkish	tel tur	Dravidian, South-Central Dravidian Altaic, Turkic
	languages are high-resource while test languages			Bambara Erzya	bam Mande, Western Mande myv Uralic, Mordvin	7958 7958
	are low-resource. For most of the test languages,			Manx Yoruba	glv yor	Indo-European, Celtic Niger-Congo, Defoid	3994
	there are fewer than 8000 verses available in the
	Parallel Bible Corpus; 4 for Manx, fewer than 4000.
	We evaluate POS tagging performance on Univer-
	sal Dependencies (UD) (Zeman et al., 2019) test
	sets. As UD and PBC tokenizations differ, we fur-
	ther adopt the following rule: if a PBC token cor-responds to a sequence of several UD tokens, we replace the sequence with the original word, tagged with the tag of the UD token in the sequence that is highest in the dependency tree (cf. (Agić et al.,	alignments using Eflomal (Östling and Tiedemann, 2016), 6 a high-quality statistical word aligner, with the "intersection" symmetrization heuristic. Other than parallel data, Eflomal does not need any super-vision signal; we can thus use it for any language
		pair in PBC. Details on models' hyperparameters
		are in Appendix A.3. All tagging results reported
		below are averages over three runs of the neural

2016)). To tag the high-resource training and dev languages, we use Stanza

[START_REF] Qi | Stanza: A python natural language processing toolkit for many human languages[END_REF]

, 5 a stateof-the-art NLP Python library. We create word

  Table 3 compares EFLOMAL, ESKANDER and GLP. In this table -Eng stands for when only English is used as the source language in GLP and -All stands for when all training languages are used (see §6.1).

	GLP outperforms both baselines in all cases but In-
	donesian, where ESKANDER is 0.7 points better.
	However, they tune their hyperparameters on this
	language using dev data while we only tune them
	on dev languages. Compared to ESKANDER, we
	use a simpler neural POS tagger and less resources,
	as we do not use affix embeddings nor word clus-
	ters. Our initial experiments indicated that word
	clusters were not helping in our setup. The higher
	quality of the annotated data created by GLP may
	already contain the information provided by word
	clusters.
	Table 4 compares AGIC, BUYS, CTRL, and
	GLP-SL. AGIC

Table 2 :

 2 Accuracy on UD v2.10 test for GLP-SL when transferring from all training source languages (i.e., GLP-SL-All). See the other tables for comparison with prior work, which uses older versions of UD.

					with XLM-R								without XLM-R
	afr amh	eus	bul	hin	ind	lit	pes	por	tel	tur	bel mar	bam myv	glv	yor
	87.7 82.4 70.9 90.1 81.8 85.3 85.7 81.8 89.2 83.8 80.1 85.9 87.9	65.4 64.4 63.9 59.9
			afr amh	eus	bul	hin	ind	lit	pes	por	tel	tur AVG	bel mar AVG
	EFLOMAL-Eng	73.7 74.9 60.4 78.9 58.1 72.4 80.3 59.2 74.1 77.5 67.6 70.6	76.2 73.2 71.3
	EFLOMAL-All		83.9 79.3 64.5 85.0 68.1 78.4 82.8 68.6 83.8 77.1 74.8 76.9	79.6 77.8 77.2
	ESKANDER-Eng 86.9 75.3 67.3 85.6 73.9 84.1 80.9 77.2 86.1 80.0 74.3 79.2
	ESKANDER-All 89.3 79.3 67.1 88.2 72.8 83.0 82.5 77.3 87.8 77.1 74.6 79.9
	GLP-B-Eng		86.6 81.9 67.5 85.7 76.8 82.7 81.1 76.2 87.6 82.5 76.4 80.4	80.0 82.3 80.6
	GLP-SL-Eng		84.4 81.9 68.6 84.0 75.8 81.3 81.0 73.5 86.4 80.6 75.8 79.4	75.1 81.5 79.2
	GLP-B-All		89.7 83.6 67.4 89.7 79.9 82.8 85.9 79.6 87.7 81.4 80.3 82.5	87.9 83.2 83.0
	GLP-SL-All		87.5 82.9 70.6 89.7 81.9 83.4 85.8 81.9 89.6 83.7 78.4 83.2	88.8 88.4 84.0

Table 3 :

 3 Accuracy on UD v2.5 test for EFLOMAL, ESKANDER[START_REF] Eskander | Unsupervised cross-lingual part-ofspeech tagging for truly low-resource scenarios[END_REF] and GLP. "-Eng": transfer from English only. "-All": transfer from all training languages (see[START_REF] Eskander | Unsupervised cross-lingual part-ofspeech tagging for truly low-resource scenarios[END_REF] and Table1). Bold: best score for each language.

		Target	AGIC	GLP-SL-All
		bul	70.0 mul	86.1
	v1.2	hin ind	50.5 mul 75.5 mul	79.0 79.5
		pes	33.7 mul	75.2
		por	84.2 mul	87.7
		Target	BUYS	GLP-SL-All
	v1.2	bul por	81.8 eng 88.0 esp	86.1 87.7
		Target	CTRL	GLP-SL-All
		Bul	68.8 rus-100	89.3
	v2.0	Bul Por	83.1 rus-1000 81.8 esp-100	89.3 90.1
		Por	88.9 esp-1000	90.1

Table 4 :

 4 Accuracy on UD test for AGIC

Table 3

 3

	reports results when

We do not use PBC-specific features. Thus, our work is in principle applicable to any multiparallel corpus.

Our code, data, and trained models are available at https: //github.com/ayyoobimani/GLP-POS

|L| might be different for different sentences.

Bible versions are described in Appendix A.1.

https://stanfordnlp.github.io/stanza/

7github.com/robert/eflomal

XLM-R embeddings are used even for languages unseen during its pretraining as they improve performance. This is probably due to the fact that some words (e.g., names) can be well represented even for an unseen language.

Obtained by crawling http://wol.jw.org

http://www.statmt.org/europarl/

Acknowledgements

We would like to thank Sebastian Ruder for his kind and thoughtful suggestions on this work.

This work was funded by the European Research Council (grant #740516) and the German Federal Ministry of Education and Research (BMBF, grant #01IS18036A).

A Reproducibility details

A.1 Data editions Table 7 lists the edition used for all the experiments in this paper.

A.2 Universal Dependency tests specification

A.3 Models parameters GLP

The GLP is implemented using the PyTorch geometric library. 11 All hyperparameters are tuned on the dev set. GLP-B has 2 layers of MLP of size 2048 while GLP-SL uses four layers of transformer with hidden size 2048 and 16 attention heads. Although we didn't observe a difference between different sizes from 512 to 2048. We tuned the learning rate, batch size, and γ (the self-learning threshold) over the validation languages. In GLP-B learning rate and batch size are respectively 0.001, 8, and in GLP-SL 0.00001, and 32. In general, when using XLM-R embeddings, the model has higher confidence, so the γ parameter is set to 0.95 when not using XLM-R embeddings and 0.98 when using XLM-R embeddings. The whole model needs about 16GB of GPU memory. GLP-B takes about 2 hours to train and GLP-SL about 12 hours. We 11 https://pytorch-geometric.readthedocs.io/en/ latest/ used early stopping with patience of 8 for both GLP-B and GLP-SL.

Neural POS tagger We run our method on up to 48 cores of Intel(R) Xeon(R) CPU E7-8857 v2 with 1TB memory and a single GeForce GTX 1080 GPU with 8GB memory. The POS tagger uses the Flair framework [START_REF] Akbik | FLAIR: An easy-to-use framework for state-of-theart NLP[END_REF] and Sequenc-eTagger model with 128 hidden size, the "xlmroberta-base" embeddings, and AdamW optimizer [START_REF] Loshchilov | Decoupled weight decay regularization[END_REF]. The hyperparameters, including the fixed number of epochs (15) are tuned using the UD development sets of the development languages. Each Neural POS tagger was trained in less than 30 minutes.

B Contextualized vs. Static embeddings

Table 9 shows results obtained with our GLP-SL with and without using XLM-R embeddings for projection. Note that the final neural POS tagger models always use XLM-R embeddings, even for languages unseen during XLM-R pretraining.