F Dubs

X Le Roux

V Allard

B Andrieu

S Barot

A Cantarel

C De Vallavielle-Pope

A Gauffreteau

I Goldringer

C Montagnier

T Pommier

E Porcher

S Saint-Jean

J Borg

S Bourdet-Massein

D Carmignac

A Duclouet

E Forst

N Galic

L Gerard

M Hugoni

A Hure

A Larue

J-C Lata

C Lecarpentier

M Leconte

E Le Saux

I Le Viol

P L'hote

P Lusley

M Mouchet

A Niboyet

R Perronne

E Pichot

S Pin

S Salmon

D Tropée

A Vergnes

T Vidal

J Enjalbert

An experimental design to test the effect of wheat variety mixtures on biodiversity and ecosystem services

Website: https://www6.inra.fr/wheatamix

The present document details how the Wheatamix consortium, inspired by ecological experiments exploring relationships between plant biodiversity and ecosystem functioning (e.g. the Jena experiment [START_REF] Weisser | Biodiversity effects on ecosystem functioning in a 15-year grassland experiment: patterns, mechanisms, and open questions[END_REF], selected bread wheat (Triticum aestivum L.) lines, phenotyped them across a range of functional traits and used this information to set up an experimental design able to unravel the effects of variety number and of the functional diversity and identity within variety mixtures for evaluating the impact of intraspecific crop diversity on a range of ecosystem services.

Wheat line selection

The Wheatamix project investigates the potential benefits of variety mixtures in the Paris basin wheat supply chain, and therefore focuses on varieties and lines adapted to the local climate. A consensus list of 57 wheat lines (Table 1) was thus settled on these grounds and to meet the expectations of agronomists, geneticists, phytopathologists and ecophysiologists of the group. This list is composed of i) 32 elite bread wheat varieties registered in the French catalogue, selected for their high yields under conventional farming, ii) 5 modern varieties bred for organic farming (OF), iii) 10 landraces resulting from farmers' mass-selection, cultivated in France in the early 1900es, and iv) 11 lines from an INRA-MAGIC multiparental and highly recombinant population [START_REF] Thepot | Efficiently tracking selection in a multiparental population: the case of earliness in wheat[END_REF], adapted to Northern France. Due to the heterogeneity of information available for each variety and line, various criteria were used for this selection. The 32 elite bread wheat varieties were chosen on the basis of their wide use in the Paris Basin, and to ensure representativeness of the diversity for earliness, disease resistance or bread-making quality, using the available information in the variety catalogue (https://www.geves.fr/catalogue/). These varieties originated from the principal breeding companies. Some elite varieties were also included because they are often used in experimental research (Soisson, Apache, Caphorn for example). Landraces and organic varieties were selected on the basis of their wide use in low input or organic farming systems, and for their diversity of traits (plant height, earliness...). Finally, the 11 Magic lines were selected among 1000 available lines, based on their yield and to offer a broad diversity for earliness, plant height and genetic diversity (on the basis of SNP genotyping data, see [START_REF] Thepot | Efficiently tracking selection in a multiparental population: the case of earliness in wheat[END_REF]. Seeds from all varieties and lines were multiplied in 2013 and 2014 in INRA-GQE, Le Moulon, Gif-sur-Yvette, France, to provide sufficient seeds for the various experiments performed during the Wheatamix project.

Multi-trait phenotyping of the variety panel

The Wheatamix panel was phenotyped by different teams of the Wheatamix consortium to characterize both agronomic (e.g. yield, earliness and disease resistance) and ecological (e.g. specific leaf area and root absorption capacity of mineral N forms) traits. A matrix of 27 traits was used to summarize the functional diversity of the variety panel (Table 2).

Multi-trait classification of the variety panel and selection of a sub-panel

To select a subset of varieties representative of the diversity of traits in the panel of 57 varieties, different multivariate clustering analyses were performed on the 27 trait matrix. Missing value imputation, for a total of 19 missing values across all traits and varieties, was done assuming that there was no cluster, that the data came from a single multivariate normal distribution, and that missing values were distributed at random. Using the Pairwise method, a single covariance matrix was formed for all the data. Then each missing value was imputed by a method that is equivalent to regression prediction using all the non-missing values as predictors. The lines were classified using either hierarchical clustering (Ward method, using JMP Pro v.13 SAS software) and a cutoff was set to separate six functional groups of varieties. Two of them were excluded because they were not stable across statistical analyses. The first branch in the tree roughly discriminates Landraces from modern varieties, and the landrace branch is itself strongly diverse, with 3 subtrees, as illustrated on the Fig. 1 with k=6. For practical reasons, we had decided to base our experimental design on only 4 groups and to obtain a comparable number of varieties /lines within the 4 selected subgroups, only one Landrace subtree was selected (the largest). We finally checked that the 4 remaining groups are stable, i.e. keep a similar composition when one of the 27 traits is removed (Jacknife) from the clustering analysis. The four functional groups retained, hereafter c1-c4, are presented in Fig. 1. Functional group c1 includes 14 varieties (6 MAGIC lines and 8 elite varieties) that are generally sensitive to fungal diseases and have a low potential for soil exploration/exploitation, as characterized by root traits and capacity for absorption of nitrate and ammonium. This group contrasts with functional group c4, containing 17 elite varieties resistant to fungal diseases and with a higher potential for soil exploration/exploitation. Functional group c3 includes 9 varieties (5 landraces and 4 organic varieties) characterized by their slow growth but elevated aggressiveness regarding plant-plant competition. Finally, functional group c2 is composed of 8 varieties (7 elite and 1 organic varieties), without obvious pattern in terms of functional traits. A sub-panel of four varieties within each functional group (Table 3), i.e. a total of 16 varieties, was selected, allowing manipulation of a reasonable number of varieties when choosing mixtures to set up the experimental design (next step). The choice of varieties within a functional group was constrained by seed availability. We also made sure that the 4-groups clustering remained robust after sub-sampling. This balanced contribution of each functional group maximized the overall functional diversity within the pool of 16 varieties. Among these, 7 were modern winter varieties, 5 were landraces or modern organic varieties, and 2 were INRA MAGIC lines. The first three axes of a principal component analysis implemented on wheat traits of the 16 varieties selected (Fig. 2) extracted 59.2% of the total variance (26.2%, 18.2% and 14.8%, respectively). As expected, the 16 varieties were clustered consistently with their functional groups: c1, c2 and c4 appeared on the positive side and c3 on the negative side of axis 1. Besides, c1 and c2-c4 were distinguished on the second axis with c2 standing on the negative side and c4 on the positive side of the third axis. The wheat varieties (Altigo, Trémie, F426 and A22) of the c1 functional group were characterised by sensitivity to fungal disease and low flag leaf nitrogen content. Functional group c2 was composed of wheat varieties (Renan, Skerzzo, Midas, Alauda) with short root length, high level of NO 3 -absorption and high relative growth rate. The c3 functional group was composed of tall wheat varieties (Blé Autrichien, Hermès, Maxi, Ritter) that are not very aggressive and have high NH 4 + absorption capacity. Finally, the c4 functional group contained varieties (Grapeli, Soissons, Arezzo, Boregar) with high specific root length, low relative growth rate and low NO 3 -absorption capacity (Table 3).

Variety mixtures used for the Wheatamix experimental plan

Using the 16 wheat varieties selected as detailed above, 72 different mixtures of varieties were created to explore a wide range of variety number and intraspecific functional diversity (Table 4). These include 24 different combinations of two varieties, 28 combinations of 4 varieties and 20 combinations of 8 varieties (Table 5). The 72 mixtures allowed us to explore extensively the gradient of functional diversity while using each of the 16 varieties in a perfectly balanced way at each richness level. For a given richness level (i.e. number of varieties), we varied the number of functional groups whenever possible (e.g. one or two functional groups in binary mixtures ; two to four functional groups in octonary mixtures ; see Table 5). This means that for each number of varieties in a mixture, the whole gradient of functional diversity was explored from mixtures with low functional diversity (only one functional group) to mixtures with low functional diversity (as many functional groups as possible given the number of varieties in the mixture). Three replicates of monocultures (here plots with a single variety) were used. However, as our objective was not to assess significant differences between pairs of mixtures but rather to quantify the effects of variety richness and functional diversity levels or functional group number, one replicate of each variety mixture was used.

Monocultures and mixtures were sown in Versailles (48°48'26"N 2°05'13"E) on November 2014. All plots were of identical size (10.5 m x 8.0 m) and divided into 6 sowing units of 1.75 m x 8 m. Each sowing unit consisted of 8 sowing lines spaced from each other leaving a 17.5 cm gap (Fig. 3). In autumn 2014, a seedbed was prepared by ploughing (20-30 cm deep) and the plots were sown in early November with monocultures or variety mixtures (see Table 4) at a density of 180 seeds m -². Each plot was buffered from adjacent plots or field edge by a 1.75 m-wide row of triticale (x Triticosecale) to standardize plot edge and possibly limit the dispersal of pathogen spores between neighbouring plots (Fig. 3). The crop was grown with a target yield of 60 q/ha (75 = French average national wheat yield in 2015). No insecticide and fungicide was used except for seed coating for which CELEST (0.2 l.quintal -1 -Fludioxonil 25g.l -1 ) and SIGNAM (60 g.quintal -1 -Cypermethryne 300

Fig. 1 :Fig. 2 :Fig. 3 :

 123 Fig. 1: Heatmap of the doubly ordered dendrograms done on wheat traits and lines. The shading from red to blue represents gradation from low to high trait values.

Table 1 :

 1 List of the 57 varieties phenotyped in the Wheatamix project

	Variety name	Functional group	Variety type Variety name	Functional group	Variety type
	Premio	c1	Elite	Sy Moisson	c1	Elite
	Altigo	c1	Elite	Trémie	c1	Elite
	Apache	c4	Elite	Tulip	c1	Elite
	Arezzo	c4	Elite	A22	c1	MAGIC
	Arlequin	c4	Elite	A160	c1	MAGIC
	Attlass	c4	Elite	A208		MAGIC
	Barok		Elite	A210		MAGIC
	Boregar	c4	Elite	F236		MAGIC
	Caphorn	c4	Elite	A243		MAGIC
	Fanion	c4	Elite	A248		MAGIC
	Farmeur	c1	Elite	A398	c1	MAGIC
	Flamenko	c4	Elite	F426	c1	MAGIC
	Folklor	c4	Elite	A446	c1	MAGIC
	Goncourt	c1	Elite	A490	c1	MAGIC
	Grapeli	c4	Elite	Blé Autrichien	c3	Landrace
	Isengrain	c4	Elite	Rouge de Bordeaux		Landrace
	Koreli	c2	Elite	Noé		Landrace
	Lyrik	c4	Elite	Barbu de Champagne	c3	Landrace
	Midas	c2	Elite	Alauda	c2	Organic
	Odyssee	c4	Elite	Hermès	c3	Organic
	Pakito	c1	Elite	Karneol	c3	Organic
	Quebon	c4	Elite	Ritter	c3	Organic
	Bermude	c4	Elite	Maxi	c3	Organic
	Renan	c2	Elite	Prince Albert	c3	Landrace
	Rubisko	c4	Elite	Rouge du Roc		Landrace
	Skerzzo	c2	Elite	Saint Priest		Landrace
	Sogood	c2	Elite	Sixt sur Aff	c3	Landrace
	Soissons	c4	Elite	Royo de Pamplona	c3	Landrace
	Solehio	c2	Elite

Elite: modern commercial variety, registered on the seed market for conventional agriculture. Organic: modern commercial variety registered for Organic Farming. Landrace: old traditional variety that evolved over decades and adapted locally under the unconscious selection of farmers, and was commonly cultivated till 1930 th . MAGIC: inbred lines developed from a multiparental (60 parents) and highly recombinant INRA population.

Table 2 :

 2 Traits measured on the 57 varieties of bread wheat

		Traits Unit			Meaning	Stage/age	Growth conditions	Location and year
							of the plant		of measurements
	Growth and allocation Nutrients contents and nitrogen cycling	SRR RDMC mg.g No dimension -1 RGR mg.day -1 RNC % SRL m.g -1 nit mg.g -1 .L -1 .min -1 amo mg.g -1 .L -1 .min -1 DEA µgN-N 2 O.g dry soil NL1 %	-1 .h	-1	Shoot root ratio Root Dry Matter content Relative Growth rate Root Nitrogen Content Specific Root Length --absorption NO 3 + absorption NH 4 Denitrification Flag leaf nitrogen content	8 weeks Flowering	Greenhouse conditions in 2L pots with sand and hydroponic solution, 3 plants per pot Field conditions, 170 plants m -²	Lyon, 2014 Grignon, 2014
		RD	mm			Mean root diameter	8 weeks	Greenhouse conditions; 2L pots with sand and hydroponic solution, 3 plants per pot	Lyon, 2014
	Architecture	RNb RA L1MD S4L	# Degree g.cm -² cm 2			Mean root number Mean root angle Flag leaf dry mass density Surface of the four superior leaves	6 weeks Flowering	Hydroponic growth in a 2D rhyzotron Field conditions, 170 plants m -²	Clermont, 2014 Grignon, 2014
		MSH	cm			Mean height of the main stem shoot	Grain filling	Field conditions, 100 plants m-²	Le Moulon, 2014
		GAIT1 No dimension			Green Area Index in December (ratio of leaf	Leaf 2 has
		GAIT6 No dimension			green area to the area of ground) Green Area Index in April (ratio of leaf green	emerged Stem	Field conditions, 170 plants m	-²
						area to the area of ground)	elongation		Grignon, 2014
		Agg	No dimension			Capacity of compensation between two
	Ground cover					seeding densities (Ratio of the ear density		Field conditions, 36 and 170 plant m-²
	capacity	EarD Agg2	Ears.m No dimension -²			between sowing at 36 and 170 plant m-²) Ear density per square meter Aggressiveness index, computed as the ratio between tillering in low density, high nitrogen growing conditions (Grignon D2), and tillering	Grain filling	Field conditions, 150 plants m Field conditions, 150 plants m -2 and 170 plants m -2 (Le Moulon) -2 (Grignon)	Le Moulon, 2014 Le Moulon & Grignon, 2014
						in high density, low nitrogen (Le Moulon).
		Yr	%			Sensitivity to yellow rust, percentage of the leaf
	Disease	Septo	%			surface attacked Sensitivity to septoria, percentage of the leaf	Tillering to flowering	Compilation of data from ARVALIS, and the ECOGER and ECOSYS laboratories
						surface attacked
	Yield components	VEL EarP FD TKW KEar	No dimension Ears/plant -1 Days g Kernels.ear -1			Vertical coefficient of extinction of light Mean number of ears per plant Flowering date Thousand Kernels Weight Mean number of kernels per ear	Stem elongation Post Harvest	Field conditions, 170 plants m Field conditions, 100 plants m-2 -²	Grignon, 2014 Le Moulon, 2014

Table 3 :

 3 Description of functional groups

Variety name Functional groups Description

	Altigo	c1
	Trémie
	F426
	A22	Varieties sensitive to fungal
		diseases (Septo, YR) and with
		weak flag leaf nitrogen content
		(NL1)
		Functional group sensitive to
		fungal diseases
	Renan	c2
	Skerzzo
	Midas
	Alauda	Varieties with short roots (SRL),
		high level of NO 3 -absorption
		(nit) and high relative growth
		rate (RGR)
		Functional group with limited
		soil exploration
	Blé Autrichien	c3
	Hermès
	Maxi
	Ritter	Varieties with tall main stem
		(MSH), with high level of NH 4 +
		absorption (amo) and not very
		aggressive (Agg)

Functional group with slow phenology

	Grapeli	c4
	Soissons
	Arezzo
	Boregar	Varieties with long roots (SRL), low level of NO 3 -absorption
		(nit) and low relative growth
		rate (RGR)
		Functional group with good
		soil exploration

Table 4 :

 4 Synthetic view on the experimental plan used, with the list of all plots indicating for each plot their composition in term of variety number, number of variety groups varieties present (V1: Altigo, V2: Trémie, V3: F426, V4: A22, V5: Renan, V6: Skerzzo, V7: Midas, V8 Alauda, V9: Blé Autrichien, V10: Hermès, V11: Maxi, V12: Ritter, V13: Grapeli, V14: Soissons, V15: Arezzoet V16: Boregar). The number of replicate of each plot is also indicated.

		Functional groups
	c1	c2	c3	c4

Table 5 :

 5 Number of plots for each combination of variety number and number of functional groups. Numbers in italic refer to replicates of monocultures already counted once in the plain figures.

				Number of
			functional groups
			1	2	3	4
		1	16x3				16+32
	Number of	2	8	16			24
	varieties	4	4	8	8	8	28
		8		6	8	6	20
			28+	30	16 14
			32

g.l -1 ) were used. One spraying of an herbicide (Archipel® and Harmony Extra®) was performed on March 14, 2015. 160 kgN.ha -1 was used as compared to the estimated optimal amount of 180 kgN.ha - 1 (Carlotti 1992). The fertilizer (ammonium-nitrate) was spread as follows: 40 kgN.ha -1 on March 5, 2015, 80 kgN.ha -1 on April 16, 2015, 40 kgN.ha -1 on May 11, 2015. All plots were harvested between the last week of July and the first week of August 2015.


Cédric Maigrot

Ewa Kijak

Vincent Claveau

CONTEXT-AWARE FORGERY LOCALIZATION IN SOCIAL-MEDIA IMAGES: A FEATURE-BASED APPROACH EVALUATION

Keywords: Forgery localization, Image forensics, Splicing, Context-aware digital forensics, Image tampering

In this paper, we study context-aware methods to localize tamperings in images from social media. The problem is defined as a comparison between image pairs: an near-duplicate image retrieved from the network and a tampered version. We propose a method based on local features matching, followed by a kernel density estimation, that we compare to recent similar approaches. The proposed approaches are evaluated on two dedicated datasets containing a variety of representative tamperings in images from social media, with difficult examples. Context-aware methods are proven to be better than blind image forensics approach. However, the evaluation allows to analyze the strengths and weaknesses of the contextual-based methods on realistic datasets.

INTRODUCTION

Nowadays images are widely spread through social networks and the dissemination of false information is a scourge. In particular, modifying image content is an increasingly popular practice thanks to the widely available, user-friendly image editing software. Therefore, there is a real need for lay or professional users to have tools facilitating the detection of fraudulent images. To address the task of image content verification, any piece of information is valuable, such as geolocation, camera model, contextual information, applied filters, and explicit tampering detection. Recently, a few frameworks or platforms tackle this problem [START_REF] Pasquini | Towards the verification of image integrity in online news[END_REF][START_REF] Zampoglou | Web and social media image forensics for news professionals[END_REF][START_REF] Zampoglou | Detecting image splicing in the wild (WEB)[END_REF].

Various types of modification can occur. One can distinguish malicious from non-malicious tamperings. The first ones try to change the way humans interpret the content of the images: some visual elements are added, removed, or changed, such that the original semantic of the image is altered. This is typically performed by a copy-move operation (a part of the image is replicated at another location in the same image), or by a copy-paste operation (or splicing; the inserted object comes from another image). In contrast, nonmalicious tamperings are more concerned with the aesthetics of the images Filtering, cropping, rescaling or histogram adjustments are examples of broad-scope image operation, which are generally considered as non-malicious tamperings in the context of images circulating on the Web.

Tampering detection and localization in images has drawn a lot of attention. Passive, blind image forensics are based on single image analysis. Many efforts have been devoted to copy-move detection, where the general principle consists in finding similar blocks, or regions, in the image. Such approaches rely on blocks or keypoints matching to find the highest similarities in the image. Then, a post-processing step groups the matched pairs into coherent candidate regions. The estimation of the parameters of possible affine transforms is used to ensure the robustness of the approach to transformations [START_REF] Christlein | An evaluation of popular copy-move forgery detection approaches[END_REF][START_REF] Pun | Image forgery detection using adaptive oversegmentation and feature point matching[END_REF][START_REF] Cozzolino | Efficient dense-field copy-move forgery detection[END_REF][START_REF] Amerini | A sift-based forensic method for copy-move attack detection and transformation recovery[END_REF]. Unlike copy-move, splicing detection relies on the assumption that the spliced area significantly differs from the rest of the image. Identifying local discrepancies can rely on traces left by JPEG compression [START_REF] Lin | Fast, automatic and fine-grained tampered jpeg image detection via dct coefficient analysis[END_REF][START_REF] Bianchi | Image forgery localization via block-grained analysis of jpeg artifacts[END_REF][START_REF] Amerini | Splicing forgeries localization through the use of first digit features[END_REF][START_REF] Farid | Exposing digital forgeries from jpeg ghosts[END_REF], Color Filter Array (CFA) interpolation [START_REF] Ferrara | Image forgery localization via fine-grained analysis of cfa artifacts[END_REF], or noise patterns [START_REF] Cozzolino | Splicebuster: A new blind image splicing detector[END_REF][START_REF] Mahdian | Using noise inconsistencies for blind image forensics[END_REF]. Tampering localization methods generally produce a Tampering Heat Map (THM) that indicates regions of the image where tampering is likely to occur. In this work, we are interested in the problem of splicing localization on images found on the Web and social media environments. Such images have particular characteristics. They are frequently subjected to transformations, such as rescaling, filtering, cropping, format conversion, recompression. Users often modify an image before re-posting it, and some image publishing platforms, e.g. Facebook or Twitter, automatically operate such transformations, e.g. scaling, changing format to JPEG, varying quality, and erasing metadata. These images, posted again and again by users are challenging traditional signal-based method for tampering detection, which are known to perform poorly in this context [START_REF] Zampoglou | Large-scale evaluation of splicing localization algorithms for web images[END_REF][START_REF] Brogan | Spotting the difference: Context retrieval and analysis for improved forgery detection and localization[END_REF]. While such methods could help in some cases, they certainly can not cope with all the range of user and system visual edits, suggesting that complementary mechanisms should be used to uncover more forgeries.

An emerging paradigm in verifying information from the social media is that several versions of an image are likely to be found on the Web. Finding these images facilitates the tampering detection by using comparisons. Reverse image search can often retrieve near-duplicate versions of the image to be analyzed and then serve as a basis for further manual inspection [START_REF] Pasquini | Towards the verification of image integrity in online news[END_REF], or as an input for automatic tampering detection based on comparison between pairs of images [START_REF] Amerini | A feature-based forensic procedure for splicing forgeries detection[END_REF][START_REF] Maigrot | Tampering detection and localization in images from social networks: A cbir approach[END_REF][START_REF] Brogan | Spotting the difference: Context retrieval and analysis for improved forgery detection and localization[END_REF][START_REF] Gaborini | Multi-clue image tampering localization[END_REF][START_REF] Bestagini | Image phylogeny tree reconstruction based on region selection[END_REF]. The focus of this paper is on this latter case. The paper proposes an approach based on local features in order to localize tampering in images from social media, under the assumption that we are able to crawl the web for similar images and that the source image can be retrieved. In Section 2, we propose a splicing localization method based on matching of local features. Section 3 presents the experimental setup and introduces a realistic dataset we collected, containing characteristic difficult examples and available online for free academic usage and accessible reproducible research. We also use another recent dataset to evaluate the proposed method as well as methods from the state of the art in Section 4, to analyze the strengths and weaknesses of the contextual-based methods on realistic datasets.

FORGERY LOCALIZATION METHOD

Localizing regions in an image where tamperings are likely is based on detecting the inconsistencies resulting from a detailed comparison of two versions of the same image: the probe image F and a retrieved near-duplicate C. In the datasets used for evaluation (see section 3), F is the forged image while C is considered as the original one. The tampered areas in images from the web and social medias have most likely undergone transformations such as rotations, scaling, cropping, or affine transformations. The proposed method hence relies on the matching of local features that are robust to such transformations. Matches are then used to both estimate the homography H between the two images, and identify outliers as matches that do not fit H.

1. Local Features Matching based approach (LFM) In our implementation, we use SURF as local features. SURF features are matched from F to C according to Lowe's criterion as introduced in [START_REF] Lowe | Distinctive image features from scaleinvariant keypoints[END_REF] that allows to discard most of the false matches. Based on these matches, the homography H between the two images is estimated using RANSAC, which is robust to outliers. We evaluate both dense and detected keypoints extraction. Keypoint detection often results in a higher rate of matches fitting the homography, but provides fewer points, and consequently less matches, which causes troubles in many images. Dense keypoint extraction is retained (stride of 10 pixels) and descriptors are computed over 4 different scales, as in other recognition problems [START_REF] Chatfield | The devil is in the details: an evaluation of recent feature encoding methods[END_REF].

Once H estimated, outliers are the matches H does not explain. The best candidate match for each keypoint of F is its nearest neighbor in the keypoints from C. Dense sampling ensures dense outliers detection, and therefore facilitates their localization. A keypoint of F is considered as an outlier if the distance between its re-projection in C according to H and the matched keypoint for all scales is above 0.1×diag, where diag is the diagonal size of the image. A keypoint is an inlier if at least one matched keypoint at one scale is correctly located according to H (Fig. 1d).

The THM is finally obtained by estimating the density of outliers, using a Kernel Density Estimation with a gaussian kernel (gKDE), the bandwidth being selected by Scott's Rule of thumb. The objective of this last step is to identify the spatially close outliers and remove the isolated ones. Examples are given in Figs. 1e and1f. We compare this approach to a simpler one, called morpho, consisting in applying morphological operators (dilation, open-close filtering, hole filling) to the binary map given by the outlier points. This can be very roughly seen as an approximation of a kernel density estimation with a uniform (rectangular) kernel (Fig. 1g).

In some particular cases, H is wrongly estimated: when F has been flipped, or when the retrieved original image C is a cropped version of F: the area in F that does not exist in C will be detected as a tampering while it's not a splicing. These cases are easily detectable after the estimation of H, and prevented by flipping or cropping F according to H.

2. Pixel-wise comparison An image comparison framework was also proposed in [START_REF] Brogan | Spotting the difference: Context retrieval and analysis for improved forgery detection and localization[END_REF]. We compare the LFM approach to the two best performing methods in [START_REF] Brogan | Spotting the difference: Context retrieval and analysis for improved forgery detection and localization[END_REF], namely IRPSNR (based on Peak Signal to Noise Ratio), and SSIM (based on Structural Similarity Index Measure). We use the code and parameters provided by the authors. For IRPSNR, the THM is computed as the pixel-wise PSNR between the Gaussian blurred versions of the two images (with standard deviation σ = 4). The SSIM THM is the pixel-wise SSIM between the two images, using a neighborhood radius of 32 pixels. In both methods, low values of the THM indicate probable tampered areas. As these methods are based on pixel-wise computations, a warp operation is required to transform one of the images. In [START_REF] Brogan | Spotting the difference: Context retrieval and analysis for improved forgery detection and localization[END_REF], C is warped according to the homography H that map points of C to the coordinate system of F. In our experiments, as for the LFM approach, F can be warped when C is detected as a cropped version of F.

EXPERIMENTAL SETUP

1. Image datasets. We use two datasets dedicated to splicing localization. 1) The WildWeb dataset (Wi), proposed by [START_REF] Zampoglou | Detecting image splicing in the wild (WEB)[END_REF][START_REF] Zampoglou | Large-scale evaluation of splicing localization algorithms for web images[END_REF], contains real-world forgeries that have circulated on the web in the last years, accompanied by their original, untampered sources, when those were found by reverseimage search engines. We filtered out cases for which the original, untampered sources are not available and end up with 82 cases of forgeries (65 unique cases) along with their unspliced sources and ground-truth binary masks. In case there are multiple ground truths for a forged image (corresponding to different splices, possibly committed at different times), we merge them. 2) The Reddit dataset (Re), that we built and made publicly available 1 , is composed of 107 photoshopped images and their original version from the Photoshop challenges on the Reddit website2 , on which we manually annotated the tampered regions. The tampering operations are mainly splicing of various size, from very small tampered areas to very large ones, when for example the whole background of the original image has been replaced. Following the policy used on download media platforms such as Twitter or Facebook, large images are scaled down such that one of the dimension is at most 900 pixels. Fig. 2 shows examples of the 189 image pairs obtained.

2. Performance of the localization. Evaluation of the performance consists in comparing a ground-truth mask with a binarized THM, and is thus sensitive to both the binarization threshold and the quality of the ground-truth mask. Different metrics are used, each evaluating different properties. For each metric, the THMs were binarized using multiple threshold values covering the entire range of possible map values. Pixel-wise score: One common evaluation method for localization is based on pixel-wise comparison between binarized output and the ground-truth mask. The tampering localization is evaluated by the False Positive Rate (FPR) and the False Negative Rate (FNR), which should both be minimized. False positives pixels may either come from an unfavorable binarization threshold giving regions too large compared to the ground-truth, or from a wrongly detected region. Thus, the same error rate does not have the same meaning in both cases. We can make the same observation for false negatives. Another inherent difficulty is that such evaluation does not take into account the connected components and the ability to separate different (close) objects.

Connected component (CC)-wise score: For these reasons, we also propose a metric based on connected component (CC) that penalizes over-detections. A one-to-one assignment is made between the detected output CCs and the ground-truth CCs. A detected CC is considered as a correct detection if its overlap with a ground-truth CC, defined by the intersectionover-union (IoU) criterion, is greater than 0.5. Hence each detected CC is either a true positive or a false positive, and each ground-truth CC is either a true positive or a false negative. In CC-wise comparison, we compute the precision, the recall and the F-measure which should all be maximized.

WW score: For the sake of comparison, we also use the evaluation methodology presented in [START_REF] Zampoglou | Detecting image splicing in the wild (WEB)[END_REF]. The similarity between a binarized THM and the ground-thruth mask is also a per-pixel performance E (see [START_REF] Zampoglou | Detecting image splicing in the wild (WEB)[END_REF]). Any binarized THM achieving an E larger than a given threshold is considered as an accurate detection. Algorithm performance is given by the number of cases considered as correctly detected.

RESULTS AND ANALYSIS

Contextual based methods efficiency. We evaluate the three presented contextual methods on the Wi dataset, using the same evaluation protocol as in [START_REF] Zampoglou | Detecting image splicing in the wild (WEB)[END_REF]. Table 1 presents the results in terms of the number of detected cases w.r.t. the total number of considered cases, along with the three best performances reported in [START_REF] Zampoglou | Detecting image splicing in the wild (WEB)[END_REF]. This confirms the importance of using contextual clues when they are available. Tamperings localization. Evaluation results at pixel level are given in Fig. 3 for the two datasets. The binarization threshold for the THMs is controlled by a parameter σ, ranging from 0 to 100, which represents the percentage of the maximum possible value T HM max . For LFM, the larger σ, the smaller the detected area, while the opposite applies for IRPSNR and SSIM. LFM performs slightly better than SSIM on Reddit and IRPSNR is always worse. On the other hand, the CC-based evaluation, reported in Tab. 2, gives another perspective. The gaussian KDE approach used by LFM to cluster outliers tends to produce large and homogeneous detected areas. In contrast, IRPSNR and SSIM tend to over-segment, which is penalized by the CC-wise score. This behaviour is illustrated in Fig. 4. Even if the binary maps can be improved with morphological operations, this post-processing step is sensitive to the structuring elements used and induces new wrong localizations while correcting some defects.

Sensitivity to matching. The studied approaches perform relatively well, strongly depend on the quality of the estimated homography H between F and C; For IRPSNR and SSIM because they rely on pixel-wise comparison and thus on the quality of the warping, and for LFM because the THM is based on outliers w.r.t. H. IRPSNR and SSIM have to be particularly careful to the registration step, and simply using a warp from F to C as in [START_REF] Brogan | Spotting the difference: Context retrieval and analysis for improved forgery detection and localization[END_REF] is often insufficient, when dealing with images cropped in different ways.

Results are very different between Wi and Re. We note two main causes of failure on Wi. The first is when the proposed near-duplicate image C is quite different from the forged one F (e.g. same location but different point of view, or different period). The matching, and therefore the homography estimation, usually fail. The second is when the near- duplicate image C is the donor i.e. the image containing the (usually small) element used for the splicing (alien), and not the host image (i.e. which received the alien, producing the forgery). When it occurs the comparable part between the images F and C corresponds to the alien, and the forgery will be detected as all the rest of the image.

CONCLUSION

In this paper, we propose a context-aware method to localize tamperings in images from social media. The tackled problem is placed in the context of comparison of images, assuming to have an original version of the image to be analyzed, retrieved on the Net. The method is based on local features (e.g. SURF) matching. KDE or morphological postprocessings are proposed and compared for producing THMs. This approach is compared to state-of-the-art methods. We evaluate the approaches on two datasets containing real complex examples covering a large variety of tampering in images from social media. Our study allows us to observe the general strengths and limitations of local feature-based approaches, and underline the sensitivity of the results to the evaluation protocol. The proposed task and datasets are very challenging and should grow in interest in the community as there is room for improvements.

Fig. 1 .

 1 Fig. 1. Example of results of the tampering detection method. In (d): outliers key points are in blue, inliers in green.

Fig. 2 .

 2 Fig. 2. Examples of 3 pairs of images: Example (a) corresponds to images from Wi, examples (b) and (c) correspond to images from Re. For each pair, the original image is on the left side, and the tampered one is on the right.

Fig. 3 .

 3 Fig. 3. ROC curve of generated binary THMs, with Pixelwise score on Wi and Re datasets. Red and blue triangles are performances of LFM morpho, and IRPSNR binarized with a fixed threshold set to the mean THM value.

  (a) LFM prediction, IRPSNR prediction, groundthruth (b) LFM prediction, SSIM prediction, groundthruth

Fig. 4 .

 4 Fig. 4. Two examples of errors of methods IRPSNR and SSIM compared with the LFM binarized prediction. Red areas are the predictions of methods and groundthruth.

Table 1 .

 1 Tampering localization on Wi: ratio between the number of detected cases and the number of cases, using WW score with E > 0.45

	LFM	IRPSNR	SSIM	GHO [11]	NOI1 [14]	ADQ1 [8]
	0.60	0.72	0.79	0.35	0.18	0.16

Table 2 .

 2 F1-measure on Wi and Re, with CC-wise score based on the best possible binary THM per image.

	Dataset	LFM	IRPSNR	SSIM
	Re	0.84	0.62	0.69
	Wi	0.21	0.40	0.21
	All images	0.63	0.53	0.48

http://hoaxdetector.irisa.fr/

http://www.reddit.com