Cédric Maigrot

Ewa Kijak

Ronan Sicre

Vincent Claveau

Tampering detection and localization in images from social networks : A CBIR approach

Tampering detection and localization in images from social networks : A CBIR approach Cédric Maigrot Ewa Kijak Ronan Sicre Vincent Claveau {firstname}.{lastname}@irisa.fr

Tampering detection and localization in images from social networks : A CBIR approach Cédric Maigrot Ewa Kijak Ronan Sicre Vincent Claveau {firstname}.{lastname}@irisa.fr

Step 1 : Content-based image retrieval system

• Goal : Find the best image in our database for a comparaison with the query

• Method :

1. Find the 10 most similar images with a dot product using the descriptors explain below. Research is accelerated with a KD-Tree approach 2. Reorder these 10 candidates in order to find the best candidate.

Define an homography between the image query and each candidate in order to found the best homography

Descriptors used based on VGG19 [1]

• Two sizes used : training images size or based on a kernelization step like [START_REF] Tolias | Particular object retrieval with integral max-pooling of CNN activations[END_REF] • Three vectors are analyzed based on the output of three layers: last convolutional layer C 5 with a output lenght of 512 and the fullyconnected layers C 6 and C 7 with a output lenght of 4096 for both

• A mean or max pooling have to be apply in the case of C 5 when we used the standard training images size and the three outputs in case of kernelized approach • Twitter : 152,754 + 16

• Me [START_REF] Boididou | Verifying multimedia use at mediaeval 2016[END_REF] : 310

• Reddit : 128

• F2000 [6] : 1,300
• SATS-130 [START_REF] Christlein | On rotation invariance in copy-move forgery detection[END_REF] : 50

Example

Query : Candidates :

Rank #1 Rank #2 Rank #3 Rank #4
Results of the first step

Step 2 : Homogaphy estimation

• Goal : compare the two images and find the forged area if there is one

• Method :

1. Match keypoints from the images by similarity. 2. Define an homography based on RANSAC [START_REF] Fischer | A paradigm for model-fitting with applications to image analysis and automated cartography[END_REF] and find the outliers which correspond to matching not verifying the homography 3. Define a density map based on the list of outliers and binarize the map with a known threshold.

Example

Outliers map Density map Mask predicted Ground truth mask

Results of the second step Refs

F600 [ 3 ]

 3 : 160, forged F600 [3] : 400, original Twitter : 23, forged Holidays [4] : 1491, negative queries Database: (Total : 154,558 images)


Cédric Maigrot

Ewa Kijak

Vincent Claveau

Should I trust this image ?

Is there any visual information relevant to identify the original image?

This work is partly supported by the Direction Générale de l'Armement, France (DGA).