Maysoon Alkhair
email: maysoonalkhier111@gmail.com

Abdelouahab Hocini
email: abdelouahab.hocini@loria.fr

Kamel Smaili
email: smaili@loria.fr

Spotting fake news in Arabic with Machine and Deep Learning Techniques

Keywords: Rumors, classifiers, Arabic Fake news corpus, Machine learning, Deep learning

The rapid spread of rumors due to the growth of the internet and social media has prompted researchers to search for solutions to detect fake news, which is treated as a text classification problem. In this study, we examine the content of fake news in the Arabic-speaking world through YouTube comments. To start, we have updated our Arabic corpus for fake news analysis, incorporating the most frequently discussed topics in rumors. Next, we conducted experiments to determine the best combination of preprocessing, classical machine learning, and neural networks in classifying comments as either rumors or non-rumors. The models we used include Support Vector Machine (SVM), Decision Tree (DT), Multinomial Naïve Bayes (MNB) for machine learning, and LSTM, BILSTM, and CNN for deep learning. Finally, we compare the results of previous machine learning models and deep learning techniques to determine which one is more effective in detecting fake news. Both models showed high accuracy in the results.

Introduction

Internet has changed the whole planet, it introduced new means of communication between people all around the globe mainly for two reasons: its inexpensiveness and lack of restrictions. This wide network opened new doors for fast and easy information transmission and it became the user's favourite source of news replacing the classic media. This fast growth, has led to new concerns and challenges: the spread of false stories (fake news) [START_REF] Allcott | Social media and fake news in the 2016 election[END_REF][START_REF] Rasool | Multilabel fake news detection using multi-layered supervised learning[END_REF]. To make things worse, the fact that internet users can freely express their own opinion on any given topic and share it on many networks, platforms and with different people, generates a huge amount of unverified information [START_REF] Chomsky | A propaganda model[END_REF]. In order to mitigate the problem of the fast and wild spread of false information, research works proposed automated or semi-automated techniques for the evaluation of the veracity of these fake stories. The first step to these processes of evaluation is usually the extraction of data from multiple social networks such as Twitter, Facebook, Youtube and other [START_REF] Zubiaga | Analysing how people orient to and spread rumours in social media by looking at conversational threads[END_REF][START_REF] Kp | Detecting misinformation in online social networks using cognitive psychology[END_REF]. These last networks are considered as the main source of fake news that spread troughout the web. These news could take multiple forms: junk news, bogus and hoaxes are examples of the many other forms of fake news on Internet and on classical media [START_REF] Helmstetter | Weakly supervised learning for fake news detection on twitter[END_REF]. Fake news could be characterised and differentiated from real news. According to [START_REF] Zhang | An overview of online fake news: Characterization, detection, and discussion[END_REF], fake news has a spreading pace that is much faster than of the real news'. In addition to that, in [START_REF] Park | Social media hoaxes, political ideology, and the role of issue confidence[END_REF] the authors describe fake news as false news that are made to be believed true for a specific purpose. A very widely known example for this last point is the US presidential elections in 2016 [START_REF] Shu | Fake news detection on social media: A data mining perspective[END_REF].

In this work, we investigate the content of fake news in the Arabic world through the information posted on YouTube. We selected fake news concerning the death of three known Arab personalities: the dancer Fifi Abdou, the ex Algerian president Bouteflika Abdelaziz and the comedian Adel Imam. The two main objectives of this work are: first, to crawl Arabic rumors in order to build a corpus that we will share with the international community. Then, present a comparative analysis on the performances of existing machine learning and deep learning classification methods for fake news detection on the built corpus.

The rest of this article is structured as follows: in Section 2, we discuss research works related to the current study. Then we give an overview of the datasets to build corpus rumors analysis and extraction. Thereafter, we give details about the machine and deep learning approaches that are described in section3. In Section 4, we present comparative results of the machine and deep learning classification algorithms and finally, "Conclusion and future work" presents the conclusion and future work on the problem.

Related Works

In this section, we provide recent research into social media rumors with a focus on comparability methods that were widely used to identify similar data related to the same rumors when the dataset was collected. Fake news detection research is limited, mostly centered around the 2016 US presidential elections, and lacks consistent features. It mainly employs machine and deep learning techniques based on the defined features.

Machine learning

Hadeer et al. in [START_REF] Ahmed | Detecting opinion spams and fake news using text classification[END_REF] proposed a four-step pipeline for fake news detection: preprocessing the dataset, extracting n-gram features to represent the document, training the classifier, and performing classification.In this work six machine learning algorithms have been investigated, the best performing was LSVM. In [START_REF] Reis | Supervised learning for fake news detection[END_REF], hand-crafted features are employed, extracted from the news content, source, and context.Then a variety of machine learning models have been tested against a dataset consisting of 2282 BuzzFeed news articles related to the 2016 U.S. election with a split of 73% -27% true-false. The best results were achieved with Random Forest and XGBoost using a threshold-based decision. In [START_REF] Pérez-Rosas | Automatic detection of fake news[END_REF], two datasets were gathered: one crowdsourced and the other obtained from public figures' websites. An SVM was trained with a five-fold cross-validation, where one dataset was used for training and the other for testing, yielding an accuracy of 67%. In [START_REF] Pal | Comparative performance of machine learning algorithms for fake news detection[END_REF], the authors introduced 163 features across 5 categories (ngrams count, tf-idf, word embedding, sentiment polarity, and Linguistics). They then tested 7 ML algorithms and found XGBoost to be the best performer.

Deep learning

Fake news detection is a challenging problem that can be tackled using deep learning techniques. One approach is to use neural networks to analyze the text of an article or news item and identify patterns that are indicative of fake news, such as the use of certain words or phrases, the sentiment of the text, or the credibility of the source. Another approach is to use deep learning to analyze images and videos that accompany a news item, in order to detect signs of manipulation or forgery.

In [START_REF] Yang | liar, liar pants on fire[END_REF] researchers presented a new benchmark dataset collected from fact-checked news claims, the dataset is composed of 12.8k instances and use 6 labels (Pants-fire, False, Barely-true, Half-true, Mostly-true, True). A hybrid Convolutional Neural Networks framework that integrates text and metadata has also been proposed. In this work, CNNs and Bi-LSTM are utilized to extract features from both news text and metadata. However, the main challenge is overfitting, with validation and test accuracy around 27%. Similarly, [START_REF] Agarwal | Fake news detection using a blend of neural networks: An application of deep learning[END_REF] employed automatic feature extraction. The authors utilized a combination of Kaggle's and GrorgeMcIntire's instances as the new dataset. Features were extracted using DOC2Vec and Word Embedding, followed by classification using a Deep CNN and an LSTM. The CNN demonstrated the best performance with an accuracy of 94.22%. [START_REF] Abedalla | A closer look at fake news detection: A deep learning perspective[END_REF] addresses the fake news challenge (FNC-1) by utilizing a deep learning model on a dataset. The article body and headline are utilized as prediction features and an accuracy of 71.2% was achieved.

Methods and Techniques

The following section outlines the details of the dataset collected for this study, including its updated version, the fake news features in the dataset, and the various Machine Learning and Deep Learning algorithms utilized for classification.

Overview on the dataset

In a previous study [START_REF] Alkhair | An arabic corpus of fake news: Collection, analysis and classification[END_REF], we explored the nature of fake news in the Arabic world by analyzing the content posted on YouTube. The data was collected using the YouTube API, which enabled us to search for videos about the death of personalities. This focus was chosen as rumors about the death of celebrities are often associated with fake news. The study selected three prominent figures in the Arab world who are frequently subject to such rumors: Fifi Abdu (Egyptian dancer), Abdelaziz Bouteflika (former Algerian president), and Adel Imam (Egyptian comedian).

Table 3.1 in the previous study shows the cleaned dataset, with |C| representing the number of comments for each topic. In this new contribution, we expand the dataset by gathering more comments for each topic, increasing its size by 44% with the addition of 1509 new comments, as shown in table 3.1.

The figure [START_REF] Allcott | Social media and fake news in the 2016 election[END_REF] illustrates the distribution of the collected comments across the three topics. The dataset is imbalanced, with comments about Fifi Abdu's death accounting for 55 The data collected on rumors reveals that the President and the comedian's death garnered less attention than the death of Fifi Abdo, despite the President being a head of state. This highlights the greater interest of internet users in Fifi Abdo, both in terms of rumors and non-rumors. The tables (3 and 4) show the statistics and updated totals of rumors, and (5 and 6) show the same for non-rumors. The chart in Figure 2 illustrates the distribution of the sample datasets, real or fake. As there is a lack of sufficient samples in the dataset for a Deep Learning model to be built from scratch, one approach tested was evaluating the ability of a Deep Learning model trained on the Arabic Algerian corpus to detect fake Arabic.

Classification task 4.1 Machine Learning

This work tested the ability to differentiate between rumor and non-rumor comments using three data classification methods: Decision Tree (DT), Multinomial Naive Bayes (MNB), and Support Vector Machine (SVM). SVM [START_REF] Vapnik | Statistical learning theory[END_REF] aims to find the best linear separating hyperplane that separates the data into two classes. This is done by maximizing the distance between the classes. Classification of new data is based on which side of the hyperplane it falls on. We used a linear kernel for separation. Naive Bayes classifiers are popular in NLP, especially for text classification [START_REF] Mccallum | A comparison of event models for naive bayes text classification[END_REF] [START_REF] Su | Large scale text classification using semi-supervised multinomial naive bayes[END_REF] due to their efficiency and good predictive performance. MNB estimates the conditional probability of a particular term given a class.To train the MNB classifier, we used 1-gram, 2-gram and 3-gram of words as features supported by a TFIDF vector scores Fig. 2. Sample distribution for the resulting datasets in fake news and real news [START_REF] Spärck | A Statistical Interpretation of Term Specificity and Its Application in Retrieval[END_REF]. We conducted experiments and evaluated classifiers using common Information Retrieval metrics: Accuracy, Recall, and Precision.

Deep Learning

Deep learning is a branch of machine learning that surpasses traditional machine learning in its ability to automatically identify patterns in raw data without the need for manual feature extraction, in contrast to traditional machine learning, which still requires human aid in feature extraction [START_REF] Chowanda | Recurrent neural network to deep learn conversation in indonesian[END_REF]. In this study, we delved deeper into the classification of rumors and non-rumors using three advanced deep learning techniques: Long Short-Term Memory (LSTM), Bidirectional Long Short-Term Memory (BILSTM), and Convolution Neural Network (CNN). LSTM (Long Short-Term Memory) is a popular type of Recurrent Neural Network (RNN) that is widely used for pattern recognition and improved performance in text classification and sequential data prediction tasks d [START_REF] Ralf | Understanding lstm-a tutorial into long short-term memory recurrent neural networks[END_REF]. LSTM comprises of layers that allow it to selectively retain relevant information, making it more effective for text classification and solving sequential data prediction problems. The Bidirectional LSTM (BILSTM) model is architecture consists of two Long Short-Term Memory (LSTM) cells that are on both side to carry the information and run in parallel. So this method is called The forward and backward hidden states of the Bidirectional LSTM. Which provides context information from the past and future in order to increase the memory capabilities of LSTMs. [START_REF] Rao | Lstm with sentence representations for document-level sentiment classification[END_REF]. The Convolutional Neural Network (CNN) operates on n-grams and employs convolutional layers to identify features in the input data using various filter sizes. The layers are arranged in multiple feature mappings, allowing the CNN to learn and understand the details of the input data [START_REF] Saad Albawi | Understanding of a convolutional neural network[END_REF]. The initial layers extract high-level features with lower abstraction, while deeper layers extract low-level features with higher abstraction [START_REF] Saad Albawi | Understanding of a convolutional neural network[END_REF][START_REF] Ghosh | Fundamental concepts of convolutional neural network[END_REF]. [START_REF] Kp | Detecting misinformation in online social networks using cognitive psychology[END_REF] The experiments and results

The experiments

we work the execution of experiments, used three methods were defined in combination with the datasets. firstly,was established a baseline, training and validating the three Machine learning models listed in subsection 4.1, using the dataset in Arbic. For Machine language models, the texts were represented using tf-idf techniques; this step was carried out to have a reference point for comparison purposes with the Deep learning approaches and varying the datasets used for training and validation. secondly, uses the dataset in arbic both to train and validate three vanilla Deep learning models based on LSTM,BILSTM and CNN layers, Concerning the experiments, we tried with different values for the number of epochs, and also applied the early stopping strategy considering different values of the hyperparameters tolerance and patience. For its part,we conducted some experiments where samples from the translated dataset were progressively mixed with all the dataset in arbic during the training phase; then, the remaining portion of the tested dataset was used for validation, i.e.,some a learning curve.

Hyper-parameters

The importance of adjusting the model cannot be overstated in predictive modeling. To ensure optimal results, we carried out experiments and fine-tuned various hyperparameters for the selected machine learning approaches. This was done to evaluate the performance of the classifiers using common evaluation metrics.

The batch size used in this experiment was 50-60, which represents the number of windows of data passed at once. Dropout, a regularization technique in neural networks, was applied with a value of 0.5 to reduce interdependent learning among neurons. The model was trained for 50 epochs, which refers to the number of iterations of forward and backward propagation. The Dense class was used to specify the number of neurons or nodes in the layer and the activation function, with 1 neuron being selected. The loss argument was used to specify the loss function used to evaluate a set of weights, and the accuracy metric was used to evaluate the model's performance.

Results and discussion

We evaluated the performance of our model on test datasets after fine-tuning the hyperparameters. The classification accuracy was used as the evaluation metric to accurately determine the proximity of the predicted results to the actual values. The tables in the study present the predicted accuracy of the models discussed in sections 4. We presented the results of three machine learning classifiers and compared them to previous results. These findings are displayed in tables 7 and 8. The training was conducted using 70% of the data and the testing was done on the remaining 30% of the corpus. Additionally, two types of tests were performed. The first test was conducted on each individual rumor topic and the second test was performed on a mixture of all rumor topics. We observed that the performance results in 7 vary depending on the ru- mor topic and the classifier used. The best accuracy and precision for the Fifi Abdo topic were achieved by the SVM classifier, while the best recall was produced by the Decision Tree classifier. For the rumors related to President Bouteflika, the best results were produced by the D.Tree. In the case of the third rumor topic, the best accuracy and precision were achieved by the SVM classifier, and the best recall was produced by the D.Tree. When all the rumor topics were combined, the best results in terms of accuracy and precision were obtained by the SVM classifier, while the best recall was produced by the Decision Tree. Overall, the SVM classifier performed the best among the three classifiers, while the D.Tree. and Multinomial Naive Bayes (MNB) were not able to outperform the other classifiers in this dataset. We evaluated the updated datasets in table 10. The results showed slight variations in performance. The SVM classifier achieved the best accuracy and recall for the topic of Fifi Abdo, while the best precision was obtained by the Decision Tree. For rumors about President Bouteflika, the SVM classifier produced the best results. For the third rumor topic, the MNB classifier produced the best accuracy, precision, and recall. When all rumor topics were combined, the SVM classifier had the best accuracy and precision and the Decision Tree had the best recall. Overall, the SVM classifier performed the best in this dataset, while the other classifiers, despite their effectiveness in other classification tasks, were not able to outperform it for any of the topics.

In the second set of experiments, we utilized deep learning classifiers, such as LSTM, BILSTM, and CNN, as described in section 4. The models were trained on 70% of the Algerian corpus, and validated using the rest of the dataset. Due to the large size requirement of deep learning models, the results were shown in tables 9, and 10 which compared the predicted accuracy to the previous results. The experiments were run twice, once on individual rumor topics and once on a mixture of all the rumors. The results displayed in table 9 indicate that the highest accuracy for rumors related to Fifi Abdo was achieved by the BiLSTM and CNN models. The CNN model had the highest accuracy for the second rumor topic. Results for the third topic revealed that all classifiers had equal accuracy. In terms of accuracy, the best results were obtained by the BiLSTM and CNN models when all the rumors were combined. Overall, the BiLSTM classifier performed the best followed by CNN, while the LSTM classifier failed to surpass the other models. The experimental results in table 10 showed that the best accuracy for the rumors about Fifi Abdo were obtained by the BLSTM and CNN classifiers. The best results for the rumors about President Bouteflika were produced by the BiLSTM. The best accuracy for the third rumor topic was achieved by the BiLSTM as well. When all the rumors were combined, the best accuracy was still produced by the BiLSTM. Thus, overall, the BiLSTM classifier was the best performer, while the LSTM did not outperform the other classifiers.

To further evaluate the performance of these models, we implemented a learning curve by incorporating the Arabic dataset. The models evaluated were the LSTM, BIL-STM, and CNN layers, along with trainable embeddings. The results of the learning curve indicated an improvement in performance with different ratios. The figures in 3,4, and 5 show the cross-validation results of these models. We noted that the learning curve in Figure [START_REF] Kp | Detecting misinformation in online social networks using cognitive psychology[END_REF] displayed an unusual pattern, with both the training and validation losses remaining constant. This suggests that the deep learning algorithm may have trouble correctly classifying smaller datasets. To address this, it would be necessary to increase the size of the dataset and perform additional training on the model.

Conclusion and future work

In this article, we present a study on automatic Arabic fake news detection. As the use of social media continues to rise, the problem of fake news is becoming more prevalent. To address this issue, researchers have been working on finding solutions to protect society from fake news. In this study, we reviewed various studies related to fake news detection and presented a taxonomy of methods based on classical ML and advanced DL strategies. The study includes three different datasets and evaluates the performance of ML classifiers like SVM, Naive Base, and Decision Tree based on accuracy, precision, and recall. The results showed that SVM had the highest accuracy of 95% in fake news detection. In the second part of the study, we conducted experiments on fake news detection methods using deep learning models, including LSTM, BILSTM, and CNN. By fine-tuning important hyper-parameters and using techniques like word2vec and Tf-idf, the models were optimized to achieve higher accuracy. The results showed that CNN had the highest accuracy of 95%. Furthermore, the use of the Algerian corpus for embedding improved the Arabic fake news detection in Deep learning models compared to linear models. Future studies aim to build a larger Arabic fake news dataset and conduct further experiments using deep learning models.

Fig. 3 .Fig. 4 .

 34 Fig. 3. Model train and test vs Validation loss (LSTM)

Fig. 5 .

 5 Fig. 5. Model train and test vs Validation loss (CNN)

Table 1 .

 1 The collected stories related to Fifi abdo, Bouteflika and Adel Imam

	Topics	|c|
	Fifi Abdo	2,145
	Bouteflika	964
	Adel Imam	326

Table 2 .

 2 The collected stories related to Fifi abdo, Bouteflika and Adel Imam (update datasets)

	Topics	|c|
	Fifi Abdo	2,725
	Bouteflika	1,511
	Adel Imam	708
	Fig. 1. Distribution of the vocabulary after combination datasets
	Fifi Abdo
	55%
		President Bouteflika
	Adel Imam	31%
	14%

Table 3 .

 3 Statistics corresponding to the rumors datasets

	Topic	|C| |W |
	Fifi Abdo	187 1605
	President Bouteflika	106 3507
	Adel Imam	50 508

Table 4 .

 4 Statistics corresponding to the rumors update datasets

	Topic	|C| |W |
	Fifi Abdo	360 2767
	President Bouteflika	347 6228
	Adel Imam	187 1744

Table 5 .

 5 Statistics corresponding to the Non-rumors datasets

	Topic	|C| |W |
	Fifi Abdo	1958 22708
	President Bouteflika	858 11917
	Adel Imam	276 3085

Table 6 .

 6 Statistics corresponding to the Non-rumors update datasets

	Topic	|C| |W |
	Fifi Abdo	2300 13045
	President Bouteflika	1180 22209
	Adel Imam	521 5816

Table 7 .

 7 Performance on detecting rumors of datasets

	Topic	SVM	D. Tree	MNB
		Acc Prec Rec Acc Prec Rec Acc Prec Rec
	Fifi Abdo 95.35 87.72 82.16 93.59 79.94 82.8 92.63 78.01 73.42
	Bouteflika 94.2 92.69 78.18 95.56 94.09 83.9 93.86 90.7 77.99
	Adel Imam 93.68 85.2 78.82 89.47 73.15 80.88 90.53 74.87 72.65
	Combi 95.35 92.77 83.12 93.47 84.07 83.56 92.38 82.76 76.94

Table 8 .

 8 Performance on detecting rumors of update datasets

	Topic	SVM	D. Tree	MNB
		Acc Prec Rec Acc Prec Rec Acc Prec Rec
	Fifi Abdo 92.73 85.96 80.63 89.35 76.80 78.28 91.60 84.07 76.39
	Bouteflika 91.03 85.92 88.73 83.15 75.50 80.37 88.62 84.27 80.52
	Adel Imam 85.92 81.62 77.98 80.28 73.73 77.63 89.20 86.03 83.50
	Combi 92.09 89.47 84.13 90.87 85.43 85.24 90.52 88.65 79.27

Table 9 .

 9 Performance on detecting rumors of datasets

	Topic	LSTM BiLSTM CNN
		Accuracy Accuracy Accuracy
	Fifi Abdo	0.92	0.95	0.95
	Bouteflika	0.90	0.91	0.94
	Adel Imam 0.85	0.85	0.85
	Combi	0.90	0.93	0.93

Table 10 .

 10 Performance on detecting rumors of update datasets

	Topic	LSTM BiLSTM CNN
		Accuracy Accuracy Accuracy
	Fifi Abdo	0.86	0.92	0.92
	Bouteflika	0.79	0.88	0.85
	Adel Imam 0.79	0.82	0.80
	Combi	0.80	0.90	0.85