Mohammed Tadlaoui
email: mohammed.tadlaoui@insa-lyon.fr

Karim Sehaba
email: karim.sehaba@liris.cnrs.fr

Sébastien George
email: sebastien.george@univ-lemans.fr

Azeddine Chikh
email: az_chikh@mail.univ-tlemcen.dz

Karim Bouamrane
email: kbouamranedz@yahoo.fr

Social recommender approach for technology-enhanced learning

Keywords: personalised e-learning, educational resources, recommender systems, social networks

come

this paper, we present the results of an experiment conducted during six months in a real educational context. The goal of this experiment is to measure the relevance, quality and utility of the recommended resources. We also conduct an offline analysis by using a dataset in order to compare our approach with four baseline algorithms.

Introduction

The features of interaction and mutual aid offered by online social networks have allowed them to be increasingly present in training programs. These social networks could be a part of a learning management system (LMS) or a standalone component [START_REF] Popescu | Providing collaborative learning support with social media in an integrated environment[END_REF].

As explained in Fazeli et al. (2014b), social learning platforms mix traditional LMS with social networks to provide easy content creation, access, sharing, etc. Beside forums and chat features present in LMS, social learning platforms allow for users to make more connections and improve their peer networks.

According to [START_REF] Guy | Social recommender systems[END_REF], the multitude of resources, relationships and interactions present in social media can lead users to suffer from information overload, making them unable to assimilate the information presented by the social media. To reduce this overload, it would be useful to provide users only with resources with possible interest to them. To achieve this, we propose to recommend to users relevant resources based on existing social relationships. Thus, the goal of our work is to propose an approach for personalising the recommendation of learning resources based on connections between individuals in social networks.

Our work is motivated by two principles of social sciences, namely co-citation regularity [START_REF] Bhagat | Node classification in social networks[END_REF] and social influence [START_REF] Sun | A survey of models and algorithms for social influence analysis[END_REF]. The first principle states that similar people tend to refer or connect to the same resources. This principle is used in classical recommender systems. These types of systems are primarily based on the predictions of preferences of a given user to her/his similar users' ratings (e.g., a scale from one to five stars). The second principle states that people who are socially connected are likely to share the same or similar interests. So the users of a system can be easily influenced by their friends and thus be interested in their activities. This principle is used by social recommender systems.

According to [START_REF] Ma | Recommender systems with social regularization[END_REF], recommendation techniques have been widely studied in research areas of information retrieval, machine learning and data mining.

The two most commonly used methods in recommender systems are: collaborative filtering [START_REF] Goldberg | Using collaborative filtering to weave an information tapestry[END_REF] and content-based filtering [START_REF] Pazzani | Content-based recommendation systems[END_REF]. The first method recommends resources from the similarity between users' preferences. This method is used by Amazon [START_REF] Linden | Amazon.com recommendations: item-to-item collaborative filtering[END_REF] and LinkedIn [START_REF] Wu | The browsemaps: collaborative filtering at LinkedIn[END_REF]. It is also used in technology-enhanced learning (TEL) by [START_REF] Ghauth | The effect of incorporating good learners' ratings in e-learning content-based recommender system[END_REF] and [START_REF] Shi | Social personalized adaptive e-learning environment: Topolor-implementation and evaluation[END_REF]. The second method is based on the recommendation of resources that are similar to resources for which the user has shown interest in the past. This method is used by Pandora Radio [START_REF] Lops | Content-based recommender systems: state of the art and trends[END_REF]. It is used in TEL by [START_REF] Karampiperis | Collaborative filtering recommendation of educational content in social environments utilizing sentiment analysis techniques[END_REF] and [START_REF] Salehi | Application of implicit and explicit attribute based collaborative filtering and BIDE for learning resource recommendation[END_REF].

The majority of recommender systems based on collaborative filtering consider a single criterion value (rating of an item by a user) to recommend a list of items to the user. However, other systems incorporate multiple criteria that can increase the accuracy of recommendations. [START_REF] Adomavicius | Multi-criteria recommender systems[END_REF] classify the recommendation techniques used in multi-criteria recommender systems into two categories: heuristic based and model based techniques. In the first technique, the similarities between users are calculated by aggregating similarities from each criterion or using multidimensional distance metrics. The second technique is based on a predictive model, employing statistical or machine learning methods, to predict the rating of an item. A part of the similarity computation used in the proposed system is inspired from the heuristic based technique.

In our approach, we use the collaborative filtering method as we mainly focus on the social relationships between users. In this area, recommender systems can be based on relationships of trust between users, relying on the assumption that users have similar tastes with other users they trust. Trust-aware recommender systems, as in [START_REF] Bedi | Trust based recommender system for Semantic Web[END_REF] and [START_REF] Ma | Learning to recommend with social trust ensemble[END_REF], use the inferred implicit or observed explicit trust information.

According to [START_REF] Yang | A survey of collaborative filtering based social recommender systems[END_REF], recommender systems that use social information improve the accuracy of the traditional recommender systems by taking social interests and social trusts between users in a social network as additional inputs. The use of social network information in recommender systems has been studied in many research works. [START_REF] Vassileva | Toward social learning environments[END_REF] presents social recommender systems as a solution that can help learners to find appropriate resources to support their learning process.

The idea of 'social recommendation' improves recommender systems by incorporating a social friends network. According to [START_REF] Ma | Recommender systems with social regularization[END_REF], social friendship is related to cooperative and mutual relationships that surround the user such as classmates, colleagues, relatives, etc. [START_REF] Bellogín | An empirical comparison of social, collaborative filtering and hybrid recommenders[END_REF] distinguish four types of social recommender systems:

1 Friend-based recommender: systems of this type use collaborative filtering by considering only the user's friends. According to [START_REF] Yang | A survey of collaborative filtering based social recommender systems[END_REF], most existing social recommender systems are based on collaborative filtering.

2 Social popularity recommender: this type recommends the most popular resources among the friends of a user.

3 Personal social recommender: in systems of this type, distance between users in the social graph is used to calculate the recommendation.

4 Hybrid recommender: several methods are used in this type of system.

Recent years have witnessed the emergence of the research area of recommendation systems for TEL. Recommender systems can have multiple benefits on learning. [START_REF] Drachsler | Recommender Systems for Learning[END_REF] explains that these systems help learners to more effectively identify learning content or peer learners among a potentially overwhelming set of choices.

According to [START_REF] Erdt | Evaluating recommender systems for technology enhanced learning: a quantitative survey[END_REF], TEL recommender systems support learners during their learning process in order to achieve their learning goals. Same researchers explain that recommender systems can have benefits on learning, such as the improvement of learning performance and motivation. As explained in [START_REF] Klašnja-Milićević | Recommender systems in e-learning environments[END_REF], recommender systems can help learners to overcome the information overload problem. These systems can recommend learning resources according to learners' habits and level of knowledge. [START_REF] Manouselis | Recommender Systems for Learning[END_REF] explain that TEL recommender systems can support the following tasks: find good items, find peer learners, recommend sequences of items, predict learning performance and suggest learning activity.

Users are interested in obtaining appropriate recommendations for their needs. They can be interested in having the most popular resources among their similar users. They can also be interested in resources that have been visited recently by their network of friends to encourage them to collaborate with their friends on these resources. Users can also be interested in resources that are useful in their areas of interest. Unlike existing social recommender systems for TEL, which deal only with the first type of recommendation, the approach presented in this paper addresses all three types.

We hypothesise that, in addition to users' ratings, the use of information present in the social network, in the users' profiles, in their exercise results and in the traces they leave while using the system, can help to make the recommendations richer and adapted to the needs of the learner. Existing social recommender systems for TEL principally use users' ratings to recommend appropriate resources for these users' needs and do not benefit from this information.

Thus, the goal of our research is to propose an approach that uses all this information in order to improve the accuracy of the recommendations proposed to the learner. In addition to the problem of accuracy of recommender systems for social learning problems, we try to find solutions for the sparsity problem.

User preferences are represented in a user-item matrix. This matrix of ratings is sparse when the number of rated items is very small compared to the number of all items in the system. Collaborative filtering methods use similarity between users. The problem of sparsity restricts the calculation of similarity, thus reducing recommendation accuracy [START_REF] Schafer | Collaborative filtering recommender systems[END_REF]. As explained by [START_REF] Verbert | Dataset-driven research for improving recommender systems for learning[END_REF], datasets present in educational platforms suffer from this problem.

The research questions we address in this paper are:

RQ1: How to generate more relevant recommendations for learners in TEL platforms, based on social relations and user evaluations?

RQ2: How to improve recommendations for learners on TEL platforms that contain sparse ratings of educational resources?

This paper extends our work presented in [START_REF] Tadlaoui | Recommendation of learning resources based on social relations[END_REF]. It contains a detailed study of related works and a complete evaluation of the system we propose. This paper is organised as follows. In Section 2, we provide an overview of several recommender systems in the context of social learning platforms. Section 3 describes the approach that we propose and the Section 4 describes its formalisation. The proposal evaluation is presented in Section 5, followed by the conclusion and perspectives.

Related works

Over the last decade, dozens of recommender systems in TEL were developed, such as Altered Vista [START_REF] Recker | Supporting 'word-of-mouth' social networks through collaborative information filtering[END_REF], RACOFI [START_REF] Anderson | RACOFI: a rule-applying collaborative filtering system[END_REF], LSRS [START_REF] Yueh-Min | A Markov-based recommendation model for exploring the transfer of learning on the web[END_REF] and ReMashed (Drachsler et al., 2009b). [START_REF] Drachsler | Panorama of recommender systems to support learning[END_REF] studied 82 recommender systems in TEL that were designed over the last 15 years. This study presents nine systems that recommend educational resources in a social environment. We have selected these nine systems and added a tenth one.

In the following sections, we study these ten recommender systems used on social learning platforms, before concluding with a synthesis.

The ISIS recommender system

ISIS (Drachsler et al., 2009a) is a system that helps the learner to find her/his learning path by recommending the next best resource for the learner. A sequence of multiple resources might be more valuable for the learner than a single resource.

ISIS combines an ontology based recommendation technique and a stereotype filtering technique. The recommender decides which of the two techniques will be used for the current learner situation. The first uses learner personal information and compares it with domain knowledge to recommend the most appropriate resource. The second technique is based on a selection of the most popular resources in a specific learner group using collaborative filtering. This system uses similarity between users and does not benefit from social relationships between them.

The 3A recommender system

The 3A recommender system [START_REF] El Helou | The 3A personalized, contextual and relation-based recommender system[END_REF] ranks 3A entities (actors, assets and activities) according to their importance to a specific actor and her/his context.

The 3A system uses an interaction model to identify and exploit significant user interactions, established relationships and evaluation metadata such as ratings, bookmarks, tags and reviews provided by users. This system uses the 3A ranking algorithm to rank actors, assets and activities based on their popularity. In this system, actors can be users or agents, assets represent resources and activity spaces represent mediums to conduct individual or collaborative activities. This system can suffer from sparsity if there are few available users' actions (ratings, tags, reviews, etc.) in the system. [START_REF] Brusilovsky | Social navigation for educational digital libraries[END_REF] propose a social navigation system, called Ensemble, which guides the user to the most useful information by using various users' traces. This guidance is based on adaptive link annotation and link generation. Link annotation highlights resources that are popular in the user's group, while link generation can recommend resources that were visited by the user's group in a similar context.

The ensemble recommender system

Ensemble uses the two existing forms of social navigation, namely collaborative filtering and history-enriched information spaces to guide users. Ensemble tracks low level user actions (browsing, rating, commenting and tagging) and higher level structural actions (fragment extraction and composition). This approach proposes to all a group the same recommendations. It is not personalised to each user.

The Topolor system

Topolor [START_REF] Shi | Social personalized adaptive e-learning environment: Topolor-implementation and evaluation[END_REF] is a social personalised adaptive e-learning system, which aims at improvement of social interactions in the learning process. This system offers personalised recommendations and some social features, such as sharing a learning status, engaging in a simple question/answer exchange and sharing notes.

Topolor can recommend learning content, learning paths, learning experts, learning peers, questions and learning topics. This system uses both collaborative filtering and content based recommendation. Even though Topolor is a social platform, it does not use social relationships for recommendation. It uses classical methods such as collaborative filtering and content based recommendation.

2.5

The teaching style-based recommender system [START_REF] Limongelli | A teaching-style based social network for didactic building and sharing[END_REF] propose to group teachers into four clusters based on their teaching styles. This classification uses the K-means clustering algorithm taking into account the teaching style of each teacher. Each group of users will have the same teaching attitudes. The authors consider five teaching styles: expert, personal model, formal authority, delegator and facilitator. Each one of these teaching styles is evaluated in a range from 1 to 7.

The system proposes to all the teachers of a particular cluster the same recommendations of learning materials. It is not personalised to each user and only focus on teachers not on learners.

2.6

The social learning recommender system based on learner sentiment [START_REF] Karampiperis | Collaborative filtering recommendation of educational content in social environments utilizing sentiment analysis techniques[END_REF] use sentiment analysis techniques on user generated comments when the user rating is missing. It is used to improve recommendations of educational resources in collaborative filtering recommender systems. This approach generates a rating for an educational resource from users' activities (rating and comment) on this resource. It also generates ratings for resources that have no activities.

When a user has not provided an explicit rating for a given resource and does not have any activity in common with other users, the system computes a community driven implicit rating using the average of the ratings provided by users belonging to the same user group. This approach proposes the same recommendations to an entire group in the case of a community driven implicit rating. It is not personalised to each user.

The social trust-aware recommender system for teachers

In Fazeli et al. (2014a), the authors propose a collaborative filtering recommender system that supports teachers in finding learning resources to match their needs and interests. This system is based on the idea that users prefer to receive recommendations from people they trust. The approach of this system solves the sparsity problem by assuming that trust can be transitive. This transitivity can help build a relationship between two users who have no common set of rated items but have friends in common. In this case, a new trust relationship between two unconnected users can be inferred.

This approach is based on users' ratings and it does not use other users' activities on the system or users' profile information. The approach proposed by this system is still an ongoing work and has not yet been evaluated.

2.8

The social learning recommender system based on graph-walking methods Fazeli et al. (2014b) developed a social recommender system that combines a traditional LMS with commercial social networks like Facebook (http://www.facebook.com). This system uses a graph-walking method for recommending learning resources. It improves the accuracy of recommendations even when rating data are sparse.

The algorithm used to create the graph takes into account the social index (S-index) for each user. S-index is used to help find similar users. The approach proposed by this system collects recommendations for a given user by walking through her/his neighbours. It infers users' relationships and similarities. It uses inferred similarity and depends principally on ratings. It does not maximise the use of information present in the social network.

2.9 The social learning recommender system based on learner affective state [START_REF] Santos | Exploring arduino for building educational context-aware recommender systems that deliver affective recommendations in social ubiquitous networking environments[END_REF] present an approach that detects learner affective state and uses it when the system is making recommendations in social ubiquitous networking environments. The authors of this approach propose to detect emotions by using an Arduino (https://www.arduino.cc) computing platform and sensors.

This approach focuses on the integration of contextual information to enrich the recommendation process. It is jointly used with non-context aware recommendation methods. This approach does not propose a new method for recommending resources. It can be integrated to another approach after the recommendation generation.

2.10

The semantic educational recommender system [START_REF] Santos | User centred design and educational data mining support during the recommendations elicitation process in social online learning environments[END_REF] propose 32 recommendation rules in social online learning environments. The idea is to consider several types of actions, such as recommending to a learner to join a group, give feedback to some contributions, chat with the tutor and work harder in the current activity. These types of actions are given according to some conditions, such as the learner has not yet submitted an activity, the learner has passed the assessment and the learner has not yet read the prerequisite content.

This approach also proposes a semantic recommender system that delivers the appropriate recommendations to learners according to these rules. This approach takes from the social networks the use of social tools, such as chat and forums and recommends actions that can be performed in these tools. It does not recommend resources.

Synthesis

None of these systems, except 3A and the recommender system based on sentiment enhances the method to calculate the similarity or the trust value between users. It is still based on users' ratings and does not integrate users' profile information and users' activities on the system. Also, all the presented systems do not use the information present in the profiles of similar users. None of them processes the knowledge acquired by learners to better personalise the recommendations. Only the social trust-aware recommender, the recommender based on graph-walking methods' and the recommender based on learner sentiment address the problem of sparsity using, respectively, transitive trust and graph walking methods. ISIS, ensemble, the teaching style based recommender and the recommender based on learner sentiment propose to all a group's users the same recommendations. It is not personalised to each user.

All presented systems propose one type of resource recommendation. In their recommendations, it is not possible to make a distinction between most liked resources, useful resources and resources that are read by their network of friends. Sometimes users need to see resources that have been useful for their similar users, to see popular resources and to receive recommendations of resources that are being seen by similar users. None of the aforementioned systems addressed the utility of resources according to the learning domains of the students.

Table 1 presents the principle characteristics of the ten presented social educational recommender systems. For each of them, we mention items types recommended to learners, recommendation method, elements used to compute recommendations and whether the system deals with sparse ratings problem. The Na means that the system does not use ratings. Then this characteristic is not applicable for this system.

We can conclude that these systems 1 do not benefit from all the information related to a user and her/his friends to enhance recommendations 2 do not take into account the need to have multiple types of recommendations 3 most of them do not deal with sparsity 4 some of them do not personalise their recommendations proposing only the same resources to all users of a group.

Our approach presented in the next section addresses these four points.

A recommendation approach based on social influence and co-citation regularity

The proposed approach is based on the idea that according to the principle of: social influence: two friends can have the same interests co-citation regularity: if they have some similarities in common, they will be more likely to have the same interests.

The proposed approach relies on four types of data:

1 profile data that describe users as preferences, knowledge or experience 2 social data that describe links between users and groups and links between users themselves such as friendship and group membership 3 resource data that describe the actions performed by users on educational resources such as visits or ratings 4 exercise data that describe the results of exercises performed by learners such as the results of a quiz they did.

The learning environment retrieves the results of exercises conducted by users and updates their knowledge levels stored in their profiles. These data will all be used to recommend relevant educational resources.

Figure 1 Overall principle of the proposed approach (see online version for colours)

The overall principle of the approach is illustrated in Figure 1. All learning environment users have profiles that describe them and they can be interconnected by friendship links.

In this approach, we define friendship as an explicitly declared link by a user of the system. In the learning environment, it can be created by users through requests to be added to the friends list such as on Facebook or Linkedin. Users can perform three types of actions on the educational resources, namely visit (view), evaluate (rate) quality and evaluate usefulness. Utility is given according to a domain of interest of a user, while quality is given by a user regardless of his/her area of interest.

The recommender system uses all the above information to generate recommendations that are personalised for each user of the learning environment. These recommendations are divided into three types, namely: recently viewed resources, popular resources and useful resources.

The recommendation process consists of the following steps:

1 The current user selects one of the three types of recommendation.

5

The system shows the current user a list of educational resources, ordered by score according to the selected type of recommendation.

The approach uses a formal model to support this recommendation process. It is presented below.

Formalisation of social similarity and recommendation type

Table 2 defines the basic concepts used in the formal model of the approach. These concepts are related to the profile of the user, her/his social relationships and the user actions on resources (visits and ratings). The set of resources visited by user u.

IEu

The set of resources evaluated according to their qualities by user u.

IUu

The set of resources evaluated in terms of utility by user u. D

The set of all teaching domains (areas of interest) represented in the learning environment. For example, if the learning environment is used in a research laboratory, domains can then be topics of research. If it is a university, domains can be specialties or course modules. Vis(u, i) Used to know the resources that are visited by a given user. This function is equal to 1 if user u viewed the resource i and 0 otherwise.

t(u, i)

Represents the number of days since the date of the last visit of the resource i by user u.

Eval(u, i) Used to know the evaluation of the quality of a resource by a user. For example, user u can evaluate the resource i from 1 to 5. If the resource has not been evaluated, then this function is equal to 0.

Eval(u,.) Represents the average quality rating of user u for all the resources that she/he has evaluated.

Util (u, i, d) In addition to the evaluation of the quality of a resource, a user can evaluate its utility according to a specific domain. It represents the evaluation of the usefulness of the resource i in the context of the work d (domain) of user u.

Util(u,.) Represents the average utility evaluation of user u of all the resources that she/he has evaluated. Similarity in terms of visits, explained in equation ( 4), is based on Jaccard similarity [START_REF] Yu | It takes variety to make a world: diversification in recommender systems[END_REF].

Seval

( , )

u v u v Card IV IV VisSim u v Card IV IV ( 4
)
This formula is not applied when both users have not yet previewed resources because the union is null. In this case, VisSim(u, v) is equal to zero. [START_REF] Sun | A survey of models and algorithms for social influence analysis[END_REF] explain that the more friends two users have in common, the stronger the link between them. Our approach calculates the strength of the link between two users u and v [equation ( 5)] using the number of common friends and their total number of friends.

[ ] [ ] 2 ( , ) [ ] [ ] Card B u B v LinkS u v Card B u B v ( 5 )
The similarity between two users' profiles considers the similarities between users in terms of preferences, knowledge, experience, etc. It is related to the number of shared characteristics between the two users.

The similarity between profiles of users u and v is detailed by the following formula:

[ ] [ ] ( , ) ( ) Card C u C v ProfilSim u v Card C ( 6 )
As explained above, a part of the social similarity computation used in our approach is inspired from the heuristic based technique of multi-criteria recommender systems. This technique has multiple ratings and an overall rating. The proposed approach does not use the overall rating and uses two kinds of ratings: quality and usefulness. The latter is related to the user domains and each user has her/his associated set of domains.

The heuristic technique aggregates similarity between different ratings. To calculate similarity in our approach, we also use an aggregated similarity between quality and usefulness ratings and we add other elements (users' visits and some social information).

Recommendation types

Recommendation of recently viewed resources

This type of recommendation provides the user with resources recently viewed by users, which are similar to her/him. The resource recommendation score of this type is calculated according to the resources viewed by friends of the current user, to the time elapsed since last viewing resources and to social similarities with friends of the user. We think that this type of recommendation encourages users to follow the same resources that their friends viewed recently. This will motivate users to help each other and collaborate on their courses. Equation ( 7) determines the recommendation score of resource i for user u. . ( , ). ( , )

t v i vis v B u S u i e Vis v i SocialSim u v ( 7 )
We have used as a decay factor. The score of a resource increases when a user has viewed it recently. We were inspired by the work of [START_REF] Guy | Personalized recommendation of social software items based on social relations[END_REF][START_REF] Guy | Social media recommendation based on people and tags[END_REF] to take time into account in this formula.

Recommendation of popular resources

The system can also recommend to a user a list containing highly rated resources by her/his similar friends. The resource recommendation score is determined by the values of resource evaluations made by the user's friends and the social similarities that she/he has with them. The recommendation score of resource i for user u is determined by the following formula:

[ ] ( , ) ( , ). ( , ) i eval v B u S u i k Eval v i SocialSim u v ( 8 )
We have used the weighted sum approach [START_REF] Adomavicius | Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions[END_REF] to define the S eval formula.

B i [u] is a subset of B[u]
(friends of user u). It represents friends of user u who have evaluated item i. k represents the normalising factor and is usually given as

[ ] 1 (,) . i v B u k SocialSim u v 4.2

.3 Recommendation of useful resources

In addition to the two previous types of recommendations, the user can also have learning resources matching her/his domains of interests. The formula for calculating the resource recommendation score is defined by social similarities and resource utility according to learning domains. The following formula defines the recommendation score of resource i for user u:

[ ] [ ] ( , , ) ( , ) ( , ). [ ] d E u vi i util vi v B u Util v i d S u i k SocialSim u v Card E u ( 9 ) E vi [u] is a subset of E[u]
(domains of interest of user u). It represents domains of interest of user u, which have been used to evaluate resource i by user v.

Sparsity management

Most of the collaborative filtering algorithms use similarity measure based on users' ratings to recommend resources. As explained in [START_REF] Verbert | Dataset-driven research for improving recommender systems for learning[END_REF], unlike recommender systems for ecommerce, those for education can suffer from sparsity of ratings, since this type of application involves fewer users and fewer transactions. Our approach can address this problem since it uses five components in the similarity measure and only two of them are related to user ratings. We can calculate similarity and provide relevant recommendations even if the learning environment contains sparse ratings.

Principle of the profile update

If users' profiles are partially filled, the recommendations provided by the system may be irrelevant. Since the system is mainly based on profile information, it is essential to fill incomplete users' profiles.

User profile can be filled manually by a user or automatically by the system according to user behaviour. In addition, we propose a third method based on the principle of social influence [START_REF] Sun | A survey of models and algorithms for social influence analysis[END_REF]. This consists in enriching user profile with characteristics of influential friends' profiles. This is possible using one or both of the following methods:

Update the user's profile directly with the values of her/his most common friends' characteristics.

Recommend to the user, her/his most common friends' characteristics. After a user chooses one characteristic value, the system will update the user profile with this value.

Evaluation

In order to address the two research questions described on Section 1, we conducted an evaluation to measure the relevance, the quality and the utility of recommendations generated according to our approach. According to [START_REF] Herlocker | Evaluating collaborative filtering recommender systems[END_REF], collaborative filtering recommenders can be evaluated by using an offline analysis with a natural or synthesised dataset or a live user experiment with real users.

The first one concerns the analysis of the predictive accuracy of recommendations. The recommendation algorithm to be evaluated is used to predict some withheld values from a dataset. The results of predictions are analysed using one or more metrics. The second one concerns the direct evaluation of user reaction to a recommender system. It can evaluate user performance, satisfaction, participation and other measures.

Consequently, in an initial evaluation, we attempted an offline analysis using a natural dataset extracted from existing educational recommender systems. For this, we studied several datasets, such as Mendeley [START_REF] Jack | Mendeleys reply to the datatel challenge[END_REF], MACE [START_REF] Wolpers | dataTEL challenge: CAM for MACE[END_REF] and APOSDLE [START_REF] Beham | Aposdle-ds a dataset from the aposdle work integrated learning system[END_REF]. We found that it is impossible to use these datasets to evaluate our model. This is because the data they provide do not contain all the data necessary for our approach, such as social relationships between users and resource ratings in terms of utility.

To address this problem, we established a three-step process 1 Study the feasibility of the proposed approach by using an offline analysis on a synthesised dataset.

2 Develop a platform, named Icraa, which implements our approach and experiment it with real users.

3 Compare the accuracy of the approach with four baseline algorithms using an offline study on data gathered from Icraa.

This process is illustrated in Figure 2.

For the first evaluation step, we carried out a simulation on a reduced dataset that we created, consisting of ten users and nine resources. This evaluation helped us test our formulas and algorithms in order to evaluate their effectiveness and thus to refine them. We developed a prototype in Java that calculates the similarities between users and calculates and displays the three lists of recommendations that we propose in the approach. In this section, we first present the environment supporting our approach. We shall then describe how it can be used to objectively validate approach models, before presenting the questionnaire that subjectively evaluates these models. Finally, we will present the offline analysis of our approach.

Icraa: a social learning environment

In order to validate the approach with a live user experiment, we developed a platform that offers learning resources, some social features and implements our formal models to recommend educational resources. This platform is named Icraa (Icraa is a social learning and authoring environment). Icraa functionalities are:

Resource upload: teachers can use this functionality to publish resources related to their courses on the platform. They can also describe them with metadata.

Resource access: the two types of users (learners and teachers) can access all resources available on the platform. A menu allows users to browse classes and courses. When a user clicks on a course link, a list of resources is displayed. The left side of Figure 3 illustrates a resource page. It contains a link to download the resource as well as some metadata of the resource, such as its name, author, attached course, duration, notes, etc.

Resource rating: as illustrated in Figure 4, this feature allows users to rate a resource in terms of quality and utility according to user's domains.

Resource recommendation: the right side of Figure 3 shows the recommendation widget. It shows users the three types of recommendation of the approach, namely recently viewed resources, popular resources and useful resources. This widget displays a list of the three top recommended resources for each recommendation type.

Social features: the learning environment that we have developed offers multiple social functionalities, such as we can find on online social networks like Facebook or Google+ (https://www.plus.google.com/). Icraa allows to post messages, share documents, join groups, add friends, etc.

Live user experiment

Experiment

The goal of this experiment was to investigate the user perceived relevance of recommendations given by Icraa. The latter was used as a LMS to test our approach in a real learning situation. The experiment integrated 238 users (192 students and 46 teachers) from the computer science department of the university of (anonymised).

The experiment ran for 6 months, from November 2014 until May 2015.

In the computer science department, teachers were using e-mail, Moodle (https://www.moodle.org), Edmodo (https://www.edmodo.com), Google (https://apps.google.com/products/sites) sites and Facebook to share their learning resources. Once Icraa had been installed, 14 teachers used it, while the other 31 teachers continued to use their preferred tools. Eight class years of students (two bachelor and six master levels) were integrated on a continuous flow. 146 students connected more than once to Icraa during the 6 months of the experiment.

The teachers were asked to publish their course resources and to interact with students on the Icraa platform. The 14 active teachers published 175 learning resources on 22 learning units with an overall study load of 224 hours. All these resources were available for all Icraa users.

Students were asked to download resources, rate the quality and utility of these resources, consult the resources recommended by the learning environment if they wanted and interact with other users on the social platform. We explained clearly to the students that their participation in the experiment and the survey had no impact on their results (grades). Furthermore, they knew that the resource ratings they gave were not visible by their teachers.

The user profile contained seven attributes, namely user name, class year, study level, programming experience, preferred programming language, preferred learning style and preferred domains.

We prepared and sent a video to the students, explaining how to use the Icraa platform and the interest of making recommendations, filling their profile, rating resources and connecting with their friends. Another video was designed for teachers. Also, a user guide was sent to the two types of Icraa users.

Some evaluation questions required that the user had really used the learning environment and not just discovered it. Therefore, we decided to retain only active users. 40 of the most active students were asked to answer the questionnaire. These active students were chosen according to their number of connections to the platform, number of friends, number of quality and utility ratings and number of recommendation clicks.

During the six months of learning environment use, a trace mechanism kept tracks of user actions. Users declared 255 friend relationships. The 175 resources present in Icraa were visited 5,850 times. Some resources were visited multiple times by the same user. Users made 1,640 resource ratings (1,321 utility ratings and 319 quality ratings). 17.71% of the resources were visited by clicking on recommendations, while the other resources were accessed directly via the application menu. et al. (2006) explained that accuracy metrics are very limited in evaluating recommender systems and that user satisfaction does not always correlate with high recommender accuracy. The layout of the recommender interface is attractive and adequate.

Questionnaire

McNee

Perceived ease of use I became familiar with the recommender system very quickly.

I found it easy to make the system recommend different things to me.

It is easy for me to inform the system if I dislike/like the recommended item.

4 Attitudes Overall, I am satisfied with the recommender.

I am convinced of the resources recommended to me.

I am confident I will like the items recommended to me.

The recommender can be trusted.

Behavioural intentions I will use this recommender again.

I will tell my friends about this recommender.

I would visit the items recommended, given the opportunity.

To verify whether the recommender proposed by our approach is efficient and satisfactory from the user's point of view, we have chosen to use a questionnaire based on recommender systems' quality of user experience (ResQue) [START_REF] Pu | A user-centric evaluation framework for recommender systems[END_REF]. This tool aims to evaluate the user's perceived qualities of the recommender (usability, usefulness, interface, interaction qualities, etc.), the user's satisfaction with the system and the influence of these qualities on users' behavioural intentions (use the system again in the future, tell their friend about the system, etc.).

ResQue consists of 60 questions. Its authors propose also a simplified version of this questionnaire consisting of just 15 questions. To validate the proposed approach, we have chosen 19 questions and added a new one that complies with the hypothesis that we want to verify (Table 3). The added question (marked with * in Table 3) concerns the incitation to collaborate between users that can be made by the recommender. We have used a five-point Likert scale from 1 (strongly disagree) to 5 (strongly agree) to characterise users' answers.

This questionnaire was created using Google Forms and sent to the 40 active students selected previously. We received answers from 26 users in total, thus giving us a response rate of 65%. We used the selected 20 questions and these active learners to analyse the following 4 hypotheses:

1 the recommender provides quality resources 2 the recommender provides useful resources 3 the recommender is easy to use and the recommendations are well presented to the learners 4 the provided recommendations will satisfy the users.

The first two hypotheses are related to the approach, the third is related to the Icraa learning environment and the last one is related to both of them.

Questionnaire results

The results extracted from the questionnaire are presented below. Figure 5 shows the mean Likert scale rating for the 20 questions of the questionnaire. These mean values are between 2.92 and 4.38. The standard deviation values of the overall results, shown in Figure 6, are between 0.42 and 1.14. 19 values of a mean are greater than 3 (the neutral response) and just one value of a mean is less than 3, suggesting that users' attitudes are generally positive. We grouped the 20 mean Likert scale user ratings by their respective hypotheses in Figure 7. This figure shows that the mean of the four hypotheses is greater than 3 and thus that these hypotheses are verified. We adopted the Cronbach's alpha [START_REF] Cronbach | My current thoughts on coefficient alpha and successor procedures[END_REF] to measure the reliability of our questionnaire. A Cronbach's alpha between 0.9 and 1 indicates an excellent level of reliability. Its value for our questionnaire is 0.903, suggesting a high level of reliability of the questionnaire results.

1 2 1 1 i k Y i X σ k k σ ( 1 0 )
The formula for calculating a Cronbach's value is as per equation ( 10), where k is the number of items, 2 X σ is the variance of total score and 2 i Y σ is the variance of item i.

Discussion

We have used the results of a questionnaire to validate the proposed approach. Users of Icraa globally give positive evaluation. About 95% (19 out of 20 questions) of the questionnaire items have a result above average (average mean ≥ 3). Overall, the questionnaire results show that recommendations presented to users were perceived to be relevant. We can conclude that the relevance of the resources recommended by the proposed approach meets our initial expectations.

Only the mean value of question 8 is less than three. For this question, 38.46% declared that the recommender helped them collaborate with their friends, 15.38% are neutral about this question and 46.16% disagree with it. We notice that this functionality continues to be useful for a good part of system users. We think that adding an explanation to the recommendation in our learning environment can help users understand why the system proposed these resources to them and then incited them to collaborate with their friends. This functionality can give them a list of their friends who have visited the recommended resources.

Not all 238 users of the learning environment were active, but 98 connected more than five times. This can be accounted for by the following reasons: the teachers of one student class year published no resources the teachers of four class years published resources just for one course some students connected only few times and downloaded all their course resources in just one or two shots.

For the 46 subscribed teachers, only 30.43% of them used Icraa and continue to use it after the experiment phase. This small percentage can be accounted for by the fact that teachers are well appropriated with their systems cited in Section 5.2 and do not want to change them. It is also related to the poor user interface for publishing resources present in the first version of Icraa. Since then, we have improved the teachers' user interface and added multiple features. We will discuss the practices and the ownership of Icraa by teachers in a future work.

Offline study

Experiment

This offline study aims to verify whether the proposed recommendation approach performs better than other existing approaches. During this experiment we compared the accuracy of our approach with four baseline algorithms, namely user-based collaborative filtering, item-based collaborative filtering, singular value decomposition (SVD) and social popularity. The first algorithm calculates similarity between users in terms of their ratings and predicts the rating of an item. The second follows the same process as the previous algorithm but is based on similarity between items. SVD is a matrix factorisation technique used in recommender systems. It removes insignificant users or items to reduce the dimensionalities of the user-item matrix. The social popularity method makes recommendations based on general popularity of items on the user friends' network.

The four selected baseline algorithms recommend resources based on quality ratings. Thus, from the three types of recommendations proposed in our approach, we use in this experiment just 'recommended popular resources' algorithm since it is also based on quality ratings.

We developed a Java application that calculates the accuracy of the top N recommendations for each algorithm. This application uses the dataset retrieved from Icraa. It was collected for almost 2 years, from November 2014 to August 2016. It is an upgraded version of the dataset used in the second evaluation step. The dataset contains 372 users which created 517 friend relationships. The 229 resources of Icraa were visited 10295 times. Users generated 344 quality ratings and 1456 utility ratings. The Icraa dataset has a sparsity of 99.60%. It is similar to the sparsity of other educational datasets such as Mace (99.70%) and OpenScout (99.50%) (Fazeli et al., 2014b).

The Java application first split users' ratings randomly into training sets (70%) and test sets (30%). In a second step, the application uses the training set to calculate the top N recommended resources for each user using the five algorithms. Finally, the application uses these recommendations and the test set to compute the F1 scores.

The F1 score [START_REF] Herlocker | Evaluating collaborative filtering recommender systems[END_REF] is used in this experiment to measure the accuracy of each algorithm. F1 ranges between 0 and 1 and combines precision and recall in a single number. We compute F1 for each individual user and calculate the average value to use such as our metric. We computed F1 for the top N (N ranges from 3 to 10) recommendations of the result set for each user.

Results

The results of the offline analysis are presented in Table 4. This table shows the average F1 score achieved by the five algorithms for different values of N. Values in highlighted indicate the best performing F1. This table also illustrates the improvement percentage of our algorithm compared to the best performing one among the four other algorithms. For N = 3, our algorithm outperforms the other baseline algorithms and improves the F1 score by 3.03% in comparison to SVD that takes second place. When N ranges from 4 to 9, our algorithm performs best again and the accuracy of recommendations is improved by 4.92% to 10% compared to the social popularity algorithm that takes second place.

For N = 10, the proposed algorithm and the social popularity method have the same accuracy and have the best F1 compared to the remaining three algorithms.

The improved F1 of our approach shows that our approach can enhance recommendations for learners on social learning platforms even when they contain sparse ratings.

Conclusions

In this paper, we have presented a recommendation approach that can address the information overload problem observed on social learning platforms. This approach is designed to improve the accuracy of recommendations as well as to decrease sparsity, which is a common problem of educational recommender systems.

We have developed a formal model for calculating similarity between users and for generating three types of recommendation. We have also developed a learning environment, called Icraa, which implements the proposed approach. With this system, we conduct a live user experiment with real users and an offline analysis for evaluating our contribution.

Globally, the live user experiment yields positive results. The result of the questionnaire that we submitted to Icraa users was conclusive. The offline analysis using the Icraa dataset shows that the presented approach outperforms the four selected baseline algorithms used as a comparative basis.

We are working on a more generic approach that can take advantage of all social information of a social network, such as social interactions and activities. This information can be taken from the learning platform itself or from a public social network such as Facebook or LinkedIn.

Another perspective of our work is to create a recommender system, based on our approach, which can be used by multiple learning platforms.

E

  ] = {d D: d is a domain of u}. It represents the areas of interest of user u. C The set of user profile information. C[u] C[u] = {(c, val) | c C: val is the value of the characteristic c of user u}. It represents the set consisting of characteristic/value pairs that define the profile of user u. Continuous type values may be replaced by discrete values. For example, the value of age can be transformed into child, adolescent, adult, etc.

  Seval = IEu IEv. It represents the set of co-evaluated resources in terms of quality by users u and v. Sutil Sutil = IUu IUv. It represents the co-evaluated resources in terms of usefulness by users u and v.

Figure 2

 2 Figure 2 Evaluation process (see online version for colours)

Figure 3

 3 Figure 3 (a) Resource access (b) Recommender widget (see online version for colours)

Figure 4

 4 Figure 4 Interface for rating the quality and utility of a resource (see online version for colours)

  2

Figure 5 Figure 6 Figure 7

 567 Figure 5 Mean Likert scale user ratings (see online version for colours)

Table 1

 1 Synthesis of social educational recommender systems

	System	Item types recommended	Recommendation method	Elements used in recommendation	Deal with sparsity of ratings
	ISIS	Next best learning	Ontology method,	Learner profile	Na
	recommender	resource	stereotype method	and domain
	system		and collaborative	knowledge
			filtering
	3A recommender	Users, learning	Graph method	User interactions,	No
	system	resources and		relationships
		activities		ratings,
				bookmarks, tags
				and reviews
	Ensemble	Learning resources	Collaborative	Visits, ratings,	No
	recommender		filtering and	comments and
	system		history-enriched	tags
			information spaces
	Topolor system	Learning	Collaborative	Visits, tags, user	Na
		resources, learning	filtering and content	questions and
		paths, learning	based	answers
		experts, learning
		peers, questions
		and learning topics
	Teaching style	Learning resources K-means clustering	Teaching style	Na
	based
	recommender
	system
	Social learning	Learning resources	Collaborative	Ratings,	Yes
	recommender		filtering, sentiment	comments
	system based on		analysis and
	learner sentiment		stereotype method
	Social trust-aware	Learning resources	Collaborative	Ratings and user	Yes
	recommender		filtering and graph	relationships
	system for		method
	teachers
	Social learning	Learning resources	Graph method	Ratings	Yes
	recommender
	system based on
	graph-walking
	methods
	Social learning	Learning resources	Context aware	Learner affective	Na
	recommender		recommendation	state
	system based on		method.
	learner affective
	state
	Semantic	User actions	32 recommendation	User actions	Na
	educational		rules
	recommender
	system

The system selects friends of the current user.

The system calculates the level of similarity between the current user and her/his friends. The similarity calculation is presented in Section

4.1. 4 The system calculates the score of educational resources based on the visit and evaluations performed by the active user's friends on resources. The formula calculating this score is presented in Section 4.2.

He received his doctoral thesis in 2001. Then he did a Postdoctoral Fellowship at the TeleUniversity of Quebec in Canada, before joining INSA Lyon in 2002.

In this section, we will present the four parts of our proposition:

1 how to calculate similarity between users in a social learning environment 2 how to select resources that best fit user preferences 3 how to deal with the sparsity problem 4 how to update user profiles.

Social similarity

Most recommender systems based on collaborative filtering use the Pearson correlation coefficient [START_REF] Herlocker | Evaluating collaborative filtering recommender systems[END_REF] to calculate similarities between users. This coefficient measures the linear correlation between two variables. In recommender systems, this coefficient is used to measure the dependence between two vectors of resource ratings belonging to two users.

Our approach enhances the similarity measure between users by incorporating social information present on the social learning platform. The method proposed to calculate similarity between two users, noted u and v, is based on:

1 the similarity, between u and v, in terms of visited resources denoted VisSim (u, v) and similarity in terms of resource ratings denoted EvalSim(u, v) and UtilSim (u, v) 2 the similarity between profiles of these two users denoted ProfilSim(u, v)

3 the strength of the relationship between u and v denoted LinkS(u, v).

Formally, social similarity between u and v, denoted SocialSim(u, v) is calculated as follows:

( , ) ( , ) ( , ) ( , ) ( , ) ( , ) 5

SocialSim u v EvalSim u v UtilSim u v VisSim u v LinkS u v ProfilSim u v ( 1 )

To calculate similarity in terms of evaluation EvalSim(u, v), we use the Pearson correlation coefficient formula. For similarity in terms of utility UtilSim(u, v), we have adapted this coefficient to take into account the user domains.