Xavier Lévêque

Elodie Mathieux

Véronique Nerrière-Daguin

Reynald Thinard

Laetitia Kermarrec

Tony Durand

Thomas Haudebourg

Bernard Vanhove

Laurent Lescaudron

Isabelle Neveu

Xavier L Ev Êque

Nerri Ere-Daguin

Philippe Naveilhan
email: philippe.naveilhan@univ-nantes.fr

Local control of the host immune response performed with mesenchymal stem cells: perspectives for functional intracerebral xenotransplantation

Keywords:

à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Transplantation is a promising approach for nervous system repair in neurodegenerative disorders. Clinical studies have shown beneficial effects following the transplantation of embryonic ventral mesencephalic tissue into the brains of Parkinson's disease patients, but the absence of effects or the apparition of dyskinesia in some patients has underlined the need for technical improvements. These include a better control of the host immune response, as alloimmunization to donor antigens has been observed following the transplantation of foetal neural cells into the brains of patients affected by Huntington's disease [START_REF] Krystkowiak | Alloimmunisation to donor antigens and immune rejection following foetal neural grafts to the brain in patients with Huntington's disease[END_REF]. Alternative sources of transplantable cells have also to be found as the use of human foetal tissues raises ethical and logistical problems. In this perspective, we have been studying the possibility of using porcine embryonic ventral mesencephalic tissue [START_REF] Canova | Different mechanisms mediate the rejection of porcine neurons and endothelial cells transplanted into the rat brain[END_REF][START_REF] Melchior | Temporal analysis of cytokine gene expression during infiltration of porcine neuronal grafts implanted into the rat brain[END_REF][START_REF] Michel | Dendritic cell recruitment following xenografting of pig fetal mesencephalic cells into the rat brain[END_REF]. Porcine foetal neurons have been shown to restore behavioural function in animal models of Parkinson's disease [START_REF] Galpern | Xenotransplantation of porcine fetal ventral mesencephalon in a rat model of Parkinson's disease: functional recovery and graft morphology[END_REF][START_REF] Larsson | Neural tissue xenografting[END_REF] and some clinical trials have already been performed [START_REF] Deacon | Histological evidence of fetal pig neural cell survival after transplantation into a patient with Parkinson's disease[END_REF][START_REF] Fink | Porcine xenografts in Parkinson's disease and Huntington's disease patients: preliminary results[END_REF][START_REF] Schumacher | Transplantation of embryonic porcine mesencephalic tissue in patients with PD[END_REF]. However, despite the particular immunological status of the brain, neuronal xenografts become the target of a host immune response characterized by a massive infiltration of dendritic and T cells with a Th1 profile [START_REF] Melchior | Temporal analysis of cytokine gene expression during infiltration of porcine neuronal grafts implanted into the rat brain[END_REF][START_REF] Michel | Dendritic cell recruitment following xenografting of pig fetal mesencephalic cells into the rat brain[END_REF]. Treatment with minocycline [START_REF] Michel-Monigadon | Minocycline promotes long-term survival of neuronal transplant in the brain by inhibiting late microglial activation and T-cell recruitment[END_REF] or cyclosporine A significantly delays the rejection without inhibiting it. Interestingly, mesenchymal stem cells (MSC) showed spontaneous long-term survival following their transplantation into the brain of a xenogeneic host [START_REF] Azizi | Engraftment and migration of human bone marrow stromal cells implanted in the brains of albino rats-similarities to astrocyte grafts[END_REF][START_REF] Rossignol | Mesenchymal stem cells induce a weak immune response in the rat striatum after allo or xenotransplantation[END_REF]. This long-term survival is due not only to the low immunogenicity of MSC but also to their active immunosuppressive activity. Indeed, MSC can inhibit the maturation of dendritic cells and proliferation of T-and B cells [START_REF] Uccelli | Mesenchymal stem cells: a new strategy for immunosuppression?[END_REF][START_REF] Uccelli | Mesenchymal stem cells in health and disease[END_REF]. Interestingly, transplantation of MSC in animal models of stroke or multiple sclerosis revealed that these cells may have beneficial effects through their immune and trophic properties [START_REF] Zappia | Mesenchymal stem cells ameliorate experimental autoimmune encephalomyelitis inducing T-cell anergy[END_REF]. Consequently, we decided to take advantage of these properties to develop a local immunosuppressive strategy in the brain. Our results showed that MSC protect xenogeneic neural cells from cell rejection without affecting their functional properties.

Materials and methods

Animals

Porcine embryos were obtained from domestic Large White pigs 28 days after insemination (G28; Institut National de la Recherche Agronomique, INRA, Nouzilly, France). Lewis 1A and Sprague-Dawley wild-type rats were purchased from Janvier CERJ (Le Genest saint Isle, France) and Charles River (L'Arbresle, France). The experiments were performed according to the French guidelines for the care and use of laboratory animals.

Culture of rat MSC

Mesenchymal stem cells were obtained from the bone marrow of adult Lewis 1A rats as previously described [START_REF] Rossignol | Mesenchymal stem cells induce a weak immune response in the rat striatum after allo or xenotransplantation[END_REF][START_REF] Lescaudron | Assessing the potential clinical utility of transplantations of neural and mesenchymal stem cells for treating neurodegenerative diseases[END_REF][START_REF] Remy | New lines of GFP transgenic rats relevant for regenerative medicine and gene therapy[END_REF]. The ends of femurs and tibias were cut, and bone marrow cells (BMC) were aspirated with a 25-gauge needle. To remove all the BMC, the bone canal was flushed with aMEM medium (Sigma-Aldrich, Saint-Quentin Fallavier, France) supplemented with 20% foetal calf serum (FCS; Gibco, Life technologiies ltd, Asint-Aubin, France), 100 U/ml penicillin (Sigma-Aldrich) and 100 lg/ml streptomycin (Sigma-Aldrich). MSC were selected by an overnight incubation of the BMC onto uncoated plastic dishes. Plastic-adherent cells were then plated at a density of 8 9 10 3 cells/cm 2 and split at 80% confluency. Full characterization of the cells was previously described [START_REF] Rossignol | Mesenchymal stem cells induce a weak immune response in the rat striatum after allo or xenotransplantation[END_REF][START_REF] Remy | New lines of GFP transgenic rats relevant for regenerative medicine and gene therapy[END_REF] and current analyses by flow cytometry indicate that over 95% of the cells were consistently positive for CD90 and negative for CD45. MSC were used at passage 4.

Preparation of porcine neuroblasts

Foetuses at G28 were collected in Hank's balanced salt solution (GibcoBRL) after hysterectomy. Ventral mesencephalons were dissected under aseptic conditions and conserved in a hibernation medium at 4°C for up to 3 days. Just before transplantation, porcine neuroblasts (pNb) were prepared by mechanical and enzymatic dissociation as previously described [START_REF] Canova | Different mechanisms mediate the rejection of porcine neurons and endothelial cells transplanted into the rat brain[END_REF] and the cell suspension was adjusted to a concentration of ~2 9 10 5 cells/ll.

Cell transplantation and surgical procedures

The transplantation was performed on male Lewis 1A rats weighing 250 g. Anaesthesia was performed with an intramuscular injection of 2% Rompun and 50 mg/ml ketamine (1 ml/kg) (PanPharma, Foug eres, France). Anaesthetized rats were placed in a stereotaxic frame (Stoelting, Wood Dale, IL, USA). An amount of 4 9 10 5 pNb, 1 9 10 5 rat MSC or a mix of 3 9 10 5 pNb/1 9 10 5 rMSC was transplanted unilaterally into the striatum (in mm relative to bregma and surface of the dura mater): anterior: +0.7, lateral: À2.8, ventral: À6 and À5.6, incisor bar: À3.3. Injections were performed with a Hamilton syringe placed on an automated microinjector (Phymed, Paris, France) over a time period of 1 min. After a 4-min. delay period, the needle was gently removed, the piece of skull replaced and the skin sutured.

At 28, 35, 42, 63 and 120 days post-transplantation, animals were anaesthetized with an intramuscular injection of 2% Rompun/50 mg/ml ketamine (1 ml/kg) and transcardially perfused with 150 ml 0.9% NaCl, followed by 250 ml cold 4% paraformaldehyde in PBS. The brain was then removed from the skull and cryoprotected in two successive solutions of 15% (24 hrs) and 30% (48 hrs) sucrose in PBS at 4°C. The brains were then frozen in isopentane and stored at À80°C. For the molecular analysis, grafted striata were collected from unperfused animals and directly frozen in liquid nitrogen.

6-hydroxydopamine lesion

Lesion of nigral dopaminergic neurons was carried out by injecting, unilaterally, the 6-hydroxydopamine (6-OHDA) in the medial forebrain bundle. Briefly, rats were anaesthetized with an intramuscular injection of Rompun/ ketamine (1 ml/kg) and placed in a stereotaxic frame (Stoelting). 6-OHDA (3 lg/ll) was dissolved in 0.9% saline/0.1% ascorbic acid, and 4 ll were unilaterally injected (0.33 ll/min) at the following coordinates (in mm relative to bregma and surface of the dura mater): anterior: À4.4; lateral: À1.2; ventral: À7.8; tooth set bar: À2.3. Two weeks after the lesion, the animals were injected with the DA agonist, apomorphine (0.05 mg/kg, s.c.). Only the animals scoring more than 100 contralateral rotations during a 45-min. period were selected for cell transplantation, as this score corresponds to ~90% of dopaminergic loss on the side ipsilateral to the lesion (Fig. 5A). Four experimental groups were then constituted: (i) nontransplanted (n = 8), or transplanted (ii) with pig neuroblasts (n = 8), (iii) LEW 1A MSC (n = 8), (iv) MSC plus pig neuroblasts (n = 13). Cell transplantation was performed 1 month after the 6-OHDA lesion.

Cylinder tests

Functional recovery of 6-OHDA-lesioned rats was followed by analysis of forelimb-use asymmetry every 15 days. Rats were placed in a glass cylinder and the number of wall contacts executed independently with the left and right forepaw was counted for each rat (cylinder test) [START_REF] Schallert | CNS plasticity and assessment of forelimb sensorimotor outcome in unilateral rat models of stroke, cortical ablation, parkinsonism and spinal cord injury[END_REF]. At least 20 contacts were counted for each animal. Left (contralateral) paw use was expressed as a percentage of the total number of wall contacts, knowing that an unlesioned animal achieves a score of 50%.

Total RNA preparation and qPCR analyses

To extract total RNA, cells were disrupted in 1 ml TRIzol â (Invitrogen, Carlsbad, CA, USA) and homogenized using a syringe and needle for cells or using an ULTRA-TURRAX T25 (IKA, Staufen, Germany) for tissues according to the manufacturer's specifications.

Potential genomic DNA contamination was removed by treatment with Turbo TM DNase (Ambion Inc., Austin, TX, USA). RNA was quantified using an ND-1000 UV-Vis spectrophotometer (Nanodrop Technologies, Wilmington, DE, USA) and RNA integrity was controlled on an agarose gel. cDNA was synthesized from 5 lg total RNA using the Moloney Murine Leukemia Virus reverse-transcriptase kit (Invitrogen) and diluted to a final concentration of 100 ng cDNA/ll.

Analyses of transcripts were performed with a GenAmp 7700 sequence detection system (Applied Biosystems, Life technologies ltd, Saint-Aubin, France) using SYBR Green PCR core reagents (AB). Oligonucleotide sequences are indicated in Table 1. To minimize the amplification of porcine mRNA, primers were aligned against porcine mRNA sequences and selected as follows: at least, one of the two primers matched on less than nine nucleotides and/or the 3 0 end was free if the homology was greater than 50%. These conditions were not satisfied for Glial Fibrillary Acidic Protein (GFAP) and transforming growth factor-beta (TGF-b). The PCR method and the 2 ÀDDCt quantification method, after normalization to HPRT values, have been described previously. The mRNA expression level is defined as the fold change in mRNA levels in a given sample relative to levels in a calibrator (CB). The calibrator is the 19 expression of each gene. The mRNA expression level is calculated as follows: mRNA expression level = 2 ÀDDCt where DDCt = (Ct Target À Ct HPRT ) sample À (Ct Target À Ct HPRT ) CB . Specific amplifications were checked by amplicon melting curves.

Immunohistochemical analyses

The frozen brains were sliced coronally into 16 lm sections using a cryostat (Leica, Nanterre, France). Parallel series of nine slides, each containing three brain sections, were prepared. The slides were stored at À80°C until further handling. After neutralization of endogenous peroxidase with 0.3% H 2 O 2 in PBS (Sigma-Aldrich) for 10 min., the slides were incubated for 45 min. in 10% normal goat serum (Sigma-Aldrich) diluted in PBS-4% bovine serum albumin (BSA, Sigma-Aldrich) before an incubation with primary antibodies, as presented in Table 2, overnight at 4°C. After washing, slides were incubated for 1 hr at room temperature with biotinylated antimouse or anti-rabbit Abs (1:500 in PBS-4% BSA). Sections were then washed three times and exposed to avidin-biotinylated-peroxidase complex for 1 hr and revealed with the Vector very intense purple substrate or 3,3-diaminobenzidine (Vector, Laboratories, Burlingame, CA, USA; ABC kit). After several washes in distilled water, slides were dehydrated and mounted using Eukitt (Labonord, Villeneuve d'Ascq, France).

Observations were performed with an Axioskop 2 plus microscope (Zeiss, Marly le Roi, France). Images were acquired using a digital camera (AxioCam HRC, Zeiss) driven by AxioVision Release 4.2 software.

Histological classification of the graft

Brain sections containing the graft were identified by staining one slide of each series with cresyl violet (CV). In the absence of an identifiable graft because of rejection, the needle track or scar was used to localize the injecting point. A minimum of three sections from the graft centre were stained with pNF70 and R73 antibodies to detect porcine neurons and T lymphocytes respectively. The graft status was assessed by two independent readers on blind-coded slides and classified using the following criteria. Healthy grafts: no sign of degeneration (CV + ) and/or a clear expression of the porcine NF (NF70 + ), no (R73 À ) or rare (occasionally observed in one or more sections, but always in a much lower number than 10 cells per brain section, R73+/À) T cells inside the graft. Rejecting graft: clear expression of porcine NF (NF70 + ), very large number of T cells infiltrating the graft and/or the grafted striatum (R73 + ). Rejected: no porcine NF (NF70 À ), a very large number of R73 + cells infiltrating the graft and/or the grafted striatum (R73 + ). Scar: no porcine NF, no T cells. This classification was chosen as it encompassed the five situations observed after the transplantation of pNB into the rat striatum [START_REF] Michel | Dendritic cell recruitment following xenografting of pig fetal mesencephalic cells into the rat brain[END_REF][START_REF] Michel-Monigadon | Minocycline promotes long-term survival of neuronal transplant in the brain by inhibiting late microglial activation and T-cell recruitment[END_REF][START_REF] Michel-Monigadon | Trophic and immunoregulatory properties of neural precursor cells: benefit for intracerebral transplantation[END_REF].

Efficiency of the 6-OHDA lesion

The efficiency of 6-OHDA lesion was controlled by analysing the percentage of dopaminergic loss in the substantia nigra compacta. The number of TH-positive cells was counted at three different levels of the SNc using, as a reference point, the exit of the third nerve (200 lm rostral and caudal to this site). Cell counting was performed with the Mercator stereology software (Explora nova, La Rochelle, France). The percentage of loss was expressed as the number of TH + cells on the unlesioned side minus the number of TH + cells on the lesioned side, divided by the number TH + cells at the unlesioned side, and multiplied by 100. The results were analysed by two-way ANOVA.

Results

MSC promote the long-term survival of porcine mesencephalic neuroblasts following their implantation into the striatum of immunocompetent rats

The immunosuppressive and neurotrophic activities of the MSC derived from the bone marrow of Lewis 1A rats prompted us to test their efficiencies to promote the survival of intracerebral xenotransplants. For this purpose, pNb were transplanted into the striata of Lewis 1A rats, with or without Lewis 1A rMSC and the experiments were designed to use the same total number of transplanted cells in all conditions. Survival of the graft was analysed at 28, 35, 42, 63 and 120 days posttransplantation, with a minimum of five rats for each time-point. The presence of porcine neurons in the rat striatum was controlled for using an antibody that specifically recognizes the porcine neurofilament 70 kD subunit (NF70 + ). Infiltration of the graft by T lymphocytes was investigated using R73, an antibody directed against the T-cell receptor. As illustrated in Figure 1, a combination of these two labellings provides a good evaluation of the graft status. At early stages, such as D28, pNb grafts were usually homogeneous with numerous NF70 + cells and no T cells (NF70 + /R73 À , healthy graft). At intermediate stages, such as D35, pNb grafts were often heterogeneous. Porcine neurons were still present, but the graft was clearly infiltrated by R73 + cells (NF70 + /R73 + , rejecting graft). At later stages, such as D42, NF70-positive labelling was no longer visible, but R73 + cells were still present (NF70 À /R73 + , rejected graft). As indicated in Table 3, variations in this timing existed between animals, but the rejection process in Lewis 1A rats transplanted with pNb was systematically completed at D63. At this time, only a scar was observed (Scar). The presence of MSC significantly impaired this rejection process. Indeed, all the pNb/MSC grafts looked healthy at D35 and 50% of the rats transplanted with both pNb and rMSC exhibited healthy grafts at D120 (Table 3).

Characterization of pNb-MSC cografts

In the rats transplanted with pNb, all the transplants exhibited NF70 + cells at D28 (Fig. 2A andC). In contrast, no NF70 + cells were detected in any of the cotransplants at D28 (Fig. 2B andD). NF70 + cells were found in the cotransplants from D35, but only in two of six rats. This percentage increased with time, reaching 100% at D42 (Fig. 2B). Interestingly, TH + cells were detected in all grafts as soon as they became immunopositive for NF70. In rats transplanted with pNb, TH + cells were found in all transplants at D28. In rats cotransplanted with pNb and MSC, TH + cells were detected at D35, but only in the two animals that were immunopositive for NF70. At D63 and D120, all of the healthy cotransplants exhibited clear NF70 + immunostaining and showed TH + cells (Fig. 2E-H). To evaluate the potential migration of the MSC in the brain, the experiment was performed with GFP-MSC [START_REF] Remy | New lines of GFP transgenic rats relevant for regenerative medicine and gene therapy[END_REF]. Thirty-five days after the transplantation of pNb and GFP-MSC into the rat striatum, fluorescent MSC were only observed within the graft. No migration of transplanted MSC was detected (data not shown).

Characterization of the host cellular immune response

To characterize the cellular immune response, immunohistochemistry was performed with antibodies directed against activated microglial cells/macrophages (Ox42) and T cells (R73). As previously described, the transplantation of pNb into the striatum of adult immunocompetent rats induced a strong cellular response characterized by a massive infiltration of the graft by T cells and activated microglial cells between D28 and D42. In the rats cografted with MSC, infiltration of the graft by T cells (Fig. 2J) was never observed before D42 (Table 3), and at D120 50% of these animals exhibited healthy grafts without any T cells.

To evaluate the impact of MSC on inflammatory markers, the striata of rats transplanted with pNb or pNb-MSC were collected at 28, 35, 42 or 63 days post-transplantation for RNA analyses. Seven or eight animals were culled for each time-point in a given group. Striata from six sham-operated rats were used to determine the basal levels (control). We first analysed the expression of Cb and CD11b as markers of T-cell infiltration and microglial cell activation respectively (Fig. 3). In rats transplanted with pNb, a sixfold and threefold increase in the levels of Cb and CD11b mRNA was observed at D35, respectively, as compared with the basal levels in control rats. At later stages, such as D42 or D63, Cb and CD11b mRNAs had returned to basal levels. In contrast, we did not observe significant increases in the levels of Cb and CD11b in the rats transplanted with both pNb and MSC at any of the stages analysed. To evaluate a potential impact of MSC on host reactive astrocytes, we analysed the expression of rat GFAP transcripts. The results showed an up-regulation of GFAP mRNA Fig. 1 Histological assessment of pNb graft status using pNF70 and R73 antibodies. Healthy grafts exhibit no signs of degeneration, a clear NF70 immunostaining and no (À) or few (+/À) R73 cells (NF70 + /R73 À , Healthy). Rejecting grafts are invaded by numerous T cells (enlargement in the insert), but porcine neurons are still present and express pNF70 (NF70 + R73 + , Rejecting). Rejected grafts are characterized by the disappearance of pNF70 + cells, whereas R73 + cells are still observed (enlargement in the insert) (NF70 À R73 + ). When the rejection process is complete, the T cells have totally disappeared and the graft has become a scar (Scar). NF70orCV + /R7.

3 À 4 5 1 6 2 1 0 2 3 NF70 + /R7.3 +/À 1 0 3 0 0 2 0 1 0 NF70 + /R7.3 + 1 0 1 0 2 0 0 1 0 NF70 À /R7.3 + 0 0 1 0 2 1 0 1 0 Scar 0 0 0 0 0 1 6 1 3
The graft status was assessed using cresyl violet, NF70 and R73 staining. CV + /R73 À and NF70 + /R73 À are considered as healthy graft with no (À) or low (+/À) amount of R73 T cells, no sign of degeneration (CV + ) and/or a clear expression of the porcine neurofilament in neurons (NF70 + ). NF70 + R73 + and NF70 À R73 + grafts are considered as rejecting and rejected graft respectively. Scar corresponds to the stage of almost complete disappearance of the graft. R73 +/À corresponds to a brain with 1-10 T cells per section.

at D28, preceding the peak of CD11b and Cb. The levels of GFAP transcripts further increased, reaching a 5.4-fold increase at D35 and remaining significantly increased at D42. In the rats transplanted with both pNb and MSC, a significant increase in the levels of GFAP mRNA was observed only at D35, and this increase was only 1.7-fold. We next analysed the expression of chemokines and cytokines that might favour or inhibit the host immune response (Fig. 4). As reported in Figure 5, MCP1 and Rantes were significantly up-regulated at D35 in pNb-grafted animals, and the up-regulation of these two chemokines was correlated with the peak expression of Cb and CD11b. We also found, at D28, an up-regulation of the pro-inflammatory cytokines, INFc and IL-6, which preceded the up-regulation at D35 of anti-inflammatory cytokines such as IL-10 and TGF-b. The levels of interferon-c (IFN-c) remained high up to D42 and then strongly decreased, whereas IL-6 returned to its basal level before a second up-regulation at D63. A high level of IL-10 was also observed from D35 to D63, whereas TGF-b returned to its basal level as early as D42. Interestingly, we did not observe any of these regulations in the rats that received both pNb and MSC. The levels of MCP1 and RAN-TES (CCL5), as well as the levels of the four studied cytokines, were not significantly different from their basal levels in control animals at any of the time-points determined. To further characterize the immunomodulatory mechanism triggered by rat MSC in our experimental model, we analysed the expression of iNOS and HO1, two enzymes previously shown to be mediators of MSC function. As shown in Figure 4, the mRNAs of both of these enzymes were up-regulated in the striata of rats grafted with pNb, with the difference that HO1 expression increased between D28 and D42, whereas the level of iNOS increased between D35 and D63. Interestingly, in the presence of MSC, the up-regulation of HO1 was not observed, whereas the upregulation of iNOS mRNA still occurred.

Effect of cografting on the motor behaviour of 6-OHDA-lesioned rats

To evaluate the functional impact of a pNb-MSC cotransplant, the cells were transplanted in a rat model of Parkinson's disease. Rats lesioned by unilateral injection of 6-OHDA were selected on the basis of their response to the dopaminergic agonist, apomorphine. Postmortem analysis confirmed a minimum of 90% loss of dopaminergic neurons in the substantia nigra pars compacta ipsilateral to the lesion (Fig. 5A andC). Lesioned rats were transplanted with pNb, or pNb + MSC and motor recovery was assessed every 15 days using the cylinder test. Sham-operated rats were used as controls, but 6-OHDAlesioned rats were also transplanted with 1 9 10 5 rMSC to evaluate a potential impact of this amount of MSC on the motor behaviour. As shown in Figure 5B, no recovery was observed in the rats transplanted with MSC or pNb alone. In contrast, significant signs of motor improvement were observed at D90 and D105 in the group of rats that received both pNb and MSC. As the beneficial effect was no longer significant at D120, the rats were divided according to the presence or absence of transplant. At D120, the rats displayed either a scar (Fig. 5G) or a healthy transplant with NF70 + (Fig. 5E) and TH + (Fig. 5F) cells. Intermediate stages were not observed. No activated microglial cells were observed in the control or pNb-MSC-transplanted rats. Inclusion of this parameter in the interpretation of the behavioural tests clearly showed significant recovery in rats displaying a healthy transplant (Fig. 5D). The rats that exhibited a healthy graft were those that presented the best performance in the cylinder test, whereas the performance of the rats that had rejected their graft was similar to that of the lesioned group that received vehicle (control). Motor recovery in 6-OHDA-lesioned rats was, therefore, correlated with the long-term survival of the pNb-MSC transplants.

Discussion

Our present paper provides the first evidence that cotransplantation of syngeneic MSC with xenogeneic neuroblasts promotes the longterm survival of xenografted neurons and allows motor recovery in a rat model of Parkinson's disease. Behavioural improvement has previously been observed after the transplantation of pNb into the striatum of lesioned rats, but the animals were always treated with high doses of classic immunosuppressors [START_REF] Galpern | Xenotransplantation of porcine fetal ventral mesencephalon in a rat model of Parkinson's disease: functional recovery and graft morphology[END_REF][START_REF] Huffaker | Xenografting of fetal pig ventral mesencephalon corrects motor asymmetry in the rat model of Parkinson's disease[END_REF][START_REF] Larsson | Intrastriatal ventral mesencephalic xenografts of porcine tissue in rats: immune responses and functional effects[END_REF]. Here, we show that pNb cotransplanted with rMSC can survive up to 120 days without systemic immunosuppression. This prolonged survival is most probably the result of an immunosuppressive activity, as the loss of xenografted neurons is because of a massive and sudden host immune response at 4-6 weeks [START_REF] Canova | Different mechanisms mediate the rejection of porcine neurons and endothelial cells transplanted into the rat brain[END_REF][START_REF] Michel | Dendritic cell recruitment following xenografting of pig fetal mesencephalic cells into the rat brain[END_REF]. This observation is supported by the description of MSC as potent immunosuppressors, acting at different levels of the immune response [START_REF] Uccelli | Mesenchymal stem cells: a new strategy for immunosuppression?[END_REF][START_REF] Uccelli | Mesenchymal stem cells in health and disease[END_REF]. Human MSC alter the maturation of dendritic cells as well as their ability to present antigens to T cells, which is a key point of their activation [START_REF] Maccario | Human mesenchymal stem cells and cyclosporin a exert a synergistic suppressive effect on in vitro activation of alloantigenspecific cytotoxic lymphocytes[END_REF]. They are also able to inhibit T-cell proliferation and to affect the differentiation of B cells into plasmocytes [START_REF] Corcione | Human mesenchymal stem cells modulate B-cell functions[END_REF]. The MSC isolated from the bone marrow of Lewis 1A rats share similar immunosuppressive properties, as illustrated by their ability to inhibit the proliferation of in vitro-activated T cells and by the limited host immune response in pNb/ MSCcografted rats. Indeed, numerous cellular and molecular events that are usually induced by the implantation of pNb in the rat brain are not observed or are strongly reduced in the presence of MSC. For instance, the accumulation of GFAP mRNA that precedes cell rejection in pNb-grafted rats is strongly reduced in cografted rats, suggesting an inhibition of astroglial activation. The absence of a significant increase in Cb and CD11b levels at the analysed time-points probably reflects a direct or indirect inhibitory effect of MSC on T lymphocyte and microglial cell activation. In vitro analyses raise the possibility of an effect through the secretion of anti-inflammatory molecules such as TGF-b1 or IL10 [START_REF] Lescaudron | Assessing the potential clinical utility of transplantations of neural and mesenchymal stem cells for treating neurodegenerative diseases[END_REF]. However, in vivo analyses did not reveal upregulation of such anti-inflammatory molecules. On the contrary, we observed a general abolition of pro-and anti-inflammatory signals. Indeed, MCP1, RANTES, IFN-c, IL-6, IL-10, TGF-b and HO-1 remained at their basal levels in the striata of rats grafted with both pNb and MSC in contrast to their up-regulation at around D35 in the brains of pNb-grafted rats. Only one molecule was up-regulated in both groups: iNOS. This molecule might be implicated in the immunosuppressive effects of MSC through the production of NO. Indeed, Ren et al. revealed a major role of iNOS in the immunosuppressive properties of mouse MSC [START_REF] Ren | Mesenchymal stem cell-mediated immunosuppression occurs via concerted action of chemokines and nitric oxide[END_REF] and exacerbation of the EAE inflammatory response has been observed in mice deficient for iNOS [START_REF] Fenyk-Melody | Experimental autoimmune encephalomyelitis is exacerbated in mice lacking the NOS2 gene[END_REF]. The mechanism of action remains to be fully determined, but NO was reported to be a potent inhibitor of T-cell proliferation [START_REF] Albina | Nitric oxide production is required for murine resi-dent peritoneal macrophages to suppress mitogen-stimulated T cell proliferation. Role of IFN-gamma in the induction of the nitric oxide-synthesizing pathway[END_REF][START_REF] Krenger | Interferon-gamma suppresses T-cell proliferation to mitogen via the nitric oxide pathway during experimental acute graftversus-host disease[END_REF] and leucocyte adhesion on the endothelial cell layer [START_REF] Kubes | NO donors prevent integrin-induced leukocyte adhesion but not P-selectin-dependent rolling in postischemic venules[END_REF]. A local overexpression of iNOS by MSC in the cograft might, therefore, favour the long-term survival of the intracerebral xenograft, but molecules other than NO are most probably implicated. Thus, further studies will be required to fully characterize the mechanisms by which MSC promote the long-term survival of xenografted pNb.

Neural stem/progenitor cells (NSPC) also display immunosuppressive properties and show prolonged survival in the brains of xenogenic hosts [START_REF] Michel-Monigadon | Trophic and immunoregulatory properties of neural precursor cells: benefit for intracerebral transplantation[END_REF][START_REF] Bonnamain | Expression of heme oxygenase-1 in neural stem/progenitor cells as a potential mechanism to evade host immune response[END_REF]. As they can generate neurons, these cells are considered to be a potential cell source for restorative strategies, but, in fact, very few pNSPC differentiate spontaneously into dopaminergic neurons in vivo [START_REF] Michel-Monigadon | Trophic and immunoregulatory properties of neural precursor cells: benefit for intracerebral transplantation[END_REF]. We, thus, considered combining the ability of ventral mesencephalic neuroblasts to generate dopaminergic neurons and the immunosuppressive properties of MSC. The long-term survival of these cotransplants in the brains of xenogenic hosts and their functional effects in a rat model of Parkinson's disease suggest that the intracerebral implantation of 100,000 MSC with 300,000 pNb might be a good basis to develop new restorative strategies with high efficiency and low detrimental secondary effects. Characterization of the mechanisms by which locally implanted MSC contribute to motor recovery in immunocompetent rats should be of great help to optimize such strategies.

ª

  2014 The Authors. Journal of Cellular and Molecular Medicine published by John Wiley & Sons Ltd and Foundation for Cellular and Molecular Medicine.

Fig. 2

 2 Fig. 2 Immunohistological analyses of the xenotransplants. (A-B) Numbers of rats grafted with pNb (A) or pNb and MSC (B), displaying an NF70negative healthy graft (white bar), an NF70-positive graft (black bar) or a rejected graft (grey bar). (C-H) Representative micrograph of healthy grafts following the transplantation of pNb alone (C) or in the presence of MSC (D-H). At D28, NF70 immunostaining was observed in the pNb graft (C), whereas no NF70 + cells were observed in the cografts (D). At later stages, such as D63 (E and F) and D120 (G and H), the cografts were strongly immunopositive for NF70 (E and G) and exhibited TH + cells (F and H). (I and J) Representative micrograph of a pNb-MSC cograft undergoing rejection. The grafts were immunostained with OX42 (I) or R73 (J). Scale Bars: C-H, 50 lm; I and J, 150 lm; insert, 25 lm.

Fig. 3

 3 Fig.3Relative expression of Cb, CD11b and GFAP mRNAs in grafted animals. Total RNAs from striata grafted with pNb (ÀMSC), or pNb + MSC (+MSC), were collected at days 28, 35, 42, 63 (n = 7/8 per group per day) and submitted to qPCR. D0 corresponded to ungrafted animals (n = 6). Mean values AE SEM are presented (ANOVA test, *P < 0.05; **P < 0.01; ***P < 0.001).

Fig. 4

 4 Fig.4Relative transcriptional expression of chemokines, cytokines and cytoprotective enzymes in grafted animals. Total RNAs prepared from the striata of rats transplanted with pNb (ÀMSC), or pNb + MSC (+MSC), were collected at days 28, 35, 42, 63 and submitted to qPCR (n = 7/8 per group per day). D0 corresponds to untransplanted animals (n = 6). Mean values AE SEM are presented (ANOVA test, *P < 0.05; **P < 0.01; ***P < 0.001).

Fig. 5

 5 Fig. 5 Functional recovery of 6-OHDA-lesioned rats. Rats unilaterally lesioned with 6-OHDA were transplanted with MSC, pNb, pNb+MSC, or vehicle (Control), and tested for motor recovery. (A) neuronal loss in the substantia nigra. TH immunostaining showing the loss of TH + neurons in the substantia nigra ipsilateral (right) to the lesion. (B) Forelimb asymmetry was tested every 15 days with the cylinder test. Data are expressed as percentage of limb preference for the contralateral (left) forepaw. n = 8 per group, except n = 13 for the MSC + pNb group, mean value AE SEM (ANOVA test, **P < 0.01). (C) The neuronal loss as a result of the 6-OHDA lesion was estimated by counting the numbers of TH + cells in the substantia nigra of rats transplanted with MSC, with MSC+pNb, or untransplanted rats. Mean values AE SEM are presented. (D) Functional recovery according to the presence of transplant. Cylinder test scores at D120 were expressed according to the presence (Graft, n = 4) or absence (Scar, n = 9) of healthy graft in the rat striatum, and compared with the vehicle group (Crtl, n = 8). Mean values AE SEM are presented (one-way ANOVA with Bonferroni post hoc test, *P < 0.05, **P < 0.01). (E-G) At D120, transplanted rats exhibited either a healthy graft with NF70 + (E, NF staining) and TH + (F, TH staining) neurons or a scar (G, cresyl violet staining).

Table 1

 1 Primers used in this study

	Primers	Sequence	Rattus norvegicus accession
	rHPRT	Up CCTTGGTCAAGCAGTACAGCC	NM_012583
		Lo TTCGCTGATGACACAAACATGA
	rCb	Up GTGAATGGGAAGGAGATCCG	AY228549.1
		Lo CACTGATGTTCTGTGTGACAGGTT
	CD11b	Up TTGGAGTTGCCTGTGAAGTACG	NM_012711.1
		Lo TGCGACAGACACTTGAGAGGTT
	rGFAP	Up CTAGCCCTGGACATCGAGATC	NM_017009.2
		Lo TCCTGCTTCGACTCCTTAAT GA
	MCP1	Up CCACTCACCTGCTGCTACTCAT	NM_031530.1
		Lo TTCTGATCTCACTTGGTTCTGGTC
	rRANTES Up GCATCCCTCACCGTCATCCT	NM_031116.3
		Lo TAGCTCATCTCCAAATAGTTGAT
	IFN-c	Up TGGATGCTATGGAAGGAAAGA	NM_138880.2
		Lo GATTCTGGTGACAGCTGGTG
	rIL6	Up GCAAGAGACTTCCAGCCAGTT	NM_012589.2
		Lo CATCATCGCTGTTCATACAATCA
	rIL10	Up TGCTATGTTGCCTGCTCTTACTG	NM_012854.2
		Lo TCAAATGCTCCTTGATTTCTGG
	rTGF-b	Up CTCAACACCTGCACAGCTCC	NM_021578.2
		Lo ACGATCATGTTGGACAACTGCT
	rHO-1	Up CCACAGCTCGACAGCATGTC	NM_012580.2
		Lo GTTTCGCTCT ATCTCCTCTT CCA
	iNOS	Up GACCAAACTGTGTGCCTGGA	NM_012611
		Lo TACTCTGAGGGCTGACACAAGG
	126		ª 2014 The Authors.

Journal of Cellular and Molecular Medicine published by John Wiley & Sons Ltd and Foundation for Cellular and Molecular Medicine.

Table 2

 2 Antibodies used in this study

	Specificity	Host	Clone	Dilution	Target cells	Origin
	CD11b	Mouse	Ox42	Supernatant	Macrophages, microglial cells	1
	Porcine NF70	Mouse	DP5	10 lg/ml	Porcine neurons	2
	T-cell receptor	Mouse	R73	Supernatant	T cells	1
	a-b chain
	TH	Rabbit	Polyclonal	0.1 lg/ml	Dopaminergic neurons	3
	Mouse IgG (FITC)	Goat	Polyclonal	1.5 lg/ml		4
	Mouse IgG (Biot)	Horse	Polyclonal	1 lg/ml		4
	Rabbit IgG (FITC)	Donkey	Polyclonal	3 lg/ml		4
	Rabbit IgG (Biot)	Goat	Polyclonal	2.2 lg/ml		4
	1, European Collection of Cell Culture, Salisbury, UK; 2, Universit e Paris VII, Paris, F; 3, PelFreeze, Rogers, AR, USA; 4, Jackson Immunore-
	search, Cambridgeshire, UK.

ª 2014 The Authors. Journal of Cellular and Molecular Medicine published by John Wiley & Sons Ltd and Foundation for Cellular and Molecular Medicine.

Table 3

 3 Graft survival in rats grafted with pNb or pNb+MSC

		D28		D35		D42		D63		D120
	MSC	À	+	À	+	À	+	À	+	+
	Number of rats	6	5	6	6	6	5	6	6	6

ª 2014 The Authors. Journal of Cellular and Molecular Medicine published by John Wiley & Sons Ltd and Foundation for Cellular and Molecular Medicine.

Acknowledgements

We are grateful to Dr. P. Brachet and Dr. I. Anegon for their support. The work was supported by the "Association Franc ßaise contre les Myopathies" (AFM) and the Centaure Foundation. XL and EM were supported by the Progreffe Foundation and by the "F ed eration des Groupements de Parkinsoniens": CECAP.

Disclosure

The authors indicate no potential conflicts of interest.


Kiewan Villatel
email: k.villatel@criteo.com

Elena Smirnova
email: e.smirnova@criteo.com

Jérémie Mary
email: j.mary@criteo.com

Philippe Preux
email: philippe.preux@inria.fr

Recurrent Neural Networks for Long and Short-Term Sequential Recommendation

Keywords: recommender systems, sequence modeling, recurrent neural networks, sequential recommendation

Recommender systems objectives can be broadly characterized as modeling user preferences over short-or long-term time horizon. A large body of previous research studied long-term recommendation through dimensionality reduction techniques applied to the historical user-item interactions. A recently introduced session-based recommendation setting highlighted the importance of modeling short-term user preferences. In this task, Recurrent Neural Networks (RNN) have shown to be successful at capturing the nuances of user's interactions within a short time window. In this paper, we evaluate RNN-based models on both short-term and long-term recommendation tasks. Our experimental results suggest that RNNs are capable of predicting immediate as well as distant user interactions. We also find the best performing configuration to be a stacked RNN with layer normalization and tied item embeddings.

INTRODUCTION

In this work, we consider recommender systems from the perspective of temporal user preferences. We consider two groups of tasks:

• long-term prediction (what the user will do at a longer time horizon), • short-term prediction (what item the user is going to interact with next). Previous research has extensively studied modeling of long-term user interests [START_REF] Grbovic | E-commerce in your inbox: Product recommendations at scale[END_REF][START_REF] Koren | Matrix factorization techniques for recommender systems[END_REF]. The primary approach in this task aims at capturing the latent user preferences by learning a low-dimensional representation from historical user-item interactions. Long-term prediction has also been a focus of multiple competitions where the tasks consisted of predicting the items that will eventually be bought or booked by a user [1,[START_REF] Ben-Shimon | RecSys Challenge 2015 and the YOO-CHOOSE Dataset[END_REF].

Recently, predicting short-term user interests have been studied in the context of session-based recommendation [START_REF] Hidasi | Session-based Recommendations with Recurrent Neural Networks[END_REF][START_REF] Kiam Tan | Improved Recurrent Neural Networks for Session-based Recommendations[END_REF]. Hidasi et al. [START_REF] Hidasi | Session-based Recommendations with Recurrent Neural Networks[END_REF] focused on the next item prediction task and successfully applied a RNN model in this setting. RNN model learns a fine-grained representation of user recent activity that allows it to predict the immediate user interaction. In follow-up works, multiple extensions to [START_REF] Hidasi | Session-based Recommendations with Recurrent Neural Networks[END_REF] have been proposed, namely, item content modeling [START_REF] Hidasi | Parallel Recurrent Neural Network Architectures for Feature-rich Sessionbased Recommendations[END_REF] and context modeling [START_REF] Elena | Contextual Sequence Modeling for Recommendation with Recurrent Neural Networks[END_REF].

Quadrana et al. [START_REF] Quadrana | Personalizing Session-based Recommendations with Hierarchical Recurrent Neural Networks[END_REF] showed that short-term prediction can be enhanced by considering longer sequences of interactions. In particular, authors proposed a Hierarchical RNN constituted of two Gated Recurrent Units (GRU) [START_REF] Cho | On the Properties of Neural Machine Translation: Encoder-Decoder Approaches[END_REF]. The first one is used to predict the next item that will be seen by the user during the session. The second one is responsible for modeling information across user sessions and keeping track of user's interest over time.

Hierarchical architectures have also shown promising results in other application domains. For example, authors in [START_REF] Chung | Hierarchical Multiscale Recurrent Neural Networks[END_REF] proposed the Hierarchical Multiscale Recurrent Neural Network (HM-RNN) model that automatically learns a hierarchical structure of sequences of natural text. This model aims at learning a high level representation of the sequence in a unsupervised way to allow to capture the long-term dependencies between words.

In this paper, we are interested in evaluating RNN-based methods on both short-term and long-term recommendation tasks. Similar to [START_REF] Devooght | Long and Short-Term Recommendations with Recurrent Neural Networks[END_REF], we measure the Recall@K metric at varying number of steps in the user sequence depending on the task.

We perform experiments on two real-world datasets containing user browsing activity on e-commerce websites. We evaluate multiple RNN architectures, including the GRU architecture and multi-layer hierarchical approaches, against non-RNN baselines. To achieve the best results, we also experimented with improvements suggested in RNN research, namely, layer normalization [START_REF] Lei Ba | Layer Normalization[END_REF] and tied embedding matrix for input and output layers [START_REF] Inan | Tying Word Vectors and Word Classifiers: A Loss Framework for Language Modeling[END_REF].

From our experiments, we found that the best configuration is a stacked GRU model with layer normalization and tied item embedding matrix. This model consistently achieves the best performance on both short-term and long-term recommendation tasks across datasets. In our experiments, a more complicated HM-RNN architecture is inferior to a one layer GRU with layer normalization. Our experimental results also confirm previous findings of [START_REF] Ludewig | Evaluation of Session-based Recommendation Algorithms[END_REF] that a simple baseline based on co-occurrence can achieve comparable performances to a GRU model, especially on short sequences.

To summarize, our contributions are as follows:

• We evaluate state-of-the-art RNN architectures on shortterm and long-term recommendation tasks, • We show that using state-of-the-art techniques, namely, layer normalization or tying the input embedding matrix for the output module, consistently improves the short-term and long-term performance, • We achieve further improvements using a 2-layers GRU RNN, especially on long sequences.

The rest of the paper is organized as follows. In Section 2 we present an overview of previous research related to our work. Next, in Section 3 we describe our sequence modeling approach and the evaluation metrics for short-term and long-term tasks. In Section 4 we present and discuss the results of our experiments. We conclude in Section 5 and give directions for future research.

RELATED WORK 2.1 RNNs for short-term prediction

The problem of session-based recommendations has been first introduced by Hidasi et al. [START_REF] Hidasi | Session-based Recommendations with Recurrent Neural Networks[END_REF]. Authors showed that the GRU architecture [START_REF] Cho | On the Properties of Neural Machine Translation: Encoder-Decoder Approaches[END_REF] significantly outperforms non-sequential baselines on the next item prediction task. To allow efficient training, they proposed to use session-parallel mini-batches and a ranking loss with mini-batch based output sampling. [START_REF] Kiam Tan | Improved Recurrent Neural Networks for Session-based Recommendations[END_REF] used data augmentation to further improve RNN performance. They also suggested to use item embeddings as a prediction target to reduce output dimensionality and training time.

Quadrana et al. [START_REF] Quadrana | Personalizing Session-based Recommendations with Hierarchical Recurrent Neural Networks[END_REF] proposed an hierarchical RNN-based architecture that improves next item prediction by modeling longer sequences of interactions. Similarly to short-term and long-term user profile, the proposed Hierarchical RNN computes within-session and between-session user representations.

Further improvements to next item prediction have been archived in [START_REF] Song | Multi-Rate Deep Learning for Temporal Recommendation[END_REF]. Authors proposed to combine an RNN short-term prediction with long-term static user and item features computed using a feed-forward neural network. A similar idea described in [START_REF] Twardowski | Modelling Contextual Information in Session-Aware Recommender Systems with Neural Networks[END_REF] employs an RNN to produce an embedding of the sequence of events (contextual information) that is then aggregated with the current item embedding to produce the next item recommendation.

Long and short-term interests

Several works proposed to leverage both long-term and short-term user interests. In [START_REF] Jannach | Adaptation and Evaluation of Recommendations for Short-term Shopping Goals[END_REF] authors showed that some models based only on short-term intentions (the latest interactions) already constitute very strong baselines, but better performance can be achieved by combining them with long-term user profiles. In this research direction, [START_REF] Xiang | Temporal Recommendation on Graphs via Long-and Shortterm Preference Fusion[END_REF] proposed a recommender system based on graphs and a new algorithm to balance between users' long-term and short-term preferences.

Our work is closely related to the study of Devooght and Bersini [START_REF] Devooght | Long and Short-Term Recommendations with Recurrent Neural Networks[END_REF]. In this work, authors benchmarked several models on various metrics to assess their short and long-term performances. In particular, they showed that RNNs are well suited for short-term recommendation. In addition, they proposed multiple techniques, namely dropout, sequence shuffling and the hinge loss with several targets, to improve RNNs performances on long-term recommendations. Our work is different mainly in the evaluation procedure and the long-term metric definition. Devooght and Bersini [START_REF] Devooght | Long and Short-Term Recommendations with Recurrent Neural Networks[END_REF] computed their long-term metric by splitting the test sequences in two halves: the prediction based on the first half was evaluated against items in the second half. In our work, we propose to add a parameter that controls the number of future items to predict and we evaluate the predictions after each possible time-step in the test set. This methodology allows to fully benefit from all the data in the test set and closer reflects the production setting where items are recommended throughout the user session, even after the first interaction.

Hierarchical and multi-scale RNN

Hierarchical and multi-scale RNN models are motivated by the problem of vanishing gradient which prevents RNNs from capturing long term dependencies [START_REF] Bengio | Learning long-term dependencies with gradient descent is difficult[END_REF]. Hierarchical RNN [START_REF] El | Hierarchical Recurrent Neural Networks for Long-Term Dependencies[END_REF] was an early attempt to solve the vanishing gradient issue with a hierarchical structure. It uses units with different time scales and delayed connections. Clockwork RNN [18] is similar to [START_REF] El | Hierarchical Recurrent Neural Networks for Long-Term Dependencies[END_REF] and splits the hidden state into several modules, each having its own update frequency. Each module is only connected to slower modules. HM-RNN has been proposed in [START_REF] Chung | Hierarchical Multiscale Recurrent Neural Networks[END_REF] and is able to learn the hierarchy of the data. It consists of stacked RNN layers and boundary variables controlling when each layer should be updated with three operations: COPY, UPDATE and FLUSH.

PROPOSED APPROACH

This section is organized as follows. First, we describe the RNN model in application to sequential recommendation task. Next, we introduce techniques that improve the performance of RNN models. Finally, we present the metrics that we use to assess the model performances on short-term and long-term recommendation tasks.

Setup

Given a sequence of items X = {x t }, t = 0..T , the recommendation objective is to predict the likely continuations of the sequence. Each item

x t ∈ R N O is represented by a one-hot encoded vector of dimension N O , where N O is the number of items.
The joint probability of the sequence P(X ) can be decomposed using a chain rule:

P(X ) = P(x 0 , x 1 , ..., x T ) = T i=1 P(x i |x i <t ) (1)
The task is then reduced to the task of predicting the next item given the history of the past interactions. In the following, we will model P(x i |x i <t ) using an RNN.

Model

The RNN model consists of three modules: an input module that computes a dense embedding from the one-hot encoded input, a recurrent module that models the sequence of embedded items and an output module that computes the final prediction from the sequence representation. We provide detailed description of each module below.

3.2.1 Input module. The input module maps the one-hot input representation x t into a dense embedding e t :

e t = f in (x t ), (2)
where e t ∈ R N E and N E is the size of the embedding vector.

In our experiments, we used a simple linear projection.

Recurrent module.

The recurrent module models the sequence of items. It updates the sequence representation using previous sequence representation and the current item embedding given by the input module:

h t = f r ec (e t , h t -1 ), (3)
where h t ∈ R N H and N H is the number of dimensions in sequence representation.

We experimented with different variations of the recurrent module that we describe below.

GRU [START_REF] Cho | On the Properties of Neural Machine Translation: Encoder-Decoder Approaches[END_REF]. GRU adds a gating mechanism to the vanilla RNN in order to cope with the vanishing gradient issue [START_REF] Bengio | Learning long-term dependencies with gradient descent is difficult[END_REF]. Gates control the amount of information that must be incorporated in the hidden state as well as a mechanism to forget what has been previously stored in the state. Equations describing the dynamics of the GRU are as follows:

r t = σ (W r e e t + W r h h t -1 ) ht = tanh (W h e e t + W h h (r t ⊙ h t -1 )) z t = σ (W z e e t + W z h h t -1 ) h t = (1 -z t ) ⊙ h t -1 + z t ⊙ ht , (4)
where Stacked RNN. Similar to feed-forward neural networks, we can increase the depth of a recurrent neural network by adding more layers. In stacked RNN, the output of the lower-level recurrent module is used as the input to the higher-level recurrent module. For prediction, the output of the highest-level recurrent module is used. [START_REF] Chung | Hierarchical Multiscale Recurrent Neural Networks[END_REF]. HM-LSTM is an HM-RNN with a LSTM [START_REF] Hochreiter | Long Short-Term Memory[END_REF] update rule. HM-LSTM can be seen as a variant of a stacked LSTM where the higher layers are updated only once every few time steps. In contrast to previous works on multi-scale RNN [START_REF] El | Hierarchical Recurrent Neural Networks for Long-Term Dependencies[END_REF]18], the update rate of the different layers is not fixed in advance. Instead, the hierarchical structure is automatically inferred using the joint learning. The update of higher level representations is controlled by the boundaries. Boundaries are Bernoulli random variables that decide whether to perform three different operations: COPY, UPDATE and FLUSH.

r t ∈ R N H is the reset gate, z t ∈ R N H is the update gate, h ∈ R N H is

HM-LSTM

Output module.

The output module computes the unormalized predictions of the next item based on the updated sequence embedding given by the recurrent module.

o t = f out (h t ), (5)
with o t ∈ R N 0 , N O is the number of items.

In our experiments we used two different output modules. The first one consists of a simple linear projection: o t = W O h t . The second one is also a linear projection, but the projection matrix W O is tied with the input embedding matrix W I : o t = W T I h t . This technique has been proposed in [START_REF] Inan | Tying Word Vectors and Word Classifiers: A Loss Framework for Language Modeling[END_REF].

The softmax function is applied to the output of the output module to obtain a probability distribution over items:

p(x t |x <t ) = So f tmax(o t ) = exp(o 1 t ) N o i=0 exp(o i t ) , exp(o 2 t ) N o i=0 exp(o i t ) , ..., exp(o N o t ) N o i=0 exp(o i t ) (6)

Optimization objective

The output of the network at a given time step t is an estimation of the probability distribution over items for the time step t + 1. We want this distribution to be close to the data distribution. Therefore, we minimize the negative log likelihood of the data distribution under the model:

Loss = - N s i=1 T (i ) t =1 loд(p(x (i) t |x (i) <t )), (7)
where N s is the number of sequences in the dataset, T (i) is the size of the i th sequence, and

x (i)
t is the item t of sequence i. We trained the RNN models jointly (the item embeddings W I are not pre-trained) and the parameters were learned using Back Propagation Through Time [START_REF] Mozer | A Focused Backpropagation Algorithm for Temporal Pattern Recognition[END_REF].

Model improvements

3.4.1 Layer Normalization [START_REF] Lei Ba | Layer Normalization[END_REF]. Normalization techniques like Batch Normalization [START_REF] Ioffe | Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift[END_REF] have been proven to be beneficial for training of deep neural networks. Layer normalization is another normalization technique that normalizes neuron activations across layer. It is well suited for RNN models as it does not require to compute per time step statistics. [START_REF] Lei Ba | Layer Normalization[END_REF] showed that layer normalization helps to stabilize the hidden state dynamic of RNNs and tends to reduce the training time, especially on long sequences and small batches. Browsing history datasets can potentially contain very long sequences (see Section 4.3) and a large input space that require the use of a small batch size for practical reasons (e.g., memory constraints). Therefore, we experiment by applying layer normalization to our proposed model.

3.4.2 Tied embedding matrix. [START_REF] Inan | Tying Word Vectors and Word Classifiers: A Loss Framework for Language Modeling[END_REF] proposed to tie the input embedding matrix to the output projection layer. It constrains the model to provide close predictions when items are similar in terms of embedding. In addition to improved performance, this technique also greatly reduces the number of parameters. This is particularly important when the item space is large as in our experimental datasets (see Section 4).

Long and short-term metrics

3.5.1 Definition. To assess the model performance on shortterm and long-term recommendation tasks, we extend the Recall@K metric by adding a parameter N that controls the number of future items taken into account to compute the set of relevant items. Recall@K,N is therefore the proportion of items that have been observed during the next N steps and that appear in the top K list of predicted items. More formally:

Recall@K = |S r ec ∩ S r el | |S r el | (8)
where S r ec is the set of recommended items (top K recommendations) and S r el is the set of relevant items. In Recall@K,N the set of relevant items S r el consists of the next N observed items in the sequence. Usually in sequential recommendation, the set of relevant items consist of only one element -the next observed item. This corresponds to a particular case of our metric where N = 1. To simplify notations, we use the Recall@K to denote Recall@K,1 when there is no ambiguity.

Evaluation on long-term metrics.

The Recall@K,N metric requires prediction of the next N items in the sequence. We use a method similar to [START_REF] Devooght | Long and Short-Term Recommendations with Recurrent Neural Networks[END_REF] that consists of taking top K predictions of the next item. The set of predicted items therefore consists of the top K most probable items in the prediction of the next item, and the set of relevant items is the list of the next N observed items.

EXPERIMENTS

We benchmarked multiple RNN architectures presented in Section 3.2.2 on the sequential recommendation task and studied the impact of techniques presented in Section 3.4 on both short-term and long-term recommendation.

The rest of this section is organized as follows. First, we present the baselines and datasets that we used for our experiments. Then we detail the configurations of models and the choice of hyperparameters. Finally, we discuss the results.

Baselines

We used the following baselines:

• POP. Recommends items based on their frequencies. Despite its simplicity, it can sometimes be a strong baseline depending on the nature of the data. Often in recommendation, only a few items account for the most of interactions. For example, in Figure 1 we see that only 2 ∼ 3% of items account for 50% of the interactions in our experimental datasets. • Item-KNN. Predictions of the next item are based on item similarities with the current item. Similarity between two items is defined as the number of co-occurrences of the items in a sequence divided by the product of their respective frequencies. Item-KNN was the best baseline reported in [START_REF] Hidasi | Session-based Recommendations with Recurrent Neural Networks[END_REF]. • CoEvent MF. Uses matrix factorization to compute the predictions of the next item given the current one. x t +1 = U T V x t with U and V real matrices of dimensions N I × N E .

Metrics

As presented in Section 3.5, we use the Recall@K metric with a parameter N that controls for the size of the sliding window used to get the list of relevant items for each recommendation. We used different values for N : N = 1 assesses a model ability to predict the next item (short-term recommendation), and N = 20 or N = 5 (depending of the dataset) assesses a model ability to predict items what will eventually be seen in the future (long-term recommendation). We used K = 20, meaning that the top 20 most probable items according to the models are used as predicted items and compared to the set of relevant items in a recall metric.

Datasets

We experimented on two datasets.

Yoochoose.

The first dataset is the publicly available Yoo-Choose dataset, introduced for the RecSys challenge 2015 [START_REF] Ben-Shimon | RecSys Challenge 2015 and the YOO-CHOOSE Dataset[END_REF]. It is a collection of sessions of user clicks and purchases on multiple e-commerce websites over a period of 6 months from April 2014 till September 2014. Each user session forms a separate sequence and users are not identifiable between sessions. We follow the setup proposed in [START_REF] Hidasi | Session-based Recommendations with Recurrent Neural Networks[END_REF]. We use only the clicks of the training set and the element_id feature which is the identifier of the item that has been clicked. We filtered out sessions with only one click. The sessions of the last two weeks are used for the validation set and test set respectively. The rest of the dataset (∼ 6 months) is used for the training set. Items not present in the training set have been removed.

Internal dataset.

The second dataset is a proprietary dataset consisting of browsing activity on multiple of e-commerce websites. It contains of 1 910 177 sequences collected during a period of 3 months. We used a 80/10/10 split to build respectively the training set, the validation set and the testing set. Each user is randomly assigned to one of these sets. Sequences with only one item have been filtered out and we kept only the last 40 items in each sequence. This dataset is referred to as Internal dataset in the rest of the paper. The two datasets differ in a way that one is a dataset of sessions (a user can only be identified across a single session) and the other contains browsing histories of users across multiple sessions, yielding longer sequences.

Table 1 presents statistics of both datasets. Internal dataset is more challenging as it contains about 5 times more distinct items and approximatively the same number of events. We can also see in Table 1 that in terms of item distribution the internal dataset shows a heavier tail. The distribution of sequence length is also different (see Figure 2) with internal dataset containing longer sequences.

Hyper parameters

We train our models using gradient Backpropagation Through Time [START_REF] Mozer | A Focused Backpropagation Algorithm for Temporal Pattern Recognition[END_REF] and the Adam algorithm [START_REF] Lei Ba | Layer Normalization[END_REF] with a decreasing learning rate (LR) following a polynomial decay. The hyper parameters are the same for all models and are presented in Table 2. We use the same number of hidden dimensions N H as in [START_REF] Elena | Contextual Sequence Modeling for Recommendation with Recurrent Neural Networks[END_REF][START_REF] Hidasi | Session-based Recommendations with Recurrent Neural Networks[END_REF]. The batch size is set to 128 for practical reasons (memory constraint).

Results

Tables 4 presents the performance of the benchmarked models and the baselines on both short-term and long-term metrics. In Table 5, we present the break-down of the Recall@20 metric by sequence length. We use the best baseline to compute the uplift.

Long-term metrics (Recall@20,5 and Recall@20,20) assess ability to predict the next N items. In order to better understand the contribution of each future item to the metrics, we computed the Recall@20 for each future time steps using the predictions of the next item. The results are presented in Figure 3.

In the following, we discuss the results and summarize our findings.

4.5.1 Analysis. We observe that the results on both datasets are consistent. The confidence intervals in Tables 4 and5 for the Yoochoose dataset are wider than for the internal dataset. This is due to the smaller test set in Yoochoose dataset: only 1 week of data is used, which account for 173K sessions.

Co-event baseline based on matrix factorization outperforms Item-KNN baseline, that is reported as the best baseline in [START_REF] Hidasi | Session-based Recommendations with Recurrent Neural Networks[END_REF]. On a short-term metric, CoEvent MF also achieved comparable results with a vanilla GRU model. This is particularly notable on small sequences of size up to 5 items (see Table 5) where the uplift of GRU model only represents 0.3% on Yoochoose dataset and 1% on internal dataset. We confirm previous finding of [START_REF] Ludewig | Evaluation of Session-based Recommendation Algorithms[END_REF] and conclude that CoEvent MF baseline is suitable for applications where user sequences are short.

On long-term metric, GRU achieved more significant uplifts. Indeed, we can see in Figure 3 that the uplift provided by the GRU over the CoEvent MF baseline is only of 2.4% for the next item prediction, but it is significantly larger (up to 6.8%) for the prediction of the future items.

We find that the HM-LSTM performs worse than a single layer GRU on both our datasets and metrics. This is not surprising on Yoochoose dataset as the majority of sequences consists of only of 4. Best seen in color.

a few items. However, on Internal dataset we expect this model to discover the hierarchical structure of user sequences and provide relevant recommendations based on high-level user representation. This model is also harder to train since it involves discrete variables and requires to use estimators of the gradient. We believe that this can be the reason why we did not archive satisfactory results with this model. 4.5.2 Layer normalization and tying the input embedding matrix in output module improves performance. We can see in Table 4 that both techniques presented in Section 3.4 improved performance on both datasets and metrics. Our best results were obtained by combining both techniques. The improvement is obtained on the training task (next item prediction) measured by the Recall@20,1 Table 4: Results on short-term and long-term metrics. Uplifts in percents over the best baseline with a 95% confidence interval are in parenthesis. LN denotes the layer normalization. RE means that the input embedding matrix is tied with the output module. N controls the number of future items to be predicted. metric, as well as on a long-term term metric. We hypothesize that these techniques help building a better user representation that results in more accurate predictions for future events (see Figure 3). Layer normalization seems to be particularly useful on long sequences. Indeed, as we can see in Table 5, the uplift on small sequences is only of 3.7% and 7.5% on Yoochoose and Internal datasets respectively, but accounts for 10.6% and 11.5% on long sequences.

Tying the embedding matrix in the input module also allows to greatly reduce the number of parameters (see Table 3). Indeed, the input and output projection layers account for most of the parameters. Dividing by two the number of those parameters results in a more memory efficient training. 4.5.3 Adding a second GRU layer improves performance on long sequences. Adding another layer of GRU slightly improves performance. We note that the longer is the sequence, the bigger the uplift that we observe. On very small sequences (length is less than 5), the effect of adding another layer is negligible and even slightly deteriorate results on Yoochoose dataset. We conclude that stacked RNN architecture is beneficial for long sequences.

CONCLUSION AND FUTURE WORK

In this paper, we study the use of recurrent neural networks for the task of sequential recommendation. We benchmark several models on both short-term and long-term recommendation tasks. We confirm previous findings showing that matrix factorization method provides a strong baseline, especially on datasets of short sequences. Using state-of-the-art techniques (layer normalization, shared input/output matrix) we improve the RNN performances on both short-term and long-term metrics. Best results are obtained by staking another RNN layer with notable improvements on long sequences. As a future work, we plan to study sequence-to-sequence techniques for the next N items prediction task. We also plan to perform experiments on more recommendation datasets.

Figure 1 :

 1 Figure 1: Cumulative proportion of items in experimental datasets. Only 2 ∼3% of distinct items account for 50% of interactions.

Figure 2 :

 2 Figure 2: Sequence length distribution. In Yoochoose dataset, sequences contain up to 200 items, but we cut the long tail for readability. In Internal dataset, we keep only the last 40 items of each sequences, that explains the high number of sequences with 40 items. Best seen in color.

  (a) Recall@20 on future time steps (b) Recall@20 uplift over CoEvent MF baseline on future time steps

Figure 3 :

 3 Figure 3: Recall@20 evaluated on each future time step on Internal dataset. Timestep=0 corresponds to the prediction of the next item measured by Recall@20,1 in Table4. Best seen in color.

  the candidate sequence representation, W r e , W h e and W z e are N H ×N E weight matrices andW r h ,W h h andW z h are N H ×N H weight matrices. σ and tanh denote respectively the sigmoid and the hyperbolic tangent function. ⊙ denotes element-wise product.

Table 1 :

 1 Datasets statistics.

	Dataset	Yoochoose Internal
	Sequences	7 438 177	1 910 177
	Events	31 708 408 36 547 161
	Distinct items	37 483	208 418
	4.3.3 Dataset comparison.

Table 2 :

 2 Hyper parameters.

	Parameter Value
	Start LR	0.01
	End LR	0.001
	Decay power	0.5
	LR steps	50000
	N E	100
	N H	100
	Batch size	128

Table 3 :

 3 Number of parameters.

	Model	Yoochoose Internal
	CoEvent MF	7.5 M	41.6 M
	CoEvent MF + RE	3.75 M	20.8 M
	GRU	7.6 M	41.7 M
	GRU + RE	3.8 M	20.9 M
	GRU + LN	7.6 M	41.7 M
	GRU + RE + LN	3.8 M	20.9 M
	Stacked GRU + RE + LN	3.9 M	21 M
	HM-LSTM + RE + LN	4 M	21.1 M

Table 5 :

 5 Break-down on sequence length of uplift in Recall@20 over the best baseline with a 95% confidence interval. LN denotes the layer normalization. RE means that the input embedding matrix is tied with the output module.

	Session length buckets	[2-5]	[6-25]	[26-200]
	Best baseline Recall@20	0.671	0.641	0.575
	GRU	0.3 ± 1.6% 1.4 ± 1.9%	0.2 ± 7.3 %
	GRU + RE	2.7 ± 1.6% 4.5 ± 2.0%	6.6 ± 7.7 %
	GRU + LN	3.7 ± 1.5% 7.2 ± 1.7%	10.6 ± 9.0 %
	GRU + RE + LN	4 ± 1.5%	7.5 ± 1.9%	11 ± 8.6 %
	Stacked GRU + RE + LN 3.6 ± 1.6% 8.4 ± 1.9% 12.2 ± 8.7 %
	HM-LSTM + RE + LN	3 ± 1.5%	6.2 ± 2.0 %	9.4 ± 8.8%
		(a) Yoochoose
	Sequence length buckets	[2-5]	[6-25]	[26-40]
	Best baseline Recall@20	0.480	0.477	0.460
	GRU	1.0 ± 0.9%	4.2 ± 0.4%	1.3 ± 0.4%
	GRU + RE	6.0 ± 0.9%	7.5 ± 0.4%	4.1 ± 0.5%
	GRU + LN	7.5 ± 0.9%	12.6 ± 0.4%	11.5 ± 0.4%
	GRU + RE + LN	10.0 ± 0.9%	14.3 ± 0.4%	12.8 ± 0.4%
	Stacked GRU + RE + LN	10.4 ± 0.9% 15.5 ± 0.4% 14.3 ± 0.4%
	HM-LSTM + RE + LN	9.4 ± 0.9%	12.6 ± 0.3%	11.1 ± 0.4%
		(b) Internal