Dr Ingolfur Johannessen
email: i.johannessen@ed.ac.uk

Lara Bieleski

Gillian Urquhart

Sarah Watson

Phoebe Wingate

Tanzina Haque

Dorothy Crawford

Wingate P, Haque Watson Sl

Crawford Dh

CELL LYMPHOPROLIFERATIVE DISEASE, T CELL IMMUNOTHERAPY AND SCID MOUSE MODELLING

Keywords: Epstein-Barr virus, B Cell Lymphoma, Immunotherapy, SCID mouse Epstein-Barr Virus, B Cell Lymphoma, Immunotherapy, SCID

ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

INTRODUCTION

Novel therapeutic regimes require validation in suitable animal models to assess efficacy and safety before clinical studies can be undertaken. The severe combined immunodeficient (SCID) mouse is a small animal model that has been used extensively to study novel treatments for human cancer. For example, these humanized animals constitute an in vivo model in which the treatment of Epstein-Barr virus (EBV)associated post-transplant lymphoproliferative disease (PTLD) can be investigated.

EBV is a ubiquitous agent that infects persistently over 90% of adults worldwide [START_REF] Henle | Immunofluorescence in cells derived from Burkitt's lymphoma[END_REF]. It is usually acquired in childhood when infection is generally asymptomatic although if deferred until young adulthood, infectious mononucleosis may result [START_REF] Crawford | A cohort study among university students: identification of risk factors for Epstein-Barr Virus seroconversion and infectious mononucleosis[END_REF]. Following primary infection, EBV establishes a life long persistent infection in resting memory B lymphocytes [START_REF] Babcock | EBV persistence in memory B cells in vivo[END_REF].

EBV immune control is mediated by CD4+ve and CD8+ve MHC-restricted EBV-specific cytotoxic T lymphocytes (CTLs) against latent and lytic viral proteins [START_REF] Bogedain | Specific cytotoxic T lymphocytes recognize the immediate-early transactivator Zta of Epstein-Barr virus[END_REF][START_REF] Steven | Immediate early and early lytic cycle proteins are frequent targets of the Epstein-Barr virus-induced cytotoxic T cell response[END_REF][START_REF] Bickham | EBNA1-specific CD4+ T cells in healthy carriers of Epstein-Barr virus are primarily Th1 in function[END_REF][START_REF] Moss | The immunology of Epstein-Barr virus infection[END_REF] with CTL responses to the EBNA3 antigens being immunodominant in healthy EBV seropositive individuals [START_REF] Murray | Human cytotoxic T-cell responses against Epstein-Barr virus nuclear antigens demonstrated by using recombinant vaccinia viruses[END_REF][START_REF] Murray | Identification of target antigens for the human cytotoxic T cell response to Epstein-Barr virus (EBV): Implications for the immune control of EBV-positive malignancies[END_REF] where up to 5% of circulating CD8+ve CTLs are committed to EBV immune control [START_REF] Hislop | Epitope-specific evolution of human CD8+ T cell responses from primary to persistent phases of Epstein-Barr virus infection[END_REF].

In organ transplant recipients, iatrogenic immunosuppression reduces EBVspecific CTL activity, and this may result in EBV-driven B cell proliferation and, ultimately, PTLD. PTLD develops in up to 10% of organ graft recipients and is (Williams & Crawford, 2006). The major risk factors are the degree and duration of immunosuppression required for graft maintenance and primary EBV infection following transplantation [START_REF] Thomas | Clinicopathologic implications of Epstein-Barr virus related B cell lymphoma in immunocompromised patients[END_REF].

The majority of PTLD express all EBV latent antigens and the tumour is generally susceptible to EBV-specific CTL destruction. Adoptive CTL immunotherapy provides rapid, targeted treatment that is well tolerated and effective in preventing and treating PTLD [START_REF] Rooney | Infusion of cytotoxic T cells for the prevention and treatment of Epstein-Barr virus-induced lymphoma in allogeneic transplant recipients[END_REF][START_REF] Haque | Allogeneic cytotoxic T-cell therapy for EBVpositive posttransplantation lymphoproliferative disease: results of a phase 2 multicenter clinical trial[END_REF]. However, a better understanding of the optimum CTL regime is required to realise its full potential. To this end, our laboratory has used the SCID mouse to model PTLD [START_REF] Johannessen | Essential role for T cells in human B cell lymphoproliferative disease development in SCID mice[END_REF].

Due to an autosomal mutation, SCID mice are unable to produce functional B and T cell clones [START_REF] Bosma | A severe combined immunodeficiency mutation in the mouse[END_REF] and, therefore, the animals cannot reject human xenografts. Following transfer of human EBV+ve B cells, mice develop human PTLDlike tumours (hu-SCID) that reflect patient lesions including expression of all EBV latent antigens [START_REF] Mosier | Transfer of a functional human immune system to mice with severe combined immunodeficiency[END_REF][START_REF] Johannessen | Expansion in SCID mice of Epstein-Barr virus-associated post-transplantation lymphoproliferative disease biopsy material[END_REF]. Various hu-SCID models have been established, and it is not entirely clear which is most suitable for translational treatment studies. Peripheral blood mononuclear cells (PBMCs) from EBV-seropositive healthy donors give rise to PTLD-like lesions in a proportion of cases when cells are transferred intraperitoneally (ip) into SCID mice [START_REF] Picchio | Heterogeneity among Epstein-Barr virus-seropositive donors in the generation of immunoblastic B-cell lymphomas in scid mice receiving human peripheral blood leukocyte grafts[END_REF][START_REF] Johannessen | Essential role for T cells in human B cell lymphoproliferative disease development in SCID mice[END_REF]. However, inter-donor variability exists, and PBMCs from 25% of blood donors do not form tumours in this hu-PBMC-SCID model. In contrast, in vitro EBV-infected B lymphoblastoid cell lines (BLCLs) give rise consistently to PTLD-like lesions in the animals [START_REF] Cannon | Epstein-Barr virus induces aggressive lymphoproliferative disorders of human B cell origin in scid/hu chimeric mice[END_REF][START_REF] Rowe | Epstein-Barr virus (EBV)-associated lymphoproliferative disease in the scid mouse model: Implications for the pathogenesis of EBV-positive lymphomas in man[END_REF] and such hu-BLCL-SCID models are

MATERIALS AND METHODS

Blood Donations

Buffy coats were obtained with ethical approval from the Scottish National Blood Transfusion Service (SNBTS), Edinburgh (UK). EBV serostatus was determined by anti-VCA IgG antibody titer using a standard indirect immunofluorescence assay [START_REF] Henle | Immunofluorescence in cells derived from Burkitt's lymphoma[END_REF]. Donor PBMC MHC tissue typing was carried out at SNBTS (Prof M Turner) or the Anthony Nolan Research Institute (London; Prof A Dodi) using PCR-based methods.

Establishment of BLCLs

Concentrated EBV was added to PBMCs in a total volume of 1 ml of culture medium [CM: RPMI 1640 supplemented with 2mM L-glutamine, 100 U/ml penicillin, 100 U/ml streptomycin, and 10% v/v fetal calf serum (FCS)], incubated at 37˚C for 1 hour, washed (in Hank's Buffered Saline Solution), resuspended (10 6 cells/ml; with ciclosporin), and plated out in a 24-wells plate. During culture in vitro, cultures were fed CM every 2-3 days. At 4 weeks, cells were assessed directly under a light microscope for signs of immortalization, immunophenotyped, and % viability determined using trypan blue. A viability of ≥70% was considered acceptable. T cell subsets were positively selected from CTLs using EasySep ® kits (StemCell Technologies, Grenoble, France) following the manufacturer's instructions. Briefly, 100 µl/ml EasySep ® selection mix was added to 1x10 8 cells/ml in EasySep® buffer, incubated for 15 minutes followed by addition of 50 µl/ml EasySep ® magnetic dextran iron nanoparticles, a further incubation for 10 minutes, and increase of final sample volume to 2.5 mls (EasySep ® buffer). Following 5 minutes on a magnet, supernatant was removed, 2.5 mls of buffer added to the selected cells followed by further magnetic separations (repeated thrice). The final cell fraction was resuspended in CM. The median purity of the CD4+ve and CD8+ve T cell subsets obtained was 53% and 88%, respectively.

Generation of EBV-Specific CTLs

EBV-specific CTLs were established as described previously [START_REF] Wilkie | Establishment and characterization of a bank of cytotoxic T lymphocytes for immunotherapy of Epstein-Barr virus-associated diseases[END_REF]. Briefly, a 40:1 PBMC:γ-irradiated (4,000 rads) BLCL mixture was cultured at 1x10 6 /ml in 20% v/v FCS CM. Ten days later, T cells were restimulated at a ratio of 4:1 T cells:γ-irradiated autologous BLCL followed four days later by addition of 20 IU/ml of recombinant interleukin (IL) 2 (rIL2). From this timepoint (14 days) onwards, T cell cultures were restimulated weekly with γ-irradiated autologous (4:1 ratio) BLCLs together with a thrice weekly dose of rIL2. At time of weekly stimulation, T cell concentration was adjusted to 1x10 6 cells/ml to ensure optimum growth conditions.

Flow Cytometry

Flow cytometry was carried out using standard methods. Briefly, working dilutions of directly-conjugated monoclonal antibodies (mabs) were added to cells suspended in 50 µl of flow buffer (FB: phosphate-buffered saline containing 1% w/v bovine serum albumin, 5mM EDTA and 0.1% w/v sodium azide) and the mixture incubated at 4°C for 20 minutes in the dark. Following FB washes, immunostained cells were resuspended in x1 CellFIX™ (BD Biosciences, Oxford, UK). Flow cytometric acquisition (and analysis of 1,000-10,000 acquired events per sample) was performed using the 'CellQuest Software' on a FACSCalibur machine (Becton Dickinson, Oxford, UK).

Cytotoxicity Assay

A standard 4 hour 51 Cr-release assay was carried out as described previously [START_REF] Wilkie | Establishment and characterization of a bank of cytotoxic T lymphocytes for immunotherapy of Epstein-Barr virus-associated diseases[END_REF]. Briefly, target cells were labelled with 50 µCi of 51 Cr for 1 hour at 37°C, and plated out with effector cells at a 5:1, 10:1 and 20:1 effector-to-target ratio in triplicate wells. Cell mixtures were incubated for 4 hours at 37°C, and 51 Cr release measured in a γ-counter. Spontaneous release was assessed by incubating target cells without effector cells, and maximum release determined by incubating target cells with 1% v/v Triton-X (Sigma, Gillingham, UK). The % specific cell lysis was calculated as follows (results were expressed as an average of readings from triplicate wells): shown as error bars on graphs) was calculated as: SE=SD n-1 /√n (where SD is the standard deviation and n denotes the number of samples analyzed). Similar to survival curves, data presented in Tables 2 and3 represent results obtained for each donor in triplicates (ie, each control and test group consisted of 3 SCID mice). 12

F

RESULTS

The aim was to assess the ip hu-BLCL-SCID model for tumour prevention as a tool to assess safety and efficacy of novel CTL-based immunotherapy. The model was used to investigate the role of T cell subsets and cytokines in PTLD therapeutic regimes.

A panel of 25 BLCLs (and T cell subsets; see below) was generated by in vitro EBV infection of PBMCs from 25 healthy EBV-seropositive blood donors (denoted 'donor 1-25'; for MHC typing results, please refer to Table 1). BLCLs were transferred ip into SCID mice to study tumour prevention.

Establishment of Ip Hu-BLCL-SCID Model: Tumour Prevention

In this model, tumour prevention is used as experimental read-out. Firstly, the minimum cell number required for consistent BLCL-driven ip tumour formation in vivo was assessed by inoculating groups of 3 animals ip with 1x10 6 , 2x10 6 or 4x10 6 BLCL cells from our BLCL panel. All mice inoculated ip with either 2x10 6 or 4x10 6 BLCL cells developed macroscopic ip tumours in a median time of 6.5 and 6 weeks, respectively, whereas an inoculum of 1x10 6 BLCL cells produced ip tumours in 40% of animals in a median time of 6 weeks. Therefore, in this model, ip tumours were induced in all further experiments with a standard 2x10 6 BLCL dose. In parallel control experiments, PBMCs from 3 EBV-seronegative healthy donors did not give rise to ip tumours. Previously, the effect of depleting endogenous (murine) NK cells (using rabbit anti-mouse-ASGM1 antiserum) on ip tumour development in SCID mice has been investigated and found not to impact significantly on tumour formation in vivo 13 [START_REF] Johannessen | Essential role for T cells in human B cell lymphoproliferative disease development in SCID mice[END_REF]. Therefore, we do not administer routinely an anti-NK antiserum to our animals.

Analysis Of Tissues And Tumours

Paraffin wax-embedded tissue sections from lung, liver, spleen and tumour tissue from a panel of 10 ip mice inoculated with BLCL from different donors (donors 1-10;

see Table 1) were screened using H&E staining, anti-human CD45 and CD20

immunostaining, in situ hybridization for EBERs, and immunostaining for EBV latent (EBNA2, LMP1) and immediate early (IE) lytic (BZLF1) antigens (see Figure 1).

Macroscopic ip tumours formed primarily as solitary nodules at the undersurface of the EBER+ve cells in tumour tissue were found to express the latent EBNA2 and LMP1

antigens with the occasional cell expressing the IE lytic BZLF1 lytic antigen.

Characterisation Of In Vitro Expanded T Lymphocytes

EBV-specific T cells were expanded from PBMCs from a panel of 25 EBV seropositive donors (donors 1-25; see andK562. Median specific cell lysis at a 10:1 effector:target ratio were 36% for the autologous BLCL, 6% for the allogeneic BLCL, and 7% for K562. The differences between median % specific lysis for autologous versus allogeneic targets and the NK cell target K562 were significant (Mann-Whitney test: p<0.0001 on both occasions).

Model For Tumour Prevention

Using 10 donors (donors 1-10; see indicating that tumour prevention is MHC restricted.

Role of CD8+ve T Cell Subsets

Using the same 10 donor CTLs (donors 1-10; see above), we enriched for CD8+ve T cells using magnetic beads. For each donor, 2x10 6 BLCL were inoculated ip into each mouse in groups of 3 SCID mice followed 1 hour later by inoculation ip of CD8+ve enriched T cell populations. Thus, each of 3 mice received ip a median of 37x10 6 (range: 3x10 6 -50x10 6 ) CD8+ve enriched T cells. Animals were culled when showing signs of sickness (see Materials and Methods), or at a pre-determined time limit of 100 days. On each occasion, a sick mouse was found to harbour ip macroscopic tumour at necropsy. The percentages of mice in each group that developed ip tumours are shown in between donors, we re-analysed the results using data obtained when using CD4enriched cells from the 5 donors whose CD4-enriched populations contained the highest level of CD4+ve cells (a median of 74%; range: 70-97%; see Figure 2D). The transfer of CD4-enriched cells from those individuals did not impact significantly on tumour development in vivo (Log-rank test: p=0.35) which is in line with the overall results described above.

The in vitro cell seperation manipulation per se did not affect significantly CTL performance in vivo as determined in parallel control studies. For each of 3 donors, 2x10 6 BLCLs were inoculated ip into each mouse in 2 groups of 3 SCID mice each.

Whilst 50x10 6 unfractionated autologous CTLs were inoculated ip into each animal in the control group, each mouse in a test group received 50x10 6 CTLs that had been In the present study, the CD4-and CD8-enriched T cell populations were not pure populations so calculations of the total number of these cells inoculated in each experiment were carried out. A non-correlation was found between the numbers of CD4+ve, or CD8+ve, T cells (included in the inoculated CTL, CD4-enriched, or CD8enriched T cells) and the average % survival (in days) beyond controls (Spearman's test:

p>0.05 for all analyzes). Similar results were obtained when analysing the CD8:CD4 ratio of cells inoculated (Spearman's test: p>0.05 for all analyzes). Therefore, it does not appear that survival of SCID mice treated with autologous CTLs, or T cell subsets, is a direct function of the number of transferred CD4+ve or CD8+ve T cells or their ratio.

The level of CTL cytotoxicity against the autologous BLCL (as assayed in vitro by 51 Crrelease studies) did not correlate with survival of CTL-inoculated SCID mice (Spearman's test: p=0.17; Data not shown) and was therefore not a useful parameter for predicting survival.

In Vivo Trafficking of Human T Cells

We analysed samples of lung, liver, spleen, and tumour harvested from the CTLtreated hu-BLCL-SCID mice (see above) to look for human T cell trafficking within these tissues. The percentage of specific antibody+ve cells on immunostained sections was assessed by attributing a score of (-), (+), 1+, 2+ and 3+, indicating that none, <1%, 1-29%, 30-60%, or >60% of all cells counted were antibody+ve cells, respectively.

Tissues were obtained at necropsy from 32 ip [harvested at 17-73 (median: 32) days after ip CTL inoculation] hu-BLCL-SCID mice (see Figure 3). Human CD3+ve T cells were found in 17 out of 32 (53%) tumour samples tested. Whilst 2 of these 17 (12%)

samples contained CD4+ve T cells, 11 (65%) had detectable CD8+ve T cells and 1 (6%) contained CD4+ve and CD8+ve T cells. Three (18%) samples contained only CD3+ve cells. Overall, T cells comprised <1-29% of each T marker-positive tumour section.

Human cells were not detected in control mice. The data suggest that human T cells are able to home to human tumour tissue in vivo. Since human CD45+ve cells were detected only very rarely in lung, liver and spleen tissue, these samples were not analysed further.

Immunostaining of the murine pan-nucleated cell marker MHC1 on tumour sections showed no (or the very occasional) murine cell underlining further that the lesions were human in origin (Data not shown).

Serial sections from the 17 ip tumours shown to contain CD3+ve T cells (see above) were stained for perforin and granzyme B cytoloytic molecules (see Figure 3).

Six out of 17 (35%) tumours contained granzyme B+ve cells and a further 8 out 17 (47%) contained perforin+ve cells. Overall, 14 out of 17 (82%) tumour samples expressed one or other cytoxic granule molecule with no dual expression noted.

Cytokine Conditioning

Cytokines that signal through the common γ-chain (IL2, 4, 7, 15 and 21) are important for the homeostasis of CD4+ve and CD8+ve T cells with the ability to enhance their homing and promote their expansion, function and survival in vivo ( T cells [START_REF] Wagar | Regulation of human cell engraftment and development of EBV-related lymphoproliferative disorders in hu-PBL-SCID mice[END_REF] suggesting at least a supporting role to CD8+ve T cells for CD4+ve lymphocytes. Since the CD4-enriched T cell populations transferred into our animals consisted of smaller total T cell numbers than the CD8-enriched inocula, and there was no significant difference in the rate of ip tumour development between mice inoculated with bulk CTLs and animals that received CD4-enriched cells only, the latter may still have an anti-tumour effect or work to prevent CD8+ve T cell-mediated antitumour mechanisms (see Results). Thus, the data must be interpreted with a certain degree of caution.

CD4+ve T cells show cytotoxic function in vitro and in vivo [START_REF] Appay | The physiological role of cytotoxic CD4+ T-cells: the holy grail?[END_REF], and since PTLD cells express MHC2 molecules [START_REF] Thomas | Immunohistology of Epstein-Barr virus-associated antigens in B cell disorders from immunocompromised individuals[END_REF], direct CD4+ve T cellmediated cytotoxicity is possible. However, CD4+ve T cells also have a variety of helper functions and in a recent clinical trial using allogeneic 'best MHC match' CTL treatment for PTLD, a significant trend for a better outcome with higher numbers of CD4+ve T cells in infused CTL lines was interpreted as the effect of the few CD4+ve T cells in the CTLs in enhancing the survival in vivo of the CD8+ve population [START_REF] Haque | Allogeneic cytotoxic T-cell therapy for EBVpositive posttransplantation lymphoproliferative disease: results of a phase 2 multicenter clinical trial[END_REF]. Human T cells were detected in T cell treated animals for up to 64 days which is in line with observations by others underlining that T cells can home to PTLD-like tumours in SCID mice. Thus, [START_REF] Lacerda | Human Epstein-Barr virus (EBV)-specific cytotoxic T lymphocytes home preferentially to and induce selective regressions of autologous EBV-induced B cell lymphoproliferations in xenografted C.B-17 SCID/SCID mice[END_REF] showed preferential homing of CD8+ve T cells to autologous (and not MHC mismatched) tumour tissue as did imaging studies by [START_REF] Koehne | Serial in vivo imaging of the targeted migration of human HSV-TKtransduced antigen-specific lymphocytes[END_REF] using radioactive labelling of human EBV-specific T cells. Whilst our immunostaining results support a role for the cytotoxic molecules perforin and granzyme B, the data must be interpreted with a certain degree of caution.

A combination of IL7 and 15 enhanced the ability of CTLs from across the donor panel to mediate tumour prevention in vivo although individually IL7, 15, and 21 did not (Figure 4). Furthermore, a combination of IL7 and 21 did not have an impact in vivo. It is postulated that a combination of IL7 and 15 improved the ability of the in vitro conditioned T cells to home to, and mediate destruction of, human PTLD-like tumour cells in vivo which is in line with recent in vivo data using IL15 [START_REF] Klebanoff | IL-15 enhances the in vivo antitumor activity of tumor-reactive CD8+ T cells[END_REF].

However, since a correlation between the effects of the cytokines with expression of their relevant receptors by CTLs at the time of in vitro cytokine conditioning was not Previous studies have given conflicting results when examining a possible role of cytokine administration in sustaining transferred T cells in vivo [START_REF] Boyle | Adoptive transfer of cytotoxic T lymphocytes for the treatment of transplant-associated lymphoma[END_REF][START_REF] Rencher | Activity of transplanted CD8+ versus CD4+ cytotoxic T cells against Epstein-Barr virus-immortalized B cell tumors in SCID mice[END_REF][START_REF] Baiocchi | Low-dose interleukin 2 prevents the development of Epstein-Barr virus (EBV)-associated lymphoproliferative disease in scid/scid mice reconstituted i.p. with EBV-seropositive human peripheral blood lymphocytes[END_REF][START_REF] Baiocchi | GM-CSF and IL-2 induce specific cellular immunity and provide protection against Epstein-Barr virus lymphoproliferative disorder[END_REF]. However, in line with data from other laboratories [START_REF] Baiocchi | Low-dose interleukin 2 prevents the development of Epstein-Barr virus (EBV)-associated lymphoproliferative disease in scid/scid mice reconstituted i.p. with EBV-seropositive human peripheral blood lymphocytes[END_REF][START_REF] Baiocchi | GM-CSF and IL-2 induce specific cellular immunity and provide protection against Epstein-Barr virus lymphoproliferative disorder[END_REF], the current study underlines a supporting role for IL2 in facilitating CTL survival and function in vivo (see Figure 4F and Table 3) although the results do not exclude possible contribution from IL2-mediated activation of murine NK cells. Whilst the results suggest inclusion of IL2 in immunotherapeutic regimes using CTLs against PTLD, a degree of caution must be exercised in the use of the cyotkine in a patient population due to its ability to cause side effects.

Our laboratory has successfully infused PTLD patients with EBV-specific CTLs on a 'best MHC match' basis [START_REF] Haque | Treatment of Epstein-Barr-virus-positive posttransplantation lymphoproliferative disease with partly MHC-matched allogeneic cytotoxic T cells[END_REF][START_REF] Haque | Allogeneic cytotoxic T-cell therapy for EBVpositive posttransplantation lymphoproliferative disease: results of a phase 2 multicenter clinical trial[END_REF]. Although this proof-ofprinciple trial showed that CTL therapy could succeed, a mechanism for bypassing MHC restriction would facilitate its dissemination to the clinic. A chimeric T cell receptor (cTCR) can be constructed that directs T cells against a novel surface antigen in a non-MHC restricted manner [START_REF] Schumacher | T-cell-receptor gene therapy[END_REF][START_REF] Mansoor | Engineering T cells for cancer therapy[END_REF], and currently our laboratory employs such an approach to re-target CTLs to EBV-associated cancers.

This novel therapeutic approach will be tested in the pre-clinical ip hu-BLCL-SCID model presented in the current study. pan-B cell marker CD20 on tumour sections using a peroxidase (HRP) label. Figure 1G shows CD20+ve human cells (brown membrane staining) counterstained with Mayer's haemalum. Figure 1H shows the conjugate control with counterstained cells only.

Figures 1I,J

: Photomicrographs of in situ hybridization for EBERs on tumour sections using an alkaline phosphatase (AP) label. Figure 1I shows EBER+ve cells (dark blue/black nuclear staining) on a probed tumour section which was counterstained with Mayer's haemalum (blue nuclear staining). Figure 1J shows the counterstained unprobed (negative) control tumour section.

The panels show a single mouse experiment but are representative of several experiments.

  onset, aggressive behaviour, and approximately 50% mortality despite current treatment

  liver. (1) H&E: Sections were scored for distortion of normal tissue morphology and cellular infiltrations. Large immunoblastic tumour cells were detected consistently in ip tumour tissue only. (2) Human CD45 and CD20: Scattered human CD45+ve, CD20+ve cells were found only rarely in lung, liver and spleen. Conversely, such cells formed ip tumour tissue. (3) EBERs: Scattered EBER+ve cells were found only rarely in lung, liver and spleen. Conversely, such cells formed ip tumour tissue. (4) EBV Antigens:

  and CD8+ve T cell subpopulations using magnetic beads and recombined in the original CTL subset proportions. Unfractionated CTLs and fractionated/recombined CTLs had a similar effect in vivo (Log-rank test: p=0.71; Data not shown).

  animal modelling has not diminished over the years despite efforts to offer alternative in vitro options, and pre-clinical data obtained in suitable animal models are still vital to guide translation into the clinical setting. Whilst SCID mice are a small animal model that is widely used to study human cancer (including PTLD), there is as yet no international standard defining how the model should be used to ensure high quality data. In the current study, we used an ip hu-BLCL-SCID mouse tumour prevention model because of its suitability for testing of novel immunotherapy against PTLD.[START_REF] Boyle | Adoptive transfer of cytotoxic T lymphocytes for the treatment of transplant-associated lymphoma[END_REF] engrafted SCID mice ip with BLCL and showed that autologous CTL administered simultaneously (or 7 days later) delayed significantly tumour formation whereas mismatched CTLs failed to do so, indicating the MHC restriction of CTLs. In the present study, administration of autologous, but not allogeneic, CTLs in the ip hu-BLCL-SCID model similarly delayed tumour development and improved survival time, and further support for this tumour prevention model comes from studies by[START_REF] Dimaio | Generation of tumour-specific T lymphocytes for the treatment of posttransplant lymphoma[END_REF] and[START_REF] Buchsbaum | EBV-specific cytotoxic T lymphocytes protect against human EBV-associated lymphoma in scid mice[END_REF].The current study demonstrates that ip hu-BLCL-SCID mice give rise reliably to macroscopic EBV+ve human B immunoblastic ip tumours portraying full (unrestricted) latent virus gene expression with the occasional tumour cell in lytic cycle. In this prevention model, ip tumour formation is the experimental read-out and autologous CTLs can be detected in tumour tissue. In parallel studies, we have also investigated the ) hu-BLCL-SCID model in which transfer of 2x10 6 BLCL sc into flanks of SCID mice gives rise consistently to visible sc PTLD-like tumours that can be measured directly. Theoretically, such a model can be used to assess the efficacy of novel PTLD immunotherapy when treatment is delivered by intravenous (iv) inoculation. However, such a tumour regression model poses technical challenges that undermine its use. In particular, in the sc model, sc tumours may arise at different timepoints after inoculation making the experimental protocol for T cell therapy difficult to plan. Furthermore, the transfer of large T cell inocula iv can be problematic and it is difficult to ensure successful injection of the large T cell numbers required for analysis.The ip hu-BLCL-SCID model used herein circumvents these difficulties and is technically straight-forward to manage. However, since ip tumour development can not be monitored clinically in this model, it is a prevention model with BLCLs and CTLs given 1 hour apart, a situation that does not reflect accurately the clinical setting. Whilst we have introduced MRI imaging of our animals to mimic better the clinical (patient) situation with a view to provide a non-invasive, non-destructive method of monitoring ip tumour development in longitudinal studies on the same animals, this approach has yet to prove its value in face of technical challenges in demarcation of small lymphoid lesions by MRI scanning of SCID mice. Taken together, translational data can be obtained in the ip prevention SCID mouse model that is difficult to generate in other SCID mouse models and such data can be used to inform clinical (patient) protocols.Using the ip prevention hu-BLCL-SCID model, CD8+ve T cell subpopulations from CTLs from across the donor panel delayed significantly tumour development, and tumour formation was entirely prevented in 40% of mice (see Table2). Boyle et al also used CD8-enriched CTL populations in this model with significant effect on tumour development. Whilst inoculation of CD4-enriched T cells did not have a significant impact on survival in this model (see Figure2), the data is too limited to draw conclusions from this set of experiments. Interestingly, in the hu-PBMC-NOD/SCID model (NOD/SCID animals are deficient in NK as well as B and T cells), treatment of mice with a depleting anti-human CD4 mab results in loss of both CD4+ve and CD8+ve

  the murine and human situation. Furthermore, in the ip hu-BLCL-SCID model, the most effective combination of CD4+ve and CD8+ve T cells has not yet been identified. Interestingly, studies in our laboratory have shown that PBMC-inoculated mice (hu-PBMC-SCID) deplete of T regulatory (Treg) cells develop PTLD-like tumours at a similar rate to Treg-replete PBMC populations which conflicts with current ideas that Treg cells play a major role in tumour immune escape (Data not shown). These results highlight further the discrepancies that exist between the murine and human situation.

  FIGURE 1 Immunophenotyping Of SCID Mouse Tumours: Human Leukocyte Markers And EBV Transcripts(Paraffin Wax-Embedded Sections)

(

  FIGURE 2 % Survival Using CTLs And T Cell Subsets(Datapoints reflect results in triplicates; Standard error, SE, is shown on each column as a vertical bar.)

Figure

  Figure 3A shows CD45-positive human cells (red membrane staining) counterstained with Mayer's haemalum. Figure 3B shows the conjugate control with counterstained cells only. Figures 3C,D (x400): Photomicrographs of immunostaining of the human pan-T cell CD3 marker on tumour sections using an alkaline phosphatase (AP) label.

Figure

  Figure 3C shows CD3-positive human cells (red membrane staining) counterstained with Mayer's haemalum. Figure 3D shows the conjugate control with counterstained cells only. Figures 3E,F (x400): Photomicrographs of immunostaining of the human T helper cell CD4 marker on tumour sections using an alkaline phosphatase (AP) label.

Figure

  Figure3Eshows CD4-positive human cells (red membrane staining) counterstained with Mayer's haemalum. Figure3Fshows the conjugate control with counterstained cells only. Figures 3G,H (x400): Photomicrographs of immunostaining of the human T cytotoxic CD8 marker on tumour sections using an alkaline phosphatase (AP) label.

Figure

  Figure3Gshows CD8-positive human cells (red membrane staining) counterstained with Mayer's haemalum. Figure3Hshows the conjugate control with counterstained cells only. Figures 3I,J (x400): Photomicrographs of immunostaining of the cytolytic granule molecule perforin on tumour sections using a peroxidase (HRP) label. Figure3Ishows perforin-positive human cells (brown cytoplasmic staining) counterstained with Mayer's haemalum. Figure3Jshows the conjugate control with counterstained cells only. Arrows point to examples of specific antibody+ve cells.

(

  FIGURE 4 % Survival Using Cytokine Conditioned CTLs (Datapoints reflect results in triplicates; Standard error, SE, is shown on each column as a vertical bar.) 4A) IL2 vs IL7 Conditioning 4B) IL2 vs IL15 Conditioning 5 Donors

EBER In Situ Hybridization (ISH)

	% Specific lysis = complimentary to EBER1 and 2 was applied at 55˚C for 1.5 hours, sections immersed in Test release -Spontaneous release x100 was used for data analysis and p≤0.05 was considered significant. Datapoints on survival
	Maximum release -Spontaneous release a 55˚C working solution (1:60 in pure water) of 'Stringent Wash' for 25 minutes, and curves in Figures 2 and 4 reflect donor-derived results obtained as triplicates since
	incubated with 'Anti-FITC/AP' for 30 minutes. Bound probe was visualized by an AP groups of 3 SCID mice were used for each test and control group. Standard Error (SE;
	Histopathology and Immunohistochemistry 'Substrate' (BCIP/NBT) followed by counterstaining in Mayer's Haemalum, mounting
	Paraffin wax-embedded tissues were dewaxed and rehydrated. Haematoxylin and and coverslipping.
	eosin (H&E) staining was carried out using standard methods. Immunostaining included
	an initial antigen retrieval step (using 'Target Retrieval Solution' or 'Antigen Retrieval SCID Mice
	o Solution'; DakoCytomation, Ely, UK). Sections were immunostained with primary mabs (in 'Antibody Diluent, Background Reducing'; DakoCytomation, Ely, UK) for 30 F o CB.17 scid/scid mice were bred at the University of Edinburgh under specific pathogen free conditions in individually ventilated microisolator cages and handled in F o
	r minutes at room temperature, washed in tris-buffered saline (TBS), and 'EnVision' r microbiological class 2 safety cabinets. All animal work was carried out under relevant r
	P (DakoCytomation, Ely, UK) and 'NovoLink Polymer Detection System' (Novocastra, P Home Office Project and Personal Licences in accordance with the Home Office P
	e e r Newcastle Upon Tyne, UK) kits used to visualize bound antibody following manufacturers' instructions. Sections were counterstained in Mayer's Haemalum, e e r 'Animals (Scientific Procedures) Act 1986', and monitored by Home Office Inspectors and Named University Veterinary Surgeons. Mice were observed twice daily and culled e e r
	R e v i e w dehydrated (if appropriate), mounted and coverslipped. For quantification, numbers of mab-labelled and -unlabelled cells were counted in 5 high power (x1,000 under oil) fields and results expressed as % mab+ve cells. R e v i e w if sick (defined as disinterest in their surroundings, hunched back, respiratory distress and/or ruffled fur), or after 100 days. Tissues were harvested at necropsy and fixed in neutral buffered formalin (NBF). Statistical Analysis R e v i e w
	'Epstein-Barr Virus (EBER) PNA Probe/Fluorescein' and 'PNA ISH Detection Advice on statistical analysis and presentation of data was sought from public
	Kit' (DakoCytomation, Ely, UK) kits were used in accordance with the manufacturer's health departments of the London School of Hygiene & Tropical Medicine and the
	instructions. Briefly, 0.1 mg/ml proteinase-K was applied to dewaxed and rehydrated University of Edinburgh. Tumour incidence was analysed by the Fisher's exact test (2-
	paraffin wax-embedded sections for 30 minutes. A FITC-labelled PNA probe tailed p value). Time to tumour was assessed by the Kruskall, Mann-Whitney and
	Spearman's tests. 'GraphPad Prism' software (GraphPad Software Inc, La Jolla, USA)
	9 10 11

Table 1

 1

	expressing the following markers were delineated by flow cytometry and the medians
	calculated: CD4+ve median 6%, CD8+ve median 84%, CD56+ve median 1%, CD19+ve
	<1%. In parallel, cytotoxic function was assessed by a 4 hour standard 51 Cr-release
	assay with target cells including autologous and allogeneic (MHC mismatched) BLCLs
	F o
	r
	P
	e e r
	R e v i e w
	14

) by culture in vitro. Cells were harvested after 7-17 weeks (median: 10 weeks) of culture at which time the percentage of cells

Table 2 .

 2 Individual results (together with Log-rank test results) are shown

	in Figure 2A-C. Tumours developed in 18 out of 30 (60%) mice injected with CD8-
	enriched T cells. Comparing the results with control mice inoculated with vehicle only
	(see Table 2 and Figure 2), significantly fewer animals injected with CD8-enriched T
	cells developed ip tumours (Fisher's exact test: 0.001). Furthermore, CD8-inoculated
	mice survived significantly longer than controls (Log-rank test: p=0.0002). Whilst
	groups of mice were also inoculated with a median of 7x10 6 (range: 3x10 6 -67x10 6 )

enriched T cells did not impact significantly on tumour development in vivo (Log-rank test: p=0.101). Since the proportion of CD4+ve cells in the transferred inoculum differed

  the results should be interpreted with a degree of caution. Whilst in vitro cytokine conditioning did not affect significantly CTL cytotoxicity in vitro, additional results from this study (as well as data from clinical studies in our laboratory;[START_REF] Haque | Allogeneic cytotoxic T-cell therapy for EBVpositive posttransplantation lymphoproliferative disease: results of a phase 2 multicenter clinical trial[END_REF] suggest that in vitro cytotoxicity does not correlate with CTL function in vivo.

	carried out,
	F o
	r
	P
	e e r
	R e v i e w
	25

TABLE 1 MHC Profile of 25 BLCL and T Cell Sample Sets

 1

	Donor	A		B		C		DR
	No	Allele 1 Allele 2 Allele 1 Allele 2 Allele 1 Allele 2 Allele 1 Allele 2
	1	03	31	27	45	01	06	NT	NT
	2	24	68	14	15	03	08	NT	NT
	3	23	23	44	49	04	07	NT	NT
	4	02	29	44	44	NT	NT	01	07
	5	02	32	07	14	NT	NT	07	15
	6	01	31	08	44	NT	NT	07	17
	7	01	03	07	14	07	08	NT	NT
	8	01	02	35	73	04	15	NT	NT
	9	03	68	40	44	03	08	NT	NT
	10	01	29	07	08	07	07	NT	NT
	11	01	03	07	07	NT	NT	04	15
	12	02	68	50	60	NT	NT	04	17
	13	02	11	27	44	NT	NT	01	14
	14	03	03	51	60	NT	NT	04	15
	15	11	32	27	62	NT	NT	01	11
	16	02	32	44	44	05	16	04	07
	17	02	02	08	44	05	07	03	13
	18	01	02	08	44	05	07	04	15
	19	02	31	35	51	NT	NT	07	17
	20	02	03	40	44	03	05	NT	NT
	21	23	25	44	45	NT	NT	04	12
	22	01	32	08	35	NT	NT	01	17
	23	02	02	37	44	NT	NT	01	07
	24	01	02	07	57	NT	NT	07	09
	25	01	02	08	57	NT	NT	01	07
	BLCL: B lymphoblastoid cell line; MHC: Major histocompatibility complex; No: Number; NT: Not tested;

TABLE 2 Ip Tumour Development

 2 Statistically significant difference (p=0.001) when compared with mice inoculated with medium only; CTL: Cytotoxic T lymphocytes; Ip: Intraperitoneal; Data reflects results obtained for each donor in triplicates (ie, groups of 3 SCID mice). Statistically significant difference (p=0.02) when compared with mice inoculated with medium or CTL only; CTL: Cytotoxic T lymphocytes; IL: Interleukin; Ip: Intraperitoneal; Data reflects results obtained for each donor in triplicates (ie, groups of 3 SCID mice).

	Inoculum	Medium Only	CTL	CD4+	CD8+
	Number of
	Tumours/	29/30	26/30	25/30	18/30*
	Number of Mice	(97)	(87)	(83)	(60)
	Injected
	(%)

*:

*:

John Wiley & SonsJournal of Medical Virology

ACKNOWLEDGEMENTS

We are indebted to the staff of the Biomedical Research Resources (The University of Edinburgh) for maintaining the SCID mice used in this project. We also gratefully acknowledge the technical assistance of Dr KA McAulay, Mr A Timpson and Ms N Blair. IJ was supported by a Fellowship in Medical Microbiology from the Wellcome Trust (London, UK). The study was funded by a project grant (no CZB/4/126) from the Chief Scientist Office of the Department of Health of the Scottish Executive (Edinburgh, UK).

John Wiley & Sons Journal of Medical Virology John Wiley & Sons Journal of Medical Virology John Wiley & Sons Journal of Medical Virology John Wiley & Sons Journal of Medical Virology John Wiley & Sons Journal of Medical Virology John Wiley & Sons Journal of Medical Virology John Wiley & Sons Journal of Medical Virology John Wiley & Sons Journal of Medical Virology John Wiley & Sons Journal of Medical Virology John Wiley & Sons Journal of Medical Virology John Wiley & Sons Journal of Medical Virology John Wiley & Sons Journal of Medical Virology John Wiley & Sons Journal of Medical Virology John Wiley & Sons Journal of Medical Virology John Wiley & Sons Journal of Medical Virology John Wiley & Sons Journal of Medical Virology John Wiley & Sons Journal of Medical Virology John Wiley & Sons Journal of Medical Virology John Wiley & Sons Journal of Medical Virology John Wiley & Sons Journal of Medical Virology John Wiley & Sons Journal of Medical Virology


Iliya Tsekov

Dilyan Ferdinandov

Ventzeslav Bussarsky

Svetlana Hristova

Zlatko Kalvatchev
email: kalvatchev@gmail.com

Prof. Zlatko Kalvatchev

Prevalence of JC polyomavirus genomic sequences from the large T-antigen and 1 non-coding control regions among Bulgarian patients with primary brain tumours 2 3

Keywords: brain tumours, JC virus, oncogenesis, progressive multifocal lekoencephalopathy brain tumours, JC virus, oncogenesis, progressive multifocal 21 leukoencephalopathy

Fig. 1. Real-time SYBR Green polymerase chain reaction assay A. An amplification plot showing positive control graphs with a threshold at 25th cycle and several positive samples with a threshold in the range from cycle 36 to 44. B. Melting curve analysis showing a characteristic profile of the positive control at 75.7C identical to the profile of the positive samples 117x118mm (96 x 96 DPI)

The infection remains latent but under certain conditions a viral reactivation is possible.

21

The production of mature JC viral particles, followed by the death of the infected cells JCV possess oncogenic potential, but still their role concerning the induction of 7 primary brain neoplasia is unclear and findings reported in the literature vary widely.

8

Investigations on the effect of JCV cultured in vitro and on laboratory animals rise There is evidence that JCV meets at least the last three criteria presented [Khalili et given and all specimens were collected after a signed informed consent. profile with a maximal decrease in the intensity of the signal (-dI/dT) at 75.7 0 C (Fig. 1 1A, Fig. 1B). JCV sequences among the study groups is summarised in details on Table I as LT positive. JCV sequences were also found in a brain biopsy from a patient who 5 underwent surgery because of an arterio-venous malformation (Table I).

6

All positive samples had an increase in the fluorescent signal during the late 7 cycles of the PCR, with a threshold cycle that varied from 36 to 50 (Table I).

8

The subsequent TaqMan ® quantitative assay showed that most of the samples 9 did not yield a NCCR product, although the detection limit of the system was 10 10 starting copies. Only 21 of all the 111 samples were documented as NCCR positive 11 (Table I). Low copy number in the range of 10 to 1000 was present. Notably, only eight 12 of all NCCR positive specimens were also LT positive (Table II). Studies that fail to establish a positive association of JCV with human tumours often 6 isolate DNA from formalin-fixed or paraffin-embedded tumours, where nucleic acids 7 are usually of inferior quality [START_REF] Ferrer | Effects of formalin fixation, paraffin embedding, and time of 16 storage on DNA preservation in brain tissue: A BrainNet Europe study[END_REF]. Therefore, the use of fresh or fresh- expected that these would be JCV negative, as it was the case with five of them.

5 However, one sample was positive for LT. The logic behind this remains unclear so far.

6

Two of the patients had alterations that were induced by the radiotherapy 7 following the surgical resection for fibrilar astrocytoma and glioblastoma. Their

  patients according to their histopathological diagnosis, gender, age and prevalence of viral sequences Large T antigen encoding genomic sequences detected by SYBR Green real-time polymerase chain reaction; **Ct: threshold cycle determined during the SYBR Green real-time polymerase chain reaction; ***NCCR: DNA sequences from the non-coding control region determined by quantitative assay;

20

 20

22 can

 22 lead to progressive multifocal leucoencephalopathy[START_REF] Imperiale | Polyomaviruses[END_REF]. If 23 the infected cells die, any possible disregulation has no consequence for cellular 24 , when the viral replication is in some way impaired and 1 associated with the survival of the affected cell, the events causing disturbances in 2 regulation contribute to the process of cellular transformation[START_REF] Nevins | Cell Transformation by Viruses[END_REF]. JC virus 3 has a marked oncogenic potential when cultured in vitro. The development of tumours 4 in inoculated or transgenic experimental animals has also been demonstrated [Miller et 5 al., 1984].

6

 6

9 2 6 2.

 26 evidence associating this virus with human tumours and offer possible mechanisms of 10 JCV-mediated cell transformation [Del Valle et al., 2008]. In contrast, some researchers 11 fail to identify genomic sequences in human tumours [Herbarth et al., 1998; Rollison et 12 al., 2005]. Although a possible "hit-and-run" mechanism is discussed, they put under 13 question an association of these viruses with malignant transformation. There are 14 obstacles that hamper the establishment of a causal role between JCV and human 15 neoplasia. For instance, the discrepancy between the high prevalence of HPoV in nature 16 and the relatively low frequency of JCV-associated cancer has been raised as an 17 argument against a direct association between the two phenomena. Most often the 18 primary JCV infection is subclinical, making it difficult to establish the exact moment 19 and mechanism of infection. The incubation period between the onset of the infection 20 and the appearance of the tumours might be extended to several decades. In addition, a 21 productive JCV infection cannot be reproduced in laboratory animals, limiting 22 investigations aimed to give answers to the above questions. Environmental (e.g., co-23 carcinogens) or host factors (e.g., immune system status) can additionally modify the 24 making it even more difficult for experimental research [Khalili et 1 al., 2006]. Criteria about the etiologic role of an infectious agent and a neoplastic disease 3 have been proposed: [zur Hausen, 2006] 4 1. Epidemiological plausibility and evidence that the viral infection is a risk factor 5 for the development of a certain tumours type; Regular identification and persistence in the cells of a particular tumours of the 7 respective infectious agent's nucleic acids; 8 3. Cell proliferation stimulation following transfection with given genomes or 9 sequences in the corresponding tissues and cell types;10 4. Demonstration that the proliferative induction and the malignant phenotype of a 11 certain tumours cells depend on functions executed by the persisting nucleic 12 acids of the respective infectious agent.

13

 13

20 DNA

 20 procedure biopsy samples were placed in a sterile 1.5 ml 13 tubes containing 0.6 ml of transport medium (Cat. No. K-1-1/A, Sacace, Como, Italy) 14 and were immediately frozen. Specimens were kept at -20 о С for a maximum of 48 15 hours prior to transportation to the virology laboratory. The collected samples were 16 washed three times with a sterile phosphate buffered saline and were stored at -80 о С 17 until an extraction procedure was carried out. was obtained from 3-7 mg of tumours tissue subjected to extraction with 21 AquaPure DNA Kit (Cat. No. 732-6340EDU, Bio-Rad, Hercules, CA, USA) according 22 to the manufacturer's instructions. An internal extraction control (Cat. No. Int-DNA, 23 Primer Design, Southampton, UK) was added to each sample in order to verify the 24 DNA during the quantitative real-time assay. In addition at the 1 end of the extraction the quality and quantity of DNA was determined with a 2 spectrophotometer and agarose gel-electrophoresis.

5

  All samples were analyzed for the presence of viral sequences that are part of the gene 6 encoding the LT by SYBR Green real-time polymerase chain reaction (PCR) assay. JCT 7 1 and JCT 2 primers [MacKenzie et al., 2003] for detection of JCV were used as a part 8 of a modified system. Cycling reactions and fluorescent detection was carried out in an 9 Opticon 2 instrument (MJ Research/Bio-Rad, Hercules, CA, USA). Briefly, the PCR 10 mixture consisted of 25 µl 2Х master mix (Platinum SYBR-Green qPCR SuperMix-11 UDG, Invitrogene, Carlsbad, CA, USA), 20 pmol of each primer, 1 µl ROX (as a 12 reference dye) and approximately 0.5÷1.2 µg of total DNA in a 50 µl total reaction 13 volume. Best signal strength and lowest level of background fluorescence was achieved 14 with the next protocol: 50 0 C for 2 min, 94 0 C for 10 min, followed by 50 cycles of 15 denaturation at 95 0 C for 15 sec and annealing/extension at 60 0 C for 30 sec with a plate 16 read at the end of each cycle (Fig. 1A). A dissociation profile was run after the 17 amplification stage in order to verify the specificity of the products. The samples were 18 gradually heated with an increment of 0.3 0 C and the change of the fluorescent signal 19 was determined at every step. Both negative (DNA free) and positive (JCV genomes, 20 partially sequenced and included in GenBank under accession no. GU222438 and 21 GU145384) samples were used in each run. Specimens were considered positive when a 22 significant increase of the fluorescence above the threshold line was obtained and the 23 determined specific melting temperature was corresponding to the positive control 24

2 3of NCCR sequences 4 11 1213 6 (7

 24116 QuantitationSamples were analyzed further for the NCCR. Detection and quantitation was 5 performed with the Primer Design TaqMan ® based JCV Advanced Kit (Cat. No. Path-6 JCV, Primer Design, Southampton, UK). A total number of 8 quantity control dilutions 7 with the respective control viral genomes, covering a spectrum of 10 8 down to 10 8 starting copies per reaction, were used in each run. All samples contained an internal 9 amplification control and results were interpreted according to the manufacturer's 10 guidelines. Both qualitative and quantitative PCR experiments were repeated at least twice 13 for verification of the results. All collected specimens were determined to contain sufficient amplifiable DNA 2 in the range of 0.2-1.2 µg/µl. The results from the SYBR Green ® assay targeted at LT, revealed the presence 4 of JC viral sequences in a total of 41 out of the 111 tested samples. The distribution of 5 LT positive patients among the different groups varied in a broad range. Notably, 48.7% n=19) from the tested 39 glioblastoma patients were positive for LT sequences.Another six in the astrocytoma group of 19 subjects, and four positive samples in the 8 oligodendroglioma group of 12 were determined. These formed a prevalence of 31.6% 9 and 33.3%, respectively. Two out of three samples from patients with oligoastrocytoma 10 were also positive for LT, as well as two out of five in the groups of medulloblastoma 11 and gangliogioma patients. None of the tested five ependimomas and only one from the 12 three pineocytoma harboured LT genomic sequences. Two patients, who underwent 13 radiotherapy following neurosurgery for an astrocytoma and glioblastoma are presented 14 separately. The result from the LT PCR was positive for both cases. The prevalence of 15

1

  Reports concerning the presence of HPoV sequences in human neoplasia vary 2 widely resulting in the absence of a conclusive opinion on the role of these viruses for 3 development of brain tumours. This research took into consideration several technical 4 details which might explain the observed differences between the research groups.

  5

8 2 [3 6 Further

 26 biopsies were determined to be LT positive. Also in a non tumour disease, where 9 surgery was performed due to arterio-venous malformation JCV LT fragments were 10 found. These results bring into concern a question regarding the role of the immune 11 system in controlling viral re-activation. It has been suggested that JCV is already 12 present in a latent state in the brain tissue [Lam et al.]. Most probably under conditions 13 of impaired cellular immunity viral reactivation may occur. If a severe immune 14 deficiency is present progressive multifocal leukoencephalopathy might develop. 15 However, it can be speculated that the immune response can help restrain the viral 16 reproduction and bring the viral infection again to a latent state. If it is assumed that 17 several reactivation-restriction cycles might be repeated there is a risk that viral 18 genomic element would be altered. Consequentially, uncontrolled expression of 19 regulatory elements such as LT might occur. If virions, which would normally trigger 20 the immune system, do not appear, this could result in a premise for malignant 21 transformation of the infected cell. This hypothesis is also supported by a recent 22 investigation of polyomavirs-induced tumours in mice, where NK cells and gammadelta 23 T cells mediated resistance to formation of the tumours [Mishra et al.]. Although this 24 perform an expression assay in order to show the activity of LT, 1 other researchers have also demonstrated that LT is expressed in brain tumour tissue Del Valle et al., 2001b]. It remains unclear if JCV has a decisive role in the process of brain cell 4 malignant transformation. The current results suggest that JCV might induce or play 5 role in the subsequent stages of malignant transformation. data in support of the infectious causes of cancer present a new concept that 7 warrants the need for advanced investigations.

  Grant support received from the Council for Medical Science at the Medical University 1 of Sofia, Bulgaria (Research project Contract No. 34/2010).

Table II

 II Presentation of the patients who were determined as both LT and NCCR positive

	1	ABSTRACT
	2 2	Tumor type A total of 111 fresh brain biopsies from patients with primary brain tumours were Sex and age Ct Number of NCCR copies INTRODUCTION
	3 3	Glioblastoma examined for JC polyomavirus (JCV) sequences from the Large T antigen encoding 57/F 36 10-100
	4 4	Glioblastoma region (LT) and the viral non-coding control region (NCCR). SYBR Green and TaqMan 67/M 50 10 Human polyoma viruses (HPoV) are small oncogenic viruses, comprised of an
	5 5	Radiotherapy induced alterations real-time polymerase chain reaction assays were used. In the gliobastoma group of 39 icosahedral capsid around a circular, double-stranded DNA molecule. Their genome is
		45/M	39	10
	6 6	in a patient with Glioblastoma patients 48.7% were positive for LT sequences. Among the astrocytoma group (19 approximately 5100 base pairs in length and can be divided into a non-coding control
	7 8 9	Astrocytoma Astrocytoma Oligodendroglioma patients) and the oligodendroglioma group (12 patients) 31.6% and 33.3% were also 51/F 37 10-100 58/M 41 10 41/F 38 10-100 region (NCCR) and overlapping genes encoding the viral regulatory and structural F proteins. There are no genes encoding viral replication enzymes. Therefore, the o r replicative strategy of HPoV relies on the induction of an S phase environment and
	10	Gangliogioma stimulation of the cellular machinery for specific enzyme production. This is a complex 25/F 41 10-100 P
	11	Pineocytoma process involving mainly the so called large T-antigen (LT) [Imperiale and Major, 15/F 40 10-100 e
	12	r e 2007]. This peptide can interact with several important regulatory cell molecules by
	13 14 15 16 17 18	binding to tumours suppressor proteins like pRb and p53 [Khalili et al., 2006; White R and Khalili, 2004] on the one hand; and to proto-oncogene products such as IRS-1 and e β-catenin, [Lassak et al., 2002],[Gan and Khalili, 2004] on the other. It is believed that v i this interference with the cell cycle regulation might be a key factor for induced e malignant transformation by viruses. w The primary infection with JC viruses (JCV), from the HPoV family, has been
	19	established to occur during childhood. The presence of specific JCV antibodies is
		pointed out by zur Hausen to be up to 98% of the human population [zur Hausen, 2008].
		2

7

positive. The prevalence of LT genomic sequences among the other groups was as 8 follows: in 2 out of 3 oligoastrocytomas; in 3 out 5 gangliogliomas; in 2 out of 5 9 meduloblastomas; in 1 out 3 pineocytomas; and in none of the tested 5 ependimomas. 10 All positive samples had a late threshold cycle that varied from 36 to 49, indicative of 11 very low starting viral number. Only 21 of all the 111 samples were positive for NCCR. 12 Low copy number in range of 10 to 1000 was present. Notably, only 8 of all NCCR 13 positive specimens were also LT positive. It might be suggested that the disproportion 14 between the results for LT and NCCR is either due to clonally integrated LT fragments, 15 with loss of genetic material, or changes in the NCCR. The latter would alter the 16 productive course of the infection and may establish a premise for continuous 17 interaction of viral regulatory proteins with cell molecules that are responsible for the 18 control of the cell cycle. This may lead subsequently to malignant transformation.

  Project approval from the Ethics Committee of Medical University of Sofia was

	1
	F o r
	P
	e
	e r
	R
	e
	v i e
	w
	23
	6

al., 14 2006]. 15 Differences in the patient populations and the used laboratory techniques have 16 been suggested as possible explanations for inconsistencies in the results across 17 different researchers [Del Valle et al., 2008; Rollison et al., 2005]. It has been generally 18 considered that additional facts are needed as data about the presence of polyoma virus 19 JC in primary brain tumours samples differ among the studied groups worldwide. 20 The aim of this study was to determine the prevalence of JC polyomavirus 21 sequences among patients with primary brain tumours and to evaluate the positive 22 samples in terms of viral copy number.

  Initially the samples were tested for sequences that are part of the LT gene. ItsThe presence of NCCR is essential for the proper replication of HPoV.

	1 1	"bizarre astrocytes" and formation of brain tumours in progressive multifocal tumours JCV sequences were also not found, as probably human genetic factors might
	2 2	leukoencephalopathy patients [Richardson, 1961]. Thus, the virus is considered a have a prevalent impact for the development of malignant transformation.
	3 3	plausible etiologic agent for the development of primary brain tumours [Del Valle et al., Samples were often processed and tested prior the definitive histological
	4 4	2008]. If replication of the viral genome and formation of mature particles are altered diagnosis, so six tumours, were later determined to be actually metastases. It might be
	5	and incomplete, it might be expected that viral sequences would be in low numbers
		and/or only partial genomic fragments would be present.
	7 8 9	F o r Amplification failure or low NCCR copy number might confer a prerequisite for incomplete or non-lytic course of the infection. It has been shown that JCV sequences F o r
	10	P originating from progressive multifocal leukoencephalopathy patients carry NCCR P
	11	e rearrangements compared to the archetypal strain [Miller et al., 1984]. Similarly, it is e
	12	e r possible that in brain tumour cells changes altering the function of NCCR may occur e r
	14 13 15 16 17 18	cell in an integrated state [Miller et al., 1984]. Similarly we expected a low number of R e leading to inability for mature virions to be formed. Indeed, it has been shown that R under the circumstances of continuous LT gene expression malignant transformation e sequences in the tested samples and employed assays with high sensitivity such as real-time PCR, giving a better chance for detection of viral genomes. Another novel real-v i e v i e time PCR technique that can be of use due to its high sensitivity and specificity is Light w w
		Upon Extension (LUX) real-time PCR [Kalvatchev et al., 2010].
	20 20
	21 21	product is a key factor for cell cycle disorganization and when genomic sequences that subependymal giant cell astrocytoma. This is a rare type of brain tumour that commonly
	22	encode for LT are identified this might represent indirect evidence in support of a occurs in the setting of the familial syndrome of tuberous sclerosis complex and
	23	possible role of JCV in malignant transformation. JCV has been the subject of intense
	24 24	investigation, as it possess stringent tropism towards the glial cells, the ability to induce

8 frozen materials or a laser microdissection technique is recommended [Del Valle et al., 9 2008]. It has been shown that a low copy number of viral sequences is present in human 10 tumours, necessitating the use of more sensitive assays rather than conventional PCR 11 followed by ethidium bromide staining [Del Valle et al., 2008]. Authors that identify 12 JCV sequences in monkeys inoculated experimentally that develop brain tumours, also 13 suggest that usually only a single copy of the viral genome is present in each tumour 19 6 14 may occur [Del Valle et al., 2001a]. A similar mechanism of viral-induced oncogenesis 15 was proposed for Kaposi's sarcoma associated herpesvirus [Zhao et al., 2007]. The 16 authors described that the disrupted control of lytic replication genes led to a non-lytic 17 course of the infection and a subsequent tumorigenesis. However, exact molecular 18 interactions and its significance in vivo are so far not well understood.

19

The histopathological diagnosis revealed one of the studied tumours to be 22 hereditary factors are considered to explain its development [Adriaensen et al., 2009]. 23 Therefore, as expected viral sequences were not detected. Similarly, in other rare brain


Péter Anna Fedor
email: fedoranna@gmail.com

Eörs Péter Ittzés
email: ittzes@gmail.com

Eörs Szathmáry
email: szathmary@colbud.hu

Parsing recursive sentences with a connectionist model including a neural stack and synaptic gating

Keywords: synaptic gating, neural stack, recursion, context-free grammar, finite state grammar

It is supposed that humans are genetically predisposed to be able to recognize sequences of context free grammars with center-embedded recursion while other primates are restricted to the recognition of finite state grammars with tail-recursion. Our aim was to construct a minimalist neural network that is able to parse artificial sentences of both grammars in an efficient way without using the biologically unrealistic backpropagation algorithm. The core of this network is a neural stack-like memory where the push and pop operations are regulated by synaptic gating on the connections between the layers of the stack. The network correctly categorizes novel sentences of both grammars after training. We suggest that the introduction of the neural stack memory will turn out to be substantial for any biological 'hierarchical processor' and the minimalist design of the model suggests a quest for similar, realistic neural architectures.

Introduction

Natural language is a fascinating phenomenon, very much in the focus of various disciplines, from linguistics proper to evolutionary biology [START_REF] Bickerton | Language and Species[END_REF][START_REF] Hauser | The faculty of language: what is it, who has it, and how did it evolve?[END_REF][START_REF] Hurford | The Origins of Meaning[END_REF][START_REF] Smith | The Major Transitions in Evolution[END_REF][START_REF] Pinker | The Language Instinct[END_REF]. Although there is no general agreement on how to best characterize language, let alone its biological foundations, we follow the view that it is based on the critical combination of symbolic reference with complex syntax [START_REF] Szathmáry | Towards an understanding of language origins[END_REF]. A crucial element of syntax is recursion (Corballis, 2007a;[START_REF] Hauser | The faculty of language: what is it, who has it, and how did it evolve?[END_REF]. Two main types of recursion occurring in natural language are tail-or end-recursion (including left and right-branching recursion) and centre-embedded recursion (CER). An example of left-branching tail-recursion is (after the popular British nursery rhyme The house that Jack built):

(1) The rat squeaked.

(2) The cat killed the rat that squeaked.

(3) The dog worried the cat that killed the rat that squeaked.

In these cases, sentences are characterized by the concatenation of coherent noun-verb pairs, and can be produced or parsed by simple iteration [START_REF] Christiansen | Toward a connectionist model of recursion in human linguistic performance[END_REF]. Iteration is present in animal calls, like that of primates (e.g., [START_REF] Robinson | Syntactic Structures in the Vocalizations of Wedge-Capped Capuchin Monkeys, Cebus Olivaceus[END_REF][START_REF] Zuberbühler | A syntactic rule in forest monkey communication[END_REF]) and songbirds [START_REF] Eens | Understanding the complex song of the European starling: An integrated ethological approach[END_REF]. It was also proved that some animal species are able to infer the iterative rule from samples of artificial strings and generalize over novel strings [START_REF] Robinson | Syntactic Structures in the Vocalizations of Wedge-Capped Capuchin Monkeys, Cebus Olivaceus[END_REF]. If we represented word-pairs only, where word-pairs consist of two words with dependency between them (e.g., in sentence 3, these are: dog-worried, cat-killed and ratsqueaked), such sentences composed of three word-pairs can be described by the following rule:

(4) A 3 B 3 A 2 B 2 A 1 B 1 ,
where As represent nouns, Bs represent verbs, and words with the same index form wordpairs. The above sentences (2 and 3) can be transformed to have centre-embedded structure:

(5) The rat that the cat killed squeaked.

(6) The rat that the cat that the dog worried killed squeaked.

Here, the general rule for three word-pairs is:

(7) A 1 A 2 A 3 B 3 B 2 B 1 .
CER is claimed to be a general human capacity whereas it cannot be found in animal communication systems [START_REF] Fitch | Computational constraints on syntactic processing in a nonhuman primate[END_REF]). It has not been proved either that any animal species is capable of learning CER in the laboratory. CER can be parsed by contextfree grammar (CFG), which has higher generative power than finite-state grammar (FSG) which basically concatenates items (applies tail-recursion, (Corballis, 2007a); for the comprehensive hierarchy of formal grammars, see [START_REF] Chomsky | Syntactic Structures Mouton[END_REF]).

In natural languages, every noun has several possible verb pairs, and vice versa, every verb has several possible noun pairs. In the above examples semantic relationship connects the words: usually only rats squeak, not dogs or cats. However, if the other two word-pairs are swapped, the sentence still makes sense:

(8) The dog killed the cat that worried the rat that squeaked.

In a slightly modified version of the sentence, it is not possible to swap the word-pairs, because singular and plural words must be matched:

(9) Dogs worry the cat that kills rats that squeak.

As a result of this grammatical constraint (together with the semantic constraint), even if the words were mixed without grammatical structure, it would be easy to see the coherent nounverb pairs. In artificial languages without semantics, if these dependencies between words are not established somehow, sentences could be represented by a simpler structure, A n B n , e.g. for n=3:

(10) A A A B B B.
This grammar is called counting recursion [START_REF] Christiansen | Toward a connectionist model of recursion in human linguistic performance[END_REF] because parsing of this kind of sentences is possible by counting As and Bs. If the number of As and Bs is equal and there is only one transition from As to Bs the sentence is correct (Corballis, 2007a;Corballis, 2007b).

The structures that word-pairs imply are shown on Figure 1. In the case of tail-recursion, members of word-pairs are next to each other connected by local dependencies. Additionally, in sentences with CER there are word-pairs whose members are separated by other word-pairs thus they have long-distance (or long-range) dependencies. This implies a hierarchical structure compared to the linear structure of sentences with tail-recursion. The more levels this hierarchical structure has, the more words have to be remembered to be able to parse these sentences. In a six-word-long sentence with CER, the maximum number of words that has to be stored in memory is three and the first word has to be remembered until the presentation of the last one. In sentences with tail-recursion, members of word-pairs are presented shortly after each other, hence there is only one word that has to be remembered at a time. In counting recursion no individual word has to be stored in memory for grammaticality judgement, just the category of words passed and their quantity has to be remembered until the end of the sentence. Some confusion resulted from using sentences of counting recursion in artificial language learning experiments and not differentiating them clearly from the more complex centreembedded sentences. [START_REF] Fitch | Computational constraints on syntactic processing in a nonhuman primate[END_REF] claimed that their human subjects were able to learn both FSG and CFG in a small artificial language, whereas cottontop tamarins could learn only FSG. Since they did not establish dependencies between words their CFG sentences could be parsed by counting recursion. Likewise, Gentner, Fenn, Margoliash & Nusbaum [START_REF] Gentner | Recursive syntactic pattern learning by songbirds[END_REF] claimed that their subjects (starlings) learnt CER using the same kind of structures as [START_REF] Fitch | Computational constraints on syntactic processing in a nonhuman primate[END_REF]. This in turn elicited some critique: Corballis (Corballis, 2007a;Corballis, 2007b) called attention to the fact that the sentences used could be parsed by simple counting, while others [START_REF] De Vries | Syntactic structure and artificial grammar learning: The learnability of embedded hierarchical structures[END_REF][START_REF] Perruchet | Does the Mastery of Center-Embedded Linguistic Structures Distinguish Humans from Nonhuman Primates?[END_REF] showed that in experimental situation similar to that of [START_REF] Fitch | Computational constraints on syntactic processing in a nonhuman primate[END_REF]) even human subjects used alternative strategies to solve the tasks.

Despite the controversy in artificial language experiments, there is a general agreement that centre-embedded structures are present in natural human languages, but not in natural animal communication systems. The question is, why. A simple answer would be that animals do not have Universal Grammar, but this leaves completely in the dark what the relevant biological differences could be. In fact there are serious doubts on the idea that abstract rules of Universal Grammar could ever get assimilated in the genome [START_REF] Deacon | Multilevel selection in a complex adaptive system: the problem of language origins[END_REF][START_REF] Wiles | Transient Phenomena in Learning and Evolution: Genetic Assimilation and Genetic Redistribution[END_REF] but, in contrast, it cannot be doubted that some language-related genetic differences between humans and animals do exist. It is perhaps much more rewarding to enquire about the possible neuronal operations (procedures) that the brain could implement in order to handle language.

Connectionist models have increasingly been used to model empirical data across many areas of language processing (Christiansen and Chater, 2001a). However, connectionist models aiming at parsing recursive structures are often restricted to counting recursion. Sun et al. [START_REF] Sun | The Neural Network Pushdown Automaton: Architecture, Dynamics and Training[END_REF]) implemented a hybrid system, in which a recurrent neural network was coupled to an external non-neural stack memory. After training with backpropagation, the system was able to infer a CFG from input. In another study, continuous-time recurrent networks without a stack can learn both context-free and context-sensitive languages in a prediction task, using backpropagation through time [START_REF] Bodén | Context-free and context-sensitive dynamics in recurrent neural networks[END_REF]. Since there were no long-range dependencies connecting words within the sentences, performance of these systems boiled down to counting [START_REF] Rodriguez | A recurrent neural network that learns to count[END_REF].

Other studies used input data conforming to real CER (as opposed to counting recursion) to train artificial neural networks. Elman [START_REF] Elman | Distributed Representations, Simple Recurrent Networks, And Grammatical Structure[END_REF] trained a simple recurrent network (SRN) on multiclausal sentences which contained multiply-embedded relative clauses. The network achieved a high level of performance in predicting the next word in the sentences. In a related model [START_REF] Christiansen | Toward a connectionist model of recursion in human linguistic performance[END_REF] trained SRNs on recursive artificial languages. The behaviour of these networks was similar to human performance in that they reached higher performance in right-branching structures than in centre-embedded structures. In both studies backpropagation of error was used as a learning algorithm which is generally considered biologically implausible because it requires passage of information backward through synapses and along axons and because it uses error signals that must be precise and different for each neuron in the network [START_REF] Mazzoni | A more biologically plausible learning rule for neural networks[END_REF][START_REF] O'reilly | Biologically plausible error-driven learning using local activation differences: The generalized recirculation algorithm[END_REF].

Handling of hierarchical structures occurs at high speed during language production and comprehension, and it seems reasonable to assume that it requires specialized neural networks to do so [START_REF] Fedor | The biological background of syntax evolution[END_REF]. It is well known that parsing of CER can be solved very efficiently by a stack (push-down automaton), with the necessary pop and push operations [START_REF] Hopcroft | Introduction to Automata Theory, Languages, and Computation[END_REF]. Thus it would be a step forward to present a neurally plausible simple stack architecture that could parse CER. Along this line, Chen and Honavar [START_REF] Chen | A neural-network architecture for syntax analysis[END_REF] proposed an artificial neural network architecture for syntax analysis which is assembled from neural network components for lexical analysis, stack, parsing and parse tree construction. The stack in their model is a fairly complex system that is composed of five different modules that have specifically designed connections and the stack requires four sets of binary inputs. We aimed at constructing a more minimalist architecture for a neural stack that is more similar to a push-down automaton in its architecture.

In this paper we will present a neural network which can be trained to parse sentences with tail-recursion and CER. Three key features of the model are: a) absence of backpropagation, b) a crucial role for synaptic gating, c) and a neurally implemented stack. These components of our model are not new, but the combination of these features is unprecedented -this is what makes this model very effective and minimalistic.

Methods

Grammars

We composed input sentences according to two types of recursion, namely tail-recursion and CER. Words were 0/1 binary strings, where there was only one 1 in each word (all the other digits are 0). Words were randomly divided into two groups, A and B. Each word from group A had exactly one (randomly chosen) pair from group B, and vice versa. Sentences ( 4) and ( 7)

give examples for six-word-long sentences with tail-recursion and CER, respectively. Since no word occurs twice in a sentence, 8*7*6=336 sentences could be generated for each grammar.

Additionally, random agrammatical sentences were also generated. These sentences were also composed of three A words and three B words and always started with an A, just as grammatical sentences, but did not conform to any of the above rules.

Architecture and functioning of the network after successful training

The neural network consists of the following main modules: input layer, stack, predictor, two push-pop neurons and a decision neuron (Fig. 2.). The input layer receives one word at a time from the sentence. In the case of a 16-word vocabulary, the input layer has 16 units (neurons), where each unit corresponds to a single word. Second, there is a clocked stack (for a clocking mechanism see [START_REF] Hjelmfelt | Chemical implementation of neural networks and Turing machines[END_REF]) with three layers, where every layer of the stack consists of 16 neurons. Adjacent neurons within a column of the stack are connected bidirectionally to each other, with a weight of 1. The third component, called the predictor, tries to predict the next word in the sentence based on the word that is stored in the top layer of the stack. The push-pop neurons have input connections from the predictor and the input layer. They basically compare the two, and if they store the same words (which means that the prediction was correct) signal +1, if they store different words (or if the predictor is empty) signal -1. The output connections of the push-pop neurons perform neural gating on the synapses of the stack: these connections modulate the synapses directly by enhancing or inhibiting them (i.e., the push-pop neurons are the so called 'gatekeepers'). One of the pushpop neurons is an excitatory neuron which is connected to each upward synapse in the stack with positive weights (i.e., gating acts in a permissive fashion [START_REF] Katz | Synaptic Gating: The Potential to Open Closed Doors[END_REF]) and the other one is an inhibitory neuron, which is connected to each downward synapse in the stack with negative weights (absolute suppressive gating). The signal that arrives to a synapse from a gating neuron is the product of the activation of the gating neuron and weight on the synapse of the gating neuron, just like with any other neurons. The difference is that negative signal from a gating neuron blocks the synapse it is connected to, while positive signal from a gating neuron makes it possible for the synapse to work. As a result, if the prediction was correct and the push-pop neurons signal +1, downward connections will be inhibited and upward connections will be enhanced in the stack, hence upward connections will predominate, and each layer will take the value of the layer bellow it (a pop action). In this case, the bottom layer becomes empty. On the other hand, if the prediction was not correct and the push-pop neurons signal -1, upward connections will be inhibited and downward connections will be enhanced in the stack, such that the downward synapses will predominate and every layer will take the value of the layer above it (a push action). In this case, the top layer takes its value from the input. Lastly, there is a decision neuron, which is connected to the top layer of the stack and signals only if there is a word stored on the top layer of the stack. The signalling of this neuron can be considered as the decision of the network on the grammaticality of the sentence: signalling means that the sentence encountered so far was ungrammatical, while 0 output means that the sentence is grammatical. Now, let us see how the whole network with a three-layer-deep stack is supposed to parse a six-word-long sentence after learning. First, the predictor tries to predict the first word from the top of the stack, but since the stack is empty at the beginning, the predictor will have no prediction. Next, the first word arrives to the input layer and then the push-pop neurons compare the input with the prediction. Since prediction is unsuccessful (there is no word on the predictor), the push-pop neurons perform a push action on the stack and the top layer of the stack becomes occupied by the first word. This triggers the decision neuron, which will signal that the string encountered so far is ungrammatical. In the case of a sentence with tailrecursion, the next word depends on the previous one. At the beginning of the next cycle, the predictor predicts the next word based on the previous word which is stored on the top of the stack. If the prediction is correct, the push-pop neurons will perform a pop action on the stack, which will become empty again. This will suppress the signalling of the decision neuron which means that the sentence encountered so far is grammatical. The same push-pop actions are repeated with the next two word-pairs until the end of the sentence. To measure the performance of the network we can detect its predictions for the following words or its decisions on the grammaticality of the sentence. During the processing of a sentence with tailrecursion, the network is able to predict every second word (other words can not be predicted, hence maximum performance is 50%) and decides that the sentence is incorrect three times: after the first, third, and fifth word. Note, that independently of the length of the sentence, substrings of a grammatical sentence with even number of words are in fact grammatical in the case of tail-recursion.

In the case of an agrammatical sentence, one or more words do not have a pair, which means that more than three words are unpredictable in the sentence. This results in more than three push actions, which means that the stack is not empty at the end of the sentence and the decision neuron will signal that the sentence was agrammatical.

Parsing grammatical sentences with CER also involves equal number of push and pop actions; hence the stack is empty at the end of a grammatically correct sentence. The difference is that there are three push actions until the predictor can finally predict a word. Prediction is always based on the word that is stored on the top of the stack, and after three unsuccessful predictions on the first three words, these words are stored in the three layers of the stack, with the third word being on the top. The fourth word is predictable from the third word, which means that the push-pop neurons will perform a pop action on the stack, after which the second word will be on the top. The fifth word can be predicted from the second word, which means another pop action, and finally, the sixth word is predicted from the first one. As in the case of tail-recursion, the stack is empty after the presentation of a grammatically correct sentence.

It can be seen that in the case of a sentence with tail-recursion, only the top layer of the stack is used, whereas for parsing a six-word-long sentence with CER, three layers are used.

Generally, the number of stack layers required for parsing a centre-embedded structure is half of the number of words in the sentence. If there are fewer layers, the network will categorize sentences with CER as agrammatical. Note that tail-recursion can be parsed without push-pop neurons and stack if you use simple copying from the input layer to a one-layer memory instead of a push action and deletion of the memory instead of a pop action. The difference between animals that cannot parse CER and humans can be that the former lack the stack and the gating mechanism, without which only local dependencies between words can be parsed.

It can be argued that the stack architecture in this form cannot explain why deeper embeddings are harder to process for humans. With this solution two levels of embedding (6word-long sentences) are processed perfectly, whereas three or more levels are impossible.

However, if we realize that every neural computation is prone to errors, we will see that the stack architecture with many layers also shows graceful degradation in performance as the level of embedding increases. If every push or pop operation in the stack has a small probability to result in imperfect transmission of information from one layer to another, then the more embedding the sentence have the more probable is its faulty parsing.

Training

While the architecture of the network described above is hand-crafted, its synaptic weights develop during training. For the training we randomly chose a subset (the learning set) from the grammatical sentences of either type of recursion. Training consisted of presenting the learning set several times and modifying the weights of the network. Testing was performed on the rest of the grammatical sentences that the network has not encountered before randomly mixed with agrammatical sentences. During testing no weight change occurred. The performance of the network was measured by its predictions for the following words in grammatical sentences during training and testing and its decisions on the grammaticality of the sentences at the end of the sentences. Note, that theoretically the maximum performance for prediction is 50% in grammatical sentences (obviously it was not measured for agrammatical sentences). Grammaticality judgement during testing measures if the network can differentiate grammatical from agrammatical sentences, while during training it is not very informative, since there were only grammatical sentences in that phase.

Different learning rules were used to modify the weights of the network. For the weights between the top layer of the stack and the predictor layer, a simple Hebbian learning rule was used. After the predictor layer tried to predict the next word and the push-pop neurons compared the prediction with the next word on the input, the input was copied to the predictor. Then learning occurred in this time step by increasing the synaptic weights between those neurons that were activated:

if N t =1 and N p =1 then W tp =W tp + r,
where N t is a neuron on the top of the stack, N p is a neuron on the predictor layer, W tp is the synaptic weight between them and r is the learning rate (r was set to 0.01).

For modifying the synaptic weights of the push-pop neurons coming from the predictor and the input layer and the synaptic weights of the decision neuron coming from the top of the stack, the perceptron learning rule was used with threshold transfer function [START_REF] Dayan | Theoretical neuroscience: computational and mathematical modeling of neural systems[END_REF]. This learning rule modifies the weights and the threshold to reach an output that is closer to a precalculated desired output. For this only local information is used: the activation of the input and the output layer (e.g., in the case of the decision neuron the input is the top of the stack) and the synaptic weights:

W = W + h*(O -O') * I and T = T -h*(O -O'),
where W is the weight matrix between the input and the output layer, I is the input, O is the desired output, O' is the actual output, T is the threshold for the transfer function and h is the learning rate (h was set to 0.1). The desired output for the push-pop neurons is 1 (pop) if the prediction was correct (i.e., if the input layer and the predictor layer has the same activation pattern) and -1 (push) if the prediction was incorrect. For the decision neuron, the desired output is 0 if the top of the stack is empty and 1 otherwise. (Note that starting with nonzero random weights, the decision neuron automatically works well without learning.)

Weight modification occurred online, i.e. after the presentation of each word in the case of both learning rules. (We tried batch learning too, where weight modification occurs after the presentation of the whole training set. The network basically reached the same performance, however, we find it less realistic, and hence we used online learning for generating the figures in this paper.)

Those weights that copy activation from one layer to another were not trained, but set to the desired values from the beginning: between the input and the predictor, between the input and the top of the stack, and the weights between the layers of the stack. It might seem to be quite artificial that the weights of the stack are not trained but are precalculated. However, since it is a very simple structure it would be easy to train in an extended version of this model, just as the other copy weights. Training was performed on a randomly chosen subset of the 336 grammatical sentences, while testing was performed on the rest of the grammatical sentences mixed with agrammatical sentences. Note, that the theoretical maximum for prediction performance is 50% in the case of grammatical sentences for both grammars (it was not measured for agrammatical sentences).

Results and conclusions

Learning performance

In the case of tail-recursion a training set composed of 10 grammatical sentences was usually enough for the network to generalize and reach perfect or almost perfect performance on novel sentences. For this about 5 training sessions were needed. In the case of CER, 30 training sentences presented for 11-12 training sessions were needed to reach the same performance. For both grammars, perfect performance on novel sentences is possible provided that every word-pair is presented during the training sessions. There is no generalization on the level of word-pairs; it is simply not possible since words are paired randomly. However, the network successfully generalizes on the level of sentences as can be seen from its performance on novel test sentences.

The network can also be trained with a mixed set of sentences conforming to tail-recursion and CER. With 30 sentences, only about 6-7 training sessions are needed to reach perfect performance which indicates faster learning than with CER sentences only. This is quite intuitive: tail-recursion seems easier to learn than CER since words forming a word-pair are presented immediately after each other. For the successful parsing of the grammars both memorizing the word-pairs and recognizing the particular structure is necessary. Since finding the words that depend on each other seems to be easier in the case of tail-recursion, we predict that it would help humans to learn CER if sentences were mixed with tail-recursive sentences.

Extensions

Since the performance of the network is based on the successful learning of word-pairs coupled with the push-pop operations of the stack, it can learn any language that is based on the balanced pairing of words in sentences. One example is the Dyck language of balanced parenthesis, which can be thought of as a mix of tail-recursion and centre-embedded recursion. E.g.: strings like aaabbaabbabb can be parsed by the model after learning that a and b are pairs. Another example is the palindrome (mirror) language; sentences like abccba can also be parsed by the network. The difference with centre-embedded recursion is that in this case word-pairs consist of two identical words.

The network cannot learn counting recursion in this form, since it has no module that would learn to categorize words to group A and group B. However, if we inserted a module that was able to categorize words on the predictor and on the input layer, it would make it possible to parse sentences with counting recursion too.

Conclusions

The main features of the neural network implemented here is the neurally implemented stack operated by gating neurons1 . While it is well known that recursive sentences can be parsed by a symbolic stack, to our knowledge, there was no simple neural implementation of this structure until now. Symbolic models that could not be neurally implemented can be ruled out for being implausible [START_REF] Christiansen | Connectionist psycholinguistics in perspective[END_REF]. What we show here is that the stack indeed can be neurally implemented, and it is quite simple provided that the push-pop operations are guided by gating connections.

We believe that gating will be found crucial for hierarchical tasks, just as for complex cognition in general [START_REF] Gisiger | Acquisition and performance of delayed-response tasks: a neural network model[END_REF][START_REF] O'reilly | Biologically based computational models of high-level cognition[END_REF]. The fact that it has readily evolved in a reinforcement-learning task in a simulated honeybee neural network (Soltoggio et al., 14 2007) supports this idea. We suggest that the introduction of the neural stack memory (pushdown automaton) will also turn out to be substantial for any biological 'hierarchical processor'. This is not to say that it is just a neural stack that is crucial for language, neither do we suggest that the stack architecture proposed here exists in a clean, isolated form in the brain, but it is likely that similar networks are embedded in the wider, language-related network context.

The performance of our network naturally depends on the depth of the stack, and as such it can be replaced by a finite-state automaton [START_REF] Hopcroft | Introduction to Automata Theory, Languages, and Computation[END_REF]. However, in this sense human parsing ability is also limited: no person can parse sentences with arbitrarily many levels of embeddings [START_REF] Pinker | The Language Instinct[END_REF]. The likely hierarchical processor (maybe even supramodal) in humans with normal development is Broca's area [START_REF] Friederici | Broca's area and the ventral premotor cortex in language: functional differentiation and specificity[END_REF][START_REF] Tettamanti | Broca's area: a supramodal hierarchical processor?[END_REF]. Sadly, we know next to nothing about the relevant 'internal wiring' of this area: we propose that it is likely to contain a neural stack, wherein gating will be found important.

It would be premature to contemplate about the origin of stack-like neuronal systems in evolution and development. However, there seem to be two scenarios: either stacks are hardwired (genetically coded) in our brain or we are born without them and the plasticity of our brain (under genetic control) makes us 'ready' to organize stacks during development. We think that the second scenario is more plausible but future work is needed to resolve these issues.

Figure captions

Fig. 1. Tail-recursion (a), counting recursion (b) and centre-embedded recursion (c). A and B represent word categories and As and Bs with the same index form word-pairs. Word-pairs imply only local dependencies in a) but also long-range dependencies in c). There are no word-pairs in b) (Corballis, 2007a;Corballis, 2007b).

Figure 3 a

 3 Figure 3 a) and b) show the performance of the model during several training sessions and a test session averaged over 10 runs in the case of tail-recursion and CER, respectively. Performance is measured by the correctness of grammaticality judgement at the end of sentences (Decision) and by the correctness of the prediction for words during sentences (Prediction). Black data points represent performance during training while the last white data points represent performance during testing. Training was performed on a randomly chosen

Fig. 2 .

 2 Fig. 2. Architecture of the proposed neural network model. The input layer receives words from sentences one-by-one. The stack is represented with three layers with bidirectional interlayer connections. The predictor layer tries to predict the next word based on the word that is stored at the top of the stack (stack layer 1). There are two push-pop neurons (P) that has gating connections (dashed lines) on the inter-layer connections of the stack. Inhibitory gating connections are marked by a circle at the end and excitatory gating connections are marked by a diamond at the end. The decision neuron (D) gives grammaticality judgement on the sentence. Copying connections that are not trained are indicated by empty arrows. Synapses between the top of the stack and the predictor are indicated by a thick arrow and trained by the Hebbian learning rule. All other synapses are trained by the perceptron learning rule.

Fig. 3 .

 3 Fig. 3. Performance of the neural network model on a) tail-recursion and b) centre-embedded recursion averaged over 10 runs. Performance is measured by the correctness of grammaticality judgement at the end of sentences (Decision) and by the correctness of the prediction for words during sentences (Prediction). Black data points represent performance during training while the last white data points represent performance during testing. Training was performed on 10 and 30 randomly chosen grammatical sentences in the case of tailrecursion and centre-embedded recursion, respectively. Testing was performed on the rest of the grammatical sentences mixed with agrammatical sentences.

Fig. 1a

  Fig. 1a

Fig. 1b

  Fig. 1b

Fig. 1c

  Fig. 1c

  Fig. 2

The stack follows a design borrowed from the chemical literature(Hjelmfelt et al., 1992) that rests on gating.

Acknowledgements

Thanks for the useful suggestions of Máté Lengyel, Chrisantha T. Fernando, Ferenc Huszár and Zoltán Szatmáry. This work was supported by the European Union Framework 6 (Grant IST-FET 1940). Support by the National Office for Research and Technology (NAP2005/KCKHA005) and by the National Scientific Research Fund (OTKA73047) is gratefully acknowledged.