Ronan Thibaut

Pierre Bost

Idan Milo

Marine Cazaux

Fabrice Lemaître

Zacarias Garcia

Ido Amit

Béatrice Breart

Clémence Cornuot

Benno

Benno Schwikowski

Philippe Bousso
email: philippe.bousso@pasteur.fr.

Bystander IFN-γ activity promotes widespread and sustained cytokine signaling altering the tumor microenvironment

ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

IFN-γ is a key soluble effector molecule during anti-tumor immune responses. Mice deficient for IFN-γ production are prone to develop spontaneous or carcinogen-induced tumors, highlighting the crucial importance of IFN-γ during immunosurveillance [START_REF] Kaplan | Demonstration of an interferon gamma-dependent tumor surveillance system in immunocompetent mice[END_REF][START_REF] Street | Perforin and interferon-gamma activities independently control tumor initiation, growth, and metastasis[END_REF][START_REF] Dunn | Interferons, immunity and cancer immunoediting[END_REF] . IFN-γ also plays an important role during tumor immunotherapy as the benefit of anti-Programmed Death 1 (PD-1) therapy in mice has been shown to be dependent on IFN-γ production by CD8 + T cells and the associated IL-12 production by dendritic cells [START_REF] Garris | Successful Anti-PD-1 Cancer Immunotherapy Requires T Cell-Dendritic Cell Crosstalk Involving the Cytokines IFN-gamma and IL-12[END_REF] . Moreover, in melanoma patients, acquired resistance to anti-CTLA-4 therapy has been associated with mutations in IFN-γ signaling [START_REF] Gao | Loss of IFN-gamma Pathway Genes in Tumor Cells as a Mechanism of Resistance to Anti-CTLA-4 Therapy[END_REF] .

Several mechanisms have been described to explain the beneficial impact of IFN-γ in the tumor microenvironment. For example, IFN-γ can exert direct cytotoxic or cytostatic effects on tumor cells [START_REF] Chin | Cell growth arrest and induction of cyclin-dependent kinase inhibitor p21 WAF1/ CIP1 mediated by STAT1[END_REF][START_REF] Ikeda | The roles of IFN gamma in protection against tumor development and cancer immunoediting[END_REF] , contribute to tumor senescence [START_REF] Braumuller | T-helper-1-cell cytokines drive cancer into senescence[END_REF] and tumor ferroptosis [START_REF] Wang | CD8(+) T cells regulate tumour ferroptosis during cancer immunotherapy[END_REF] . In addition, IFN-γ signaling on stromal cells can lead to reduced angiogenesis in the tumor bed, limiting nutrients and dioxygen access [START_REF] Ikeda | The roles of IFN gamma in protection against tumor development and cancer immunoediting[END_REF][START_REF] Kammertoens | Tumour ischaemia by interferon-gamma resembles physiological blood vessel regression[END_REF] . IFN-γ signaling can also lead to Major Histocompatibility Complex (MHC) class I upregulation thereby increasing tumor sensitivity to CD8 + T cell-mediated lysis [START_REF] Meunier | T cells targeted against a single minor histocompatibility antigen can cure solid tumors[END_REF] . Finally, IFN-γ orchestrates the recruitment of Natural Killer (NK) cells, T cells and invariant NK T (iNKT) cells to the tumor by triggering CXCL9, CXCL10 and CXCL11 production [START_REF] Groom | CXCR3 ligands: redundant, collaborative and antagonistic functions[END_REF] .

Conversely, intratumoral IFN-γ may have detrimental effects on the outcome of anti-tumor immune responses. In this respect, it has been reported that IFN-γ increases tumor cell genomic instability [START_REF] Takeda | IFN-gamma is required for cytotoxic T cell-dependent cancer genome immunoediting[END_REF] , a process that may favor tumor escape from the immune pressure. IFN-γ can also induce the expression of ligands for inhibitory receptors such as Programmed Death-Ligand 1 (PD-L1) and PD-L2 on stromal and tumor cells [START_REF] Garcia-Diaz | Interferon Receptor Signaling Pathways Regulating PD-L1 and PD-L2 Expression[END_REF] , contributing to the inhibition of tumor-infiltrating T cells [START_REF] Dong | Tumor-associated B7-H1 promotes T-cell apoptosis: a potential mechanism of immune evasion[END_REF][START_REF] Freeman | Engagement of the PD-1 immunoinhibitory receptor by a novel B7 family member leads to negative regulation of lymphocyte activation[END_REF] . IFN-γ could also induce the deletion of tumor-reactive T cells by favoring activation-induced cell death [START_REF] Pai | Clonal Deletion of Tumor-Specific T Cells by Interferon-gamma Confers Therapeutic Resistance to Combination Immune Checkpoint Blockade[END_REF] . Finally, long term IFN-γ exposure has been associated with deep transcriptomic changes in tumor cells that can contribute to the failure of immune checkpoint blockade therapy [START_REF] Benci | Tumor Interferon Signaling Regulates a Multigenic Resistance Program to Immune Checkpoint Blockade[END_REF] .

Despite our understanding of molecular and cellular mechanisms triggered by IFN-γ, we are critically lacking informations on the spatiotemporal activity of this cytokine in the tumor microenvironment. Quantitative rules regulating cell-cell communication are only starting to be considered [START_REF] Altan-Bonnet | Cytokine-mediated communication: a quantitative appraisal of immune complexity[END_REF] but are essential to understand complex immune responses. Fundamental questions remain to be addressed. Does IFN-γ act primarily in discrete areas or more widely in the tumor microenvironment? How is IFN-γ spatial activity shaped by the number of cytokine-producing T cells? What is the duration of IFN-γ exposure required to alter tumor cell function and phenotype? It is well established that IFN-γ is secreted by T cells in a directional manner towards the immunological synapse formed with target cells [START_REF] Huse | T cells use two directionally distinct pathways for cytokine secretion[END_REF] . Upon TCR stimulation, IFN-γ production is also thought to be a brief (few hours long) event 21, 22 . Although these two features are expected to favor a local activity of IFN-γ around the sites of cytokine production, the extent of IFN-γ diffusion in tissue has yet to be fully understood. Indeed, using in vitro settings, it has been shown that the immunological synapse does not perfectly restrict IFN-γ secretion, which accounts for bystander activity of IFN-γ in cells that are not in contact with cytokine-producing T cells [START_REF] Sanderson | Cytotoxic immunological synapses do not restrict the action of interferongamma to antigenic target cells[END_REF] . Evidence for bystander IFN-γ activity have also been reported in vivo in the mesenteric lymph node following Toxoplasma gondii infection [START_REF] Perona-Wright | Sustained signaling by canonical helper T cell cytokines throughout the reactive lymph node[END_REF] and in the dermis infected with Leishmania major [START_REF] Müller | CD4+ T cells rely on bystander effector activity to control local infection[END_REF] . Yet, the extent and functional consequences of IFN-γ diffusion in the microenvironment of distinct tumor types remain unclear.

Here, we investigated the spatiotemporal activity of IFN-γ in the tumor microenvironment using B cell lymphoma and melanoma models. Using distinct approaches including mosaic tumors and intravital imaging with a real-time reporter of Signal Transducer and Activator of Transcription 1 (STAT1) nuclear translocation, we show that IFN-γ signaling occurs with extensive bystander activity throughout the tumor microenvironment, affecting both tumor and infiltrating immune cells. Although STAT1 activity in tumor cells was sustained, it did not require continuous interaction with T cells, confirming the extensive bystander activity of intratumoral IFN-γ. We provide evidence that prolonged IFN-γ exposure is in fact needed to alter tumor cell phenotype. Our results support a model in which IFN-γ regulates the tumor microenvironment by acting as a widespread and sustained cytokine field whose concentration is dependent on the collective T cell activity.

Results

T cell-derived IFN-γ profoundly modifies the tumor microenvironment

In order to characterize how intratumoral IFN-γ acts on tumor cells, we examined in distinct tumor cell lines the expression of several markers, including MHC class I and PD-L1 upon exposure to various concentrations of IFN-γ. Using models for Myc-driven B cell lymphoma (Eμ-myc) [START_REF] Harris | The E mu-myc transgenic mouse. A model for high-incidence spontaneous lymphoma and leukemia of early B cells[END_REF] , melanoma (B16.F10) [START_REF] Hart | The selection and characterization of an invasive variant of the B16 melanoma[END_REF] and mammary tumor (E0771) [START_REF] Sugiura | Studies in a tumor spectrum. I. Comparison of the action of methylbis (2chloroethyl)amine and 3-bis(2-chloroethyl)aminomethyl-4-methoxymethyl -5-hydroxy-6methylpyridine on the growth of a variety of mouse and rat tumors[END_REF] we observed a dose-dependent upregulation of these markers for all cell lines (Fig. 1a-b, Extended data Fig. 1a). Phenotype changes upon IFN-γ treatment followed an analog, rather than a digital, pattern with a concentration-dependent shift in expression of the whole population rather than a bimodal distribution (Extended data Fig. 1b). At high concentration, IFN-γ also induced substantial cell death of B16.F10 tumor cells and an even more pronounced effect when combined with TNF-α (Extended data Fig. 1c,e). By contrast, IFNγ did not induce substantial cell death of Eμ-myc cells either alone or in combination with TNF-α, highlighting the diversity of tumor response to this cytokine (Extended data Fig. 1d,f).

To test whether these phenotypic changes also occur in response to intratumoral T cellderived IFN-γ in vivo, we adoptively transferred OT-I CD8 + T cells into Rag2 -/-mice bearing ovalbumin+ (OVA + ) Eμ-myc lymphomas and analyzed tumor phenotype in the bone marrow, a primary site of tumor growth (Fig. 1c, Extended data Fig. 2). We used recipients devoid of endogenous T cells to better control the source of IFN-γ production. A fraction of OT-I T cells at the tumor site was found to produce IFN-γ (Fig. 1d). As shown in Fig. 1e-f, the presence of T cells resulted in an increased expression of MHC class I and PD-L1 on tumor cells. Phenotypic changes were also detected in tumor infiltrating immune cells that expressed increased levels of MHC class I on their surface (Fig. 1g). All these effects appeared to be dependent on T cell -derived IFN-γ as they were abolished when IFN-γdeficient OT-I T cells were transferred. Importantly, these results were recapitulated in Rag2 -/-γ c -/-recipients that lack NK cells, suggesting that T cell-derived IFN-γ is sufficient to directly mediate phenotypic changes in tumor cells (Extended data Fig. 3). To extend these findings in a setting of Chimeric Antigen Receptor (CAR) T cell therapy, we analyzed the phenotypic changes in lymphoma cells upon transfer of either IFN-γ sufficient or deficient anti-CD19 CAR T cells. The presence of WT but not IFN-γ-deficient CAR T cells resulted in a substantial increase in MHC class I and PD-L1 expression (Fig. 1h). Altogether, these results indicate that IFN-γ produced by T or CAR T cells have a broad impact on tumor and infiltrating immune cells.

Tumor-infiltrating T cells selectively accumulate and arrest in tumor areas with cognate antigen expression

Production of IFN-γ by T cells has been associated with stable interactions with antigenpresenting targets [START_REF] Honda | Tuning of antigen sensitivity by T cell receptor-dependent negative feedback controls T cell effector function in inflamed tissues[END_REF] . However, it is unclear whether IFN-γ produced by T cells acts on engaged targets, on their close neighbors or more extensively in the tumor microenvironment. To address this question, we thought to create mosaic tumors in which only selective regions express a tumor antigen in order to spatially restrict IFN-γ production (Fig. 2a). We took advantage of our previous observation that tumor subclones tend to grow as anatomically segregated patches to create these disparate environments [START_REF] Milo | The immune system profoundly restricts intratumor genetic heterogeneity[END_REF] . When we injected a mixture of antigen-positive (OVA + ) and antigen-negative Eμ-myc tumors expressing distinct fluorescent proteins, we observed tumors patches with typical radius of 100-500 μm in the bone marrow (Fig. 2b). Using intravital imaging, we found that the majority of OT-I T cells selectively accumulated and arrested in antigen-positive areas where they displayed reduced velocity, increased confinement and formed long-lasting interactions with tumor cells (Fig. 2b-g, Extended data Fig. 4, and movie S1). The remaining T cells seen in antigen-negative tumor areas were rapidly migrating with no evidence for prolonged interactions (Fig. 2b-g, Extended data Fig. 4 and movie S1). Some of the stable contacts with antigen-positive tumors most likely represent sites of cytokine secretion as previous studies have linked T cell arrest to IFN-γ secretion [START_REF] Honda | Tuning of antigen sensitivity by T cell receptor-dependent negative feedback controls T cell effector function in inflamed tissues[END_REF] . Even if it remains formally possible that migrating T cells also release small amounts of cytokine, the increased T cell numbers and residence time in antigen-positive areas would imply higher cytokine release in these regions. Of note, T cell arrest in antigen-positive areas was by large IFN-γ-independent as prolonged interactions were also observed with IFN-γ-deficient T cells (Extended data Fig. 4). In sum, this strategy provides a mean to spatially control cognate interactions leading to IFN-γ production by T cells and to assess the occurrence of long-distance effects in antigennegative areas.

Strong bystander activity of T cell-derived IFN-γ in the tumor microenvironment

Using the aforementioned set-up, we asked how T cells impact the phenotype of tumor cells located in antigen-positive versus antigen-negative areas. Since IFN-γ is necessary and sufficient for MHC class I and PD-L1 upregulation, we relied on these readouts as a proxy for IFN-γ activity. We first used the Eμ-myc lymphoma model. When OT-I T cells were transferred in mice bearing antigen-negative tumors only, no changes in MHC class I and PD-L1 levels were observed (Fig. 3b-d) despite the presence of T cells in the tumor microenvironment (Extended data Fig. 4e). This confirmed that local antigen restimulation is required for IFN-γ production. In mosaic tumors, the presence of T cells resulted in an increased expression of MHC class I and PD-L1. Importantly, both antigen-positive and antigen-negative tumor cells upregulated these markers to the same extent (Fig. 3a-d). The widespread effects mediated by IFN-γ could be due to the activity of numerous T cells, each of them acting on a limited number of bystander cells (Fig. 3e). Alternatively, efficient diffusion of IFN-γ may result in a relatively uniform intratumoral cytokine concentration leading to a homogeneous response which intensity depends on the number of producing T cells (Fig. 3e). To distinguish between these two possibilities, we repeated our experiments with different numbers of T cells. We reasoned that if T cells act on a limited number of tumor cells, lowering the number of T cells should reduce the frequency of responding tumor cells without affecting the intensity of their response (best modeled by a bimodal distribution in tumor cell response) (Fig. 3e). By contrast, if IFN-γ primarily acts as a shared resource, reducing the number of T cells should decrease the intensity of the response in all cells (unimodal distribution of tumor cell response). Supporting the latter hypothesis, we found that a 10-fold decrease in the number of transferred T cells resulted in a homogeneous reduction in MHC class I upregulation (Fig. 3f). Altogether, these experiments support the idea that IFN-γ diffuses extensively in the tumor microenvironment. As a result, the collective activity of IFN-γ producing T cells generate a widespread field of IFN-γ whose concentration dictates phenotypic changes across the whole tumor. We measured that when transferring 20x10 [START_REF] Chin | Cell growth arrest and induction of cyclin-dependent kinase inhibitor p21 WAF1/ CIP1 mediated by STAT1[END_REF] T cells the ratio of OVA + Eμ-myc to OT-I T cells was approximately 100:1. By comparing the fold increase in MHC class I levels observed in vivo to that seen in vitro (Fig. 1a), we estimated that this T cell density resulted in an intratumoral concentration of IFN-γ of around 1 ng/mL (Extended data Fig. 5a-b). This is obviously a rough estimate as several parameters could potentially differ between the in vitro and in vivo settings (such as cytokine consumption or basal IFN-γR expression). To test whether the activity of IFN-γ may influence tumor fate, we first assessed the effect of systemic IFN-γ delivery. We observed phenotypic changes in tumor cells suggesting effective diffusion in vivo and most importantly found reduced tumor burden in the bone marrow of treated mice (Extended data Fig. 5c-d). This result suggests, that in contrast to what was seen in vitro with Eμ-myc cells, IFN-γ exerts a negative effect on tumor development in vivo. This discrepancy may originate from an indirect effect of IFN-γ on host cells or from changes in IFN-γR expression/signaling on tumor cells in vivo. To test how T cell-derived IFN-γ may affect tumor growth in vivo, we relied on our mosaic tumor system and compared the impact of T cell-derived IFN-γ on the growth of both antigen positive and negative tumors. When recipient mice were transferred with IFN-γ competent T cells, we noted that antigenpositive (OVA + ) cells were eliminated but also that the antigen-negative tumor was partly controlled (Extended data Fig. 5e-f). This partial control was lost upon transfer of IFN-γ deficient T cells (Extended data Fig. 5e-f), supporting the idea that bystander IFN-γ activity may help limit tumor growth in vivo.

To extend these findings to solid tumors, we generated mosaic tumors using the B16.F10 melanoma model. In this set-up, we again observed the hallmarks of IFN-γ signaling in both antigen-positive (OVA + B16) and antigen-negative (B16) cells in the presence of T cells (Fig. 3g-i). Thus, extensive bystander activity of T cell-derived IFN-γ is a property that also pertains to solid tumors.

Widespread and sustained STAT1 signaling in the tumor microenvironment

The observed bystander changes in tumor phenotype prompted us to better characterize the spatiotemporal dynamics of IFN-γ signaling in the tumor microenvironment. We focused on STAT1 translocation in tumor cells, as this represents an early event after cytokine exposure. To this end, we generated a fluorescent reporter combining a STAT1-GFP fusion protein and a nuclear mCherry protein, and introduced this probe into Eμ-myc lymphoma cells. As expected, in the absence of stimulation, STAT1-GFP was cytoplasmic and excluded from the nucleus (Fig. 4a). Upon in vitro addition of IFN-γ, a more uniform (cytoplasmic and nuclear localization) distribution of STAT1-GFP was rapidly observed confirming the translocation of a fraction of the STAT1-GFP pool (Fig. 4a). To quantify STAT1 translocation in multiple cells, we designed an automated image analysis procedure based on the detection of cell and nuclear shape and computing of Pearson correlation coefficients between STAT1-GFP and nuclear mCherry signals (Extended data Fig. 6). With this strategy, statistically distinct translocation scores were observed for cells in the presence or absence of IFN-γ (Fig. 4b).

We therefore used these distinct patterns of STAT1 localization to identify cells with ongoing cytokine signaling in vivo by two-photon imaging. In a first model, we analyzed the response of H-Y + B cell lymphoma in response to MataHari effector CD8 + T cells which express a H-Y specific transgenic TCR. In female Rag2 -/-mice with established tumor expressing the fluorescent reporter, we predominantly observed a cytoplasmic-only distribution of STAT1-GFP, suggesting an absence of IFN-γ signaling (Fig. 4c-e). However, in mice that were also transferred with tumor-reactive MataHari effector CD8 + T cells, we observed evidence of cytokine signaling in a large number of tumor cells as detected by STAT1 translocation. This was evident from two-photon images (Fig. 4c-d) and confirmed by automated computing of STAT1 translocation scores (Fig. 4e). Importantly, STAT1 translocation was widespread and detected in tumor cells irrespective of whether they were contacting T cells or not. Consistently, there was no correlation between STAT1 translocation score in individual tumor cells and distance to the nearest T cells (Fig. 4f). Nuclear STAT1-GFP typically persisted throughout the imaging experiments (typically 1-2 hr) suggesting sustained signaling (Fig. 4g, Movie S2). To extend these results, we repeated these experiments in a second model relying on OVA + Eμ-myc B cell lymphoma and OT-I CD8 + T cells. In this set-up also, nuclear STAT1-GFP was detected in the vast majority of tumor cells when OT-I effector CD8 + T cells were transferred but not in the absence of T cells (Extended data Fig. 7a-d, Movie S3). To further demonstrate the ability of T cellderived IFN-γ to signal in distant tumor cells, we created mosaic tumors containing antigennegative tumor that expressed the STAT1-GFP reporter and fluorescently labeled antigenpositive tumor cells. As shown in Extended data Fig. 7e-h, we detected increased STAT1 activity in antigen-negative tumor cells when T cells and antigen-positive tumor cells were present as compared to controls lacking antigen-positive tumor cells.

These results establish that STAT1 activity occurs at the tissue level rather than in discrete spots and indicate that STAT1 translocation does not require ongoing interactions with effector T cells.

Sustained IFN-γ signaling is required for alteration of tumor cell phenotype

Early events of IFN-γ signaling including STAT1 phosphorylation and STAT1 translocation are typically detected within minutes of cytokine exposure [START_REF] Oyler-Yaniv | Catch and Release of Cytokines Mediated by Tumor Phosphatidylserine Converts Transient Exposure into Long-Lived Inflammation[END_REF] . However, it is unclear whether transient or prolonged exposure to IFN-γ is required to alter tumor cell phenotype. We therefore designed an experiment in which Eμ-myc B lymphoma cells are cultured for a fixed period of time (24h) but exposed to IFN-γ during this period for variable amounts of time (by washing out the cytokine and adding a blocking anti-IFN-γ mAb to prevent any residual effects). RNAseq was then performed for these different stimulation patterns (Fig. 5a). We confirmed that sustained IFN-γ exposure resulted in prolonged STAT1 translocation (Extended data Fig. 8). As shown in Fig. 5b-c, the core of the response that included wellknown IFN-γ responsive genes (Cd74, Cd274, H2-K1, H2-D1…), required at least 6-24h of cytokine exposure. Notably, cells exposed to IFN-γ for one hour were barely distinguishable for unstimulated cells. We confirmed these findings at the protein level for H2-K b , H2-D b and PD-L1 using flow cytometry (Fig. 5d). In these settings, 6-24h of cytokine exposure was needed for substantial upregulation of these molecules. The need for long-lasting cytokine exposure was not restricted to B lymphoma cells as we observed a similar requirement for B16.F10 melanoma cells (Fig. 5e). Altogether, these results suggested that sustained exposure to IFN-γ is needed to drive phenotypic changes in tumor cells, including MHC class I and PD-L1 upregulation. Our transcriptomic analysis also revealed a strong enrichment of IRF8 and STAT1 motifs and that the expression of both transcription factors, IRF8 and STAT1, increased in response to IFN-γ (Fig. 5f-g). This raises the possibility that a two-step signaling activation, in which STAT1 and IRF8 upregulation by IFN-γ subsequently potentiates the cellular response to this cytokine, accounts for the requirement for sustained cytokine exposure.

Probing IFN-γ activity in human melanoma

To test whether bystander IFN-γ activity also occurs in human tumors, we re-analyzed single-cell RNA-seq data from the immune infiltrate of 8 non-treated melanoma patients [START_REF] Li | Dysfunctional CD8 T Cells Form a Proliferative, Dynamically Regulated Compartment within Human Melanoma[END_REF] . Overall, we found that CD8 + T cells were the main producers of IFN-γ as more than 80% of all IFN-γ mRNA molecules were specific to those cells (Fig. 6a). We decided to investigate the impact of IFN-γ production on the tumor microenvironment by focusing on the monocyte/macrophage population as a sensor of intratumoral IFN-γ. Of note, the low number of tumor cells present in the samples precluded us from performing the same analysis on those cells. By performing over-dispersion analysis [START_REF] Fan | Characterizing transcriptional heterogeneity through pathway and gene set overdispersion analysis[END_REF] , we identified one module (noted as pathway 7, Extended data Fig. 9) specifically enriched in IFN-γ responsive gene (Fig. 6b). Remarkably, the mean IFN-γ signature in monocyte/macrophages was significantly correlated (R=0.91, p=0.0018, Fig. 6c) with the frequency of IFN-γ producing CD8 + T cells. Similarly, when repeating the analysis with intratumoral neutrophils, we observed an IFN-γ signature that was correlated with the fraction of IFN-γ producing T cells (Fig. 6e-f). These data suggest that, as observed in our mouse models, IFN-γ signals in multiple cell types of the tumor microenvironment. The observation that neutrophils, which are not typical tumor antigen-presenting cells, also harbor the IFN-γ signature suggests that they have received IFN-γ signals in a bystander fashion.

We also noted that in patients with a high density of IFN-γ producing T cells, the distribution of IFN-γ signature on monocyte and neutrophils appeared unimodally shifted toward higher values (Fig. 6g). This observation is compatible with relatively uniform IFN-γ activity in the tumor which was also observed in head and neck squamous cell carcinoma patients, as analyzed by single cell RNA sequencing in tumor cells (Extended data Fig. 10). Future studies assessing the spatial distribution of IFN-γ responsive gene in relation to T cell infiltration will help refine our understanding of cytokine propagation and signaling in human tumors.

Discussion

In the present study, we established the spatiotemporal dynamics of intratumoral T cellderived IFN-γ in models of B cell lymphoma and melanoma. We provided evidence that the collective activity of effector T cells together with cytokine diffusion drive a tumor-wide response. Moreover, the formation of a sustained field of IFN-γ was required to alter tumor cell phenotype.

The range of cytokine activity is dependent on the production rate and levels, diffusion and consumption by other cells [START_REF] Altan-Bonnet | Cytokine-mediated communication: a quantitative appraisal of immune complexity[END_REF] . Our results support the idea that T cell production of IFN-γ generates a widespread cytokine field shared by the majority of tumor cells and infiltrating immune cells. MHC class I expression was uniformly upregulated in the tumor microenvironment even in cells located at distance (>100 μm) from the sites of cognate T cell interactions, a feature that may favor tumor lysis by cytotoxic T cells. Conversely, the expression of the inhibitory ligand PD-L1 was also upregulated by IFN-γ in a bystander fashion. Thus, both potentially beneficial and deleterious effects of IFN-γ appeared to be regulated by a cytokine field, rather than by very discrete cytokine hotspots.

At first, these results may appear at odds with the well-established notion that IFN-γ is 20 33 34 produced by T cells at the immunological synapse with target cells [START_REF] Huse | T cells use two directionally distinct pathways for cytokine secretion[END_REF][START_REF] Kupfer | Polarized expression of cytokines in cell conjugates of helper T cells and splenic B cells[END_REF][START_REF] Egen | Intravital imaging reveals limited antigen presentation and T cell effector function in mycobacterial granulomas[END_REF] , a phenomenon that should generate high local concentrations of IFN-γ at sites of antigen recognition and create extensive heterogeneity in cell responses. Our finding that prolonged exposure to IFN-γ is required to alter tumor cell phenotype likely provides an explanation for this apparent contradiction as these hotspots of cytokine will likely dissipate due to diffusion well before they have the time to generate a local biological effect.

A direct consequence for this mode of action is that the activity of IFN-γ will be determined at the tissue level by the overall number of cytokine-producing cells. In this model, the collective activity of T cells is essential as it determines the overall cytokine concentration and the extent of tumor cell response, in a manner resembling quorum sensing. Such a collective mode of action was recently described for nitric oxide-producing macrophages, where a single macrophage has a negligible intrinsic activity but a high number of NOproducing macrophages regulates activity at the tissue level in both NO-producing and nonproducing cells [START_REF] Olekhnovitch | Collective nitric oxide production provides tissuewide immunity during Leishmania infection[END_REF][START_REF] Postat | A metabolism-based quorum sensing mechanism contributes to termination of inflammatory responses[END_REF] .

It has recently been shown that IFN-γ can bind to phosphatidylserine on the surface of tumor cells and be slowly released to prolong inflammation [START_REF] Oyler-Yaniv | Catch and Release of Cytokines Mediated by Tumor Phosphatidylserine Converts Transient Exposure into Long-Lived Inflammation[END_REF] . Such mechanism, termed catch-and-release may favor the formation of relatively stable cytokine fields by temporally averaging cytokine availability in the tumor microenvironment. Binding of IFN-γ on the extracellular matrix should also achieve similar effects. It will be important to determine whether other cytokines rely on a similar global mode of signaling and assess how the superimposition of distinct cytokine fields may translate into complex biological responses at the tissue level, in particular in the context of tumor microenvironment. This is in particular relevant for TNF-α that can act synergistically with IFN-γ to induce tumor cell death. Cytokine consumption plays an important role in shaping the spatial range of cytokine activity [START_REF] Oyler-Yaniv | A Tunable Diffusion-Consumption Mechanism of Cytokine Propagation Enables Plasticity in Cell-to-Cell Communication in the Immune System[END_REF] and may generate fields of different sizes for distinct cytokines, possibly creating functional heterogeneity in the tissue.

In summary, our work provides new insights into the spatiotemporal activity of intratumoral IFN-γ with important implication for anti-tumor immune responses. Single-cell RNA sequencing, as illustrated here, but also spatial transcriptomics [START_REF] Stahl | Visualization and analysis of gene expression in tissue sections by spatial transcriptomics[END_REF][START_REF] Shah | seqFISH Accurately Detects Transcripts in Single Cells and Reveals Robust Spatial Organization in the Hippocampus[END_REF][START_REF] Wang | Three-dimensional intact-tissue sequencing of single-cell transcriptional states[END_REF] will help estimate the range of cytokine activity in human tumors. Finally, the use of systems immunology, together with the development of new probes to image cytokine propagation and activity in vivo, will be essential to establish the rules of cytokine-mediated communication at steadystate or during disease pathogenesis.

Methods

Mice and cell lines 6-8-week-old C57BL/6 mice were purchased from Charles River. Rag2 -/-, Rag2 -/-γ c -/-, Rag1 -/-OT-I TCR, UBC-GFP Rag1 -/-OT-I TCR, CFP (CFP expressed under the actin promoter), Rag2 -/-MataHari TCR and Ifng -/-mice were bred and crossed in our animal facility. All experiments were performed in agreement with relevant guidelines and regulations and approved by the Institut Pasteur Committee on Animal Welfare (CETEA) under protocol code 170038. A lymphoma cell line was isolated from tumor-bearing male Eμ-myc transgenic mice [START_REF] Harris | The E mu-myc transgenic mouse. A model for high-incidence spontaneous lymphoma and leukemia of early B cells[END_REF] , that develop spontaneous Burkitt-like lymphomas. This cell line was retrovirally transduced to express either CFP, GFP, YFP, mCFP, mCherry-NLS fluorescent proteins, ovalbumin, or STAT1-GFP fusion protein (all cloned in MSCV backbone). B16.F10 (B16) melanoma cells were kindly provided by Guy Shakhar.

Flow cytometry and antibodies

For ex vivo analyses, femurs and tibias were isolated from Eμ-myc tumor-bearing mice.

Bone marrow cells were extracted by flushing the bones with PBS supplemented with 1mM EDTA and 0,5% FBS (FACS Buffer), and single-cell suspensions were prepared by filtering the cells through 70 μm cell strainers. For B16 tumor-bearing mice, tumors were harvested and cells were recovered by digesting the tumors with collagenase/DNAse for 30 min at 37°C, before crushing them on 70 μm cell strainers. All cells were then stained in FACS Buffer, supplemented with 4% normal mouse and rat sera and/or 10 μg.mL -1 anti-mouse CD16/32 (Fc Block), using a combination of fluorescent or biotin-labeled monoclonal antibodies among : CD8 (53-6.7, BD Biosciences), CD11b (M1/70, BioLegend), CD45 (30-F11, BD Biosciences), H2-K b (AF6-88.5, BioLegend), H2-D b (KH95, BioLegend), Ly6C (HK1.4, BioLegend), Ly6G (1A8, BioLegend), NK1.1 (PK136, BioLegend), PD-L1 (10F.9G2 or MIH5, BioLegend or BD Biosciences respectively). IFN-γ intracellular stainings were performed using the Cytofix/Cytoperm kit (BD Biosciences) according to the manufacturer's guidelines, and XMG1.2 mAb (eBiosciences). IFN-γ expression was assayed in the presence of 1 μg.mL -1 Brefeldin A. Samples were analyzed using a BD FACSCanto II or a Cytoflex LX (Beckman Coulter) flow cytometer. Data analysis was performed using FlowJo v10.4.1 (Tree Star).

Tumor models and T cell adoptive transfer

B cell lymphomas were established by injecting 1x10 [START_REF] Chin | Cell growth arrest and induction of cyclin-dependent kinase inhibitor p21 WAF1/ CIP1 mediated by STAT1[END_REF] Eμ-myc cells intravenously in the indicated recipients. Melanoma tumors were established by inoculating 2x10 6 B16 cells subcutaneously. Splenocytes from MataHari TCR transgenic mice or OT-I TCR transgenic mice were isolated and red blood cells were removed by ACK lysis. One third of the cells was then pulsed with 50μM of H-2D b -restricted Uty246-254 peptide (WMHHNMLDI) or H-2K b -restricted OVA 257-264 peptide (SIINFEKL) for 2 hours in RPMI medium 1640-Glutamax™ at 37°C. The rest of the cells was incubated at 37°C in complete medium supplemented with 50 μg.mL -1 gentamycin. The two populations were then mixed and cultured for 2 to 3 days. Cells were then subjected to Ficoll gradient centrifugation to remove dead cells and cultured in complete medium containing human interleukin-2 (25 IU/mL; Roche or R&D). For proper in vitro activation, IFN-γ-deficient T cells were supplemented with 100 ng.mL -1 IFN-γ (Invitrogen). Cells were used on days 4-8, at which time >95% were CD8 + T cells. When indicated, T cells were retrovirally transduced to express mCherry as described [START_REF] Cazaux | Single-cell imaging of CAR T cell activity in vivo reveals extensive functional and anatomical heterogeneity[END_REF] . Tumor-bearing mice were adoptively transferred with cultured T cells by intravenous injection of 20x10 6 cells. For experiments using CAR T cells, B cell lymphoma injection, conditioning and CAR T cell transfer were performed as previously described [START_REF] Cazaux | Single-cell imaging of CAR T cell activity in vivo reveals extensive functional and anatomical heterogeneity[END_REF] .

Intravital two-photon imaging

Bone marrow imaging was performed as described [START_REF] Cazaux | Single-cell imaging of CAR T cell activity in vivo reveals extensive functional and anatomical heterogeneity[END_REF] . Mice were anaesthetized with a mixture of xylazine (Rompun®, 10 mg/kg) and ketamine (Imalgène®, 100 mg/kg), which was replenished hourly. During imaging, the mice temperature was maintained at 37°C with a heating pad. Two-photon imaging was performed with an upright microscope FVMPE-RS

Image analysis

Movies were processed and analyzed with Imaris software (Bitplane) or Fiji software (ImageJ 1.50 i). Movies and figures based on two-photon microscopy are shown as 2D maximum intensity projections of 3D data. For optimal contrast rendering, GFP was pseudocolored in yellow in some images. Contact durations were calculated manually based on close apposition between a given T cell and a tumor cell. To perform an un-biased analysis of STAT1-GFP translocation, we developed an in-house Matlab® script that first automatically detects cell envelope and cell nuclei and then computes a translocation score. Briefly both STAT1-GFP and nuclear mCherry channels are first binarized using Otsu's method. Image closing with a circular structuring element is then applied to the STAT1-GFP binarized image, followed by a filling of the holes. Euclidean distance transform is then applied to both binarized images before using watershed segmentation tool to segment the contiguous cells. For each identified cell, Pearson correlation between STAT1-GFP and mCherry signal is computed and considered as a good proxy for STAT1-GFP translocation (translocation score). This strategy was applied to both in vitro and in vivo data, however due to the increased background signal in in vivo images, we added a step of contrast enhancement before image binarization: first a Laplacian Of Gaussian (LOG) filter with a sigma parameter of size equal to the estimated size of the cells is used, followed by the extraction of the local peak with a h-dome transform [START_REF] Vincent | Morphological grayscale reconstruction in image analysis: applications and efficient algorithms[END_REF] . Whenever the automated segmentation was not sufficiently accurate, we performed manual cell segmentation and computed the Pearson correlation coefficient using the ROI manager and coloc2 Fiji plugins.

mRNA sequencing and analysis mRNA sequencing was performed on cultured Eμ-myc cells (New York Genome Center). mRNAs were extracted using the RNeasy® Mini Kit (Qiagen) according to the manufacturer's guidelines. RNA sequencing libraries were prepared using the TruSeq Stranded mRNA Library Preparation Kit in accordance with the manufacturer's instructions. Briefly, 500ng of total RNA was used for purification and fragmentation of mRNA. Purified mRNA underwent first and second strand cDNA synthesis. cDNA was then adenylated, ligated to Illumina sequencing adapters, and amplified by PCR (using 10 cycles). Final libraries were evaluated using fluorescent-based assays including PicoGreen (Life Technologies) or Qubit Fluorometer (invitrogen) and Fragment Analyzer (Advanced Analytics) or BioAnalyzer (Agilent 2100) and were sequenced on an Illumina HiSeq2500 sequencer (v4 chemistry) using 2 x 125bp cycles.

rRNA abundance was measured by mapping with Bowtie2. Quality control was performed using Picard and RSeQC. The reads were aligned with STAR (version 2.5.2a), and genes annotated in Gencode vM16 were quantified with featureCounts (v1.4.3-p1). Normalization and differential expression was done with the Bioconductor package DESeq2 (v1.18.1).

Differential expression analysis as well as normalization were performed using the DESeq2 package (version 1.18.1) on R (version 3.4.3). Analysis was performed according to the recommendation of the package with the same parameter excepting the log2 fold change shrinkage method parameter that was set to 'apeglm'. A gene was considered as significantly differentially expressed if its corresponding adjusted p-value was lower than 0.01 and its absolute shrinked log2 fold change bigger than 1. Expression heatmap was performed using the pheatmap package with Ward's criterion used for the hierarchical clustering.

Enrichment of specific Transcription Factor (TF) binding motifs in the promoter of upregulated genes was performed using the iRegulon Cytoscape plugin [START_REF] Janky | iRegulon: from a gene list to a gene regulatory network using large motif and track collections[END_REF] . Mus musculus was chosen as the reference species and all other parameters were set to default except the putative regulatory region and motif ranking databases that were set to '500 bp upstream'.

Single cell RNA-Seq Data Clustering and Analysis

The two single cell RNA-Seq datasets were downloaded on Gene Expression Omnibus repository (accession numbers GSE123139 and GSE103322 for the Melanoma and HNSCC data respectively). Annotations were loaded using the GEOquery package (version 2.48.0). As the HNSCC data were already processed (provided as Log(TPM+1) data), we used two slightly different settings for data pre-processing. In the case of the melanoma data, we processed as follows. In order to avoid removing populations of small-sized cells, a relatively low UMI threshold (350 UMIs) was used in the first instance to filter out low quality cells. Genes with less than 300 UMIs expressed were also removed to reduce computation cost and time. The filtered data were then analyzed using PAGODA2 pipeline (https://github.com/hms-dbmi/pagoda2) [START_REF] Lake | Integrative single-cell analysis of transcriptional and epigenetic states in the human adult brain[END_REF] . Expression values and variance were scaled as described in [START_REF] Lake | Integrative single-cell analysis of transcriptional and epigenetic states in the human adult brain[END_REF] . It is important to note that while the first version of PAGODA was relying on gene set analysis to perform cell clustering, PAGODA2 performs most of the analytical steps without any gene set analysis. The general approach used is similar to the one described at https://github.com/hms-bmi/pagoda2/blob/master/vignettes/ pagoda2.walkthrough.oct2018.md/. First gene variance was corrected and highly variable genes identified using the adjustVariance function (gam.k=10). The 100 first components of Principal Component Analysis based on the 3000 most variable genes were then computed using the calculatePcaReduction function. Then a K-Nearest-Neighbors (KNN) graph was computed based on the PCA reduction through the makeKnnGraph function (k=30). Cosine distance was used at this stage to increase the robustness of the analysis. Lastly Louvain community detection clustering was performed on the KNN graph using the getKnnClusters function and the multilevel.community functions from the igraph R package. Once the cells were clustered, we identified all genes that were significantly expressed by cells from a given cluster compared to all other cells identified using the PAGODA2 function getDifferentialGenes that performs Kruskal-Wallis test, with default parameters (corrected Z-score > 3, upregulated.only=TRUE). Using this gene list, we annotated the various cell cluster based on the specific expression of well-known immune marker (CD3, CD8A, TRBC2 etc..). Low dimensional embedding of the data was performed using UMAP algorithm [START_REF] Mcinnes | UMAP: uniform manifold approximation and projection for dimension reduction[END_REF] . The uwot package implementation was used (https://github.com/jlmelville/ uwot). PCA dimensionality reductions computed using PAGODA2 was used as input data.

N_neighbors parameter was set to 40, spread parameter to 4 and cosine metric was used. In the case of the HNSCC, the data were already normalized and were loaded into a Pagoda2 object with parameter 'modelType' set to 'raw'. For the next steps of the analysis, the 50 first PCs were used, the 1000 most variables genes were selected, K parameter was set to 15 and cosine distance used.

Analysis of Over-Dispersed Pathways

In order to detect biologically meaningful gene modules/pathways in a robust manner, we performed pathway over-dispersion analysis using de-novo gene-sets. We first selected a given cell population and performed gene expression and variance normalization on the raw UMI counts as described above. The variable genes were selected using the getOdGenes function from PAGODA2 package with the 200 top genes selected. These genes were then grouped into 15 pathways using hierarchical clustering (Ward linkage and correlation-based distance). Over-dispersion of these pathways was computed using the testPathwayOverdispersion function from the same package with default parameters. Gene set enrichment was performed through a binomial test (binom.test function, alternative = 'greater') using the 200 most variable genes as the reference 'universe'. The 'HALLMARK INTERFERON ALPHA RESPONSE' and 'HALLMARK INTERFERON GAMMA RESPONSE' gene sets were downloaded from the GSEA website (http:// software.broadinstitute.org/gsea/msigdb/cards). Multiple testing correction was performed using the Benjamini-Hochberg method (p.adjust R function).

Association between IFNG score and IFNG production

In the case of melanoma dataset, the macrophage population was identified as the C1Q/ APOE expressing cluster. IFNG score was computed using the testPathwayOverdispersion function as described above for the macrophage population and it mean value computed for each patient. Neutrophils identified as the S100A8/S100A9/Lyz expressing cluster. Patients with less than 25 macrophages or neutrophils were removed from the analysis. The fraction of IFNG-expressing CD8 + T cells among T cells was computed by computing the number of cells annotated as coming from a 'CD3' gating that belongs to a CD8 + T cell RNA-seq cluster and has more than zero UMI coming from the IFNG gene. Association between the two was assessed using a simple linear regression model (lm R function). Significance of the association was checked using Fisher's test (anova R function). Correlation was estimated with the cor R function. For head and neck squamous cell carcinoma samples, patients with less than 50 tumor cells in the dataset were removed from the analysis.

Statistics and reproducibility

All statistical tests were performed using GraphPad Prism v.

  OLYMPUS) and a 25x/1.05 NA water-dipping objective (OLYMPUS). Excitation was provided by an Insight deep see dual laser (Spectra physics) tuned at 920 or 960 nm. The following filters were used for fluorescence detection: CFP (483/32), GFP (520/35), YFP (542/27), background (593/35) and mCherry (624/40). To create time-lapse sequences, we typically scanned a 30 to 40 μm-thick volume of tissue at 5 μm Z-steps and 60s intervals.

6 .

 6 Points in graphs indicate individual mice, and lines indicate means. In bar graphs, bars indicate means, and error bars indicate sample standard deviation (SD). Statistical tests employed are detailed in figure legends. No statistical method was used to predetermine sample size. No data were excluded from the analyses. The experiments were not randomized. The investigators were not blinded to allocation during the experiments and outcome assessment.

Figure 1 .

 1 Figure 1. T cell-derived IFN-γ induces phenotypic changes in the tumor micro-environment. a-b IFN-γ induces phenotypic changes in tumor cells in vitro. Eμ-myc B lymphoma (a) or B16.F10 melanoma (b) cells were stimulated with indicated IFN-γ concentrations in vitro for 24h. H2-K b (left), H2-D b (middle) and PD-L1 (right) surface expression was then analyzed by flow cytometry. Each dot represents the mean of 3 technical replicates. Representative of 2 (B16.F10) or 3 (Eμ-myc) independent experiments. c-g T cell-derived IFN-γ increases MHC class I and PD-L1 levels in tumor and tumor-infiltrating immune cells. c Experimental set-up. Rag2 -/-mice were injected i.v with OVA-expressing Eμ-myc B

Figure 2 .a

 2 Figure 2. Tumor antigen expression drives the selective accumulation and arrest of intratumoral T cells a Experimental set-up. Rag2 -/-mice were injected with a 1:1 mixture of OVA-expressing and non-expressing Eμ-myc B lymphoma cells, labeled with CFP and YFP, respectively. On day 12-13, mice were injected with activated GFP + OT-I T cells. Two days later, intravital imaging of the bone marrow was performed. Scale bar: 50μm b-g CD8 + OT-I T cells specifically accumulate and arrest in antigen-expressing cellular patches of mosaic tumors. b Left. Representative image of OVA + (blue) or OVA -(orange) tumor patches infiltrated with

Figure 3 .

 3 Figure 3. Extensive bystander IFN-γ activity in the tumor microenvironment a-f Bystander activity of IFN-γ in the tumor microenvironment of B cell lymphomas. Rag2 -/-mice were injected with either Eμ-myc alone or a 1:1 mixture of Eμ-myc and OVAexpressing Eμ-myc B lymphoma cells (labeled with different fluorescent proteins). On day 12-13, recipients were injected with OT-I T cells or left untreated. Two days later, the bone marrow of the mice was recovered and analyzed by flow cytometry. a Representative histograms of H2-K b and PD-L1 surface expression on OVA-expressing (left) or nonexpressing (right) Eμ-myc B lymphoma cells isolated from mice bearing mosaic tumors and

Figure 4 .

 4 Figure 4. T cells mediate widespread and sustained STAT1 activity in the tumor microenvironment. a-b Male Eμ-myc B lymphoma cells were transduced to express a STAT1-GFP reporter and a nuclear mCherry protein. Measurement of STAT1 activity on Eμ-myc B cells cultured in the absence or presence of recombinant IFN-γ. a Images and graphs depict the quantification of STAT1-GFP (yellow) and nuclear mCherry (red) fluorescence intensity across the indicated lines for a representative cell prior to (upper panels) or after (lower panels) IFN-γ exposure. Scale bar, 5 μm. b Translocation score was computed automatically

Figure 5 .a

 5 Figure 5. Sustained signaling is required to alter tumor cell phenotype. a Experimental set-up. Eμ-myc B lymphoma cells were stimulated with 5 ng.mL -1 IFN-γ. At 1h or 6h, the stimulation was blocked by adding 50 μg.mL -1 anti-IFN-γ mAb. At 24h, cells were recovered for mRNA sequencing. b Heatmap of differentially expressed genes. Gene expression is normalized by row. c Venn diagrams of differentially expressed genes between the various stimulated and control samples. d-e Tumor cells were stimulated with 5 ng.mL -1 IFN-γ. At 1h or 6h, the stimulation was blocked by adding 50 μg.mL -1 anti-IFN-γ mAb. At 24h, cells were recovered and analyzed by flow cytometry. Graphs represent H2-K b

Figure 6 .

 6 Figure 6. Assessing IFN-γ bystander activity in human melanoma samples Measurement of IFNG signature in the tumor microenvironment of melanoma patients using single-cell RNA-seq. a Contribution of distinct cell clusters to IFNG production. Data are compiled from all patients analyzed (n=8). b Gene contribution to the IFNG signature in the monocyte/macrophage cluster. c Association between the percentage of IFNG + cells among CD8 + cells and the IFNG score in monocytes/macrophages. (R represents Pearson's correlation coefficient, statistical significance was assessed using a Fisher test). d Distribution of IFNG signature in monocytes/macrophages from three different patients

Nat Cancer. Author manuscript; available in PMC 2020 September 01.

Nat Cancer. Author manuscript; available in PMC 2020 September 01.Europe PMC Funders Author Manuscripts

Acknowledgments

We thank members of the Bousso laboratory for critical review of the manuscript. We acknowledge the mouse facility and Technology Core of the Center for Translational Science (CRT) at Institut Pasteur for support in conducting this study. The work was supported by Institut Pasteur, Inserm, a Starting (Lymphocytecontact) and an Thibaut et al. Page 23 Nat Cancer. Author manuscript; available in PMC 2020 September 01. Europe PMC Funders Author Manuscripts Europe PMC Funders Author Manuscripts Advanced grant (ENLIGHTEN) from the European Research Council (P.B) and by the Bristol-Myers Squibb Foundation for Research in Immuno-Oncology.

The data that support the findings of this study are available from the corresponding author upon reasonable request. RNA-seq data reported in this paper are deposited in NCBI Gene Expression Omnibus (GEO) database (GSE140191). Previously published human single-cell RNA sequencing data that were re-analysed here are available under accession codes GSE123139 and GSE103322. All other data supporting the findings of this study are available from the corresponding author on reasonable request.

Code availability

The code for automated quantification of STAT1 translocation is available on GitHub (https://github.com/PierreBSC/ IFNG_Cancer_project)

Competing interests statement

The authors declare no competing interests.

Reporting Summary

Further information on research design is available in the Nature Research Reporting Summary linked to this article.

Extended Data

Thibaut et al.

Supplementary Material

Refer to Web version on PubMed Central for supplementary material.


Charles-Olivier Artizzu
email: artizzu@i3s.unice.fr

Haozhou Zhang

Guillaume Allibert
email: allibert@i3s.unice.fr

Cédric Demonceaux
email: cedric.demonceaux@u-bourgogne.fr

OmniFlowNet: a Perspective Neural Network Adaptation for Optical Flow Estimation in Omnidirectional Images

Spherical cameras and the latest image processing techniques open up new horizons. In particular, methods based on Convolutional Neural Networks (CNNs) now give excellent results for optical flow estimation on perspective images. However, these approaches are highly dependent on their architectures and training datasets. This paper proposes to benefit from years of improvement in perspective images optical flow estimation and to apply it to omnidirectional ones without training on new datasets. Our network, OmniFlowNet, is built on a CNN specialized in perspective images. Its convolution operation is adapted to be consistent with the equirectangular projection. Tested on spherical datasets created with Blender 1 and several equirectangular videos realized from real indoor and outdoor scenes, OmniFlowNet shows better performance than its original network without extra training.

I. INTRODUCTION

With the arrival on the market of more affordable and more accurate 360 cameras, their use is proliferating. The latest generation of cameras, such as Instax 360 One X, Ricoh Theta Z1, and Samsung Gear 360, offers representative spherical images with up to 4K resolution. Thanks to their wide field of view, they can capture their entire environment in a single image. These sensors are therefore used in many mobile robotics tasks, such as image-based navigation [START_REF] Hirose | Deep Visual MPC-Policy Learning for Navigation[END_REF], monitoring [START_REF] Saitoh | A 2d-3d integrated interface for mobile robot control using omnidirectional images and 3d geometric models[END_REF] or simultaneous visual localization and mapping (SLAM) [START_REF] Caruso | Large-scale direct SLAM for omnidirectional cameras[END_REF].

The equirectangular projection is a simplified representation of images taken by spherical cameras, particularly to facilitate human interpretation. In this projection, the latitude and longitude of spherical images are projected in horizontal and vertical coordinates on a 2D plane. The equirectangular projection is different from classical perspective images and suffers from important distortions in the vicinity of the polar regions, as shown in Fig. 1. The methods developed for perspective have to be rethought to fit with the distortions. Distortions imply that an object will appear differently depending on its latitude due to the non-linearity of the equirectangular projection [START_REF] Pearson | Map projections : theory and applications[END_REF]. As a result, the image processing methods traditionally used for perspective images cannot be applied to equirectangular images, especially in the case of optical flow estimation.

The optical flow consists of estimating the displacement of the scene pixels between two frames. Precisely estimated, it gives information about the environment and the sensor's motion. Several methods are used to solve the aperture problem induced by the brightness constancy equation. The oldest Fig. 1. Spherical and equirectangular projection of the globe. In orange (Tissot's indicatrix) is the amplitude of distortions [START_REF] Gaba | Tissot indicatrix equirectangular projection[END_REF].

is the variational method trying to constrain the problem to be as smooth as possible [START_REF] Horn | Determining Optical Flow[END_REF]. Other approaches propose a phase-based model [START_REF] Radgui | Optical flow estimation from multichannel spherical image decomposition[END_REF]. In recent years, significant progress has been made in the accuracy of optical flow estimation with the apparition of convolutional neural networks (CNNs) [START_REF] Dosovitskiy | FlowNet: Learning Optical Flow with Convolutional Networks[END_REF]- [START_REF]A Lightweight Optical Flow CNN -Revisiting Data Fidelity and Regularization[END_REF]. They now dominate optical flow evaluation in the famous perspective datasets [START_REF] Butler | A naturalistic open source movie for optical flow evaluation[END_REF], [START_REF] Menze | Object Scene Flow for Autonomous Vehicles[END_REF]. Still, CNNs critically rely on extensive and representative training data sets.

To deal with equirectangular distortions, networks could be trained on spherical datasets. However, building a complete optical flow set would be very time-consuming and expensive. On the other hand, virtual ones often lack representativeness.

Some studies propose other approaches to deal with the equirectangular distortions. Several methods modify the input images using the Fourier transform [START_REF] Cohen | Spherical CNNs[END_REF] or spherical polyhedron [START_REF] Lee | SpherePHD: Applying CNNs on a Spherical PolyHeDron Representation of 360 degree Images[END_REF]. However, this modification of the input images requires redefining the whole feature map for a good estimation of the optical flow. Other methods directly modify the shape of the kernel used for convolution. In [START_REF] Su | Learning Spherical Convolution for Fast Features from 360 Imagery[END_REF], the kernel sizes are roughly increased near the polar regions. This method gives promising results but neglects the spherical projection and its distortions. Therefore, [START_REF] Tateno | Distortion-Aware Convolutional Filters for Dense Prediction in Panoramic Images[END_REF], [START_REF] Coors | SphereNet: Learning Spherical Representations for Detection and Classification in Omnidirectional Images[END_REF] extend this idea by taking into account the equirectangular transformation. The size of each sample grid is modified using a local perspective projection on the sphere. Consequently, the shape of the kernel is related to the amplitude of the distortion at its latitude.

In this paper, the distortion-aware convolution method is used to implement a solution that is as universal as possible and consistent with the equirectangular projection. The proposed approach preserves the feature map and architecture of high-performance perspective CNNs. The general idea consists in locally projecting the spherical image onto its perspective equivalent. The equirectangular convolution then replaces the usual 2D convolution by modifying the coordinates of the kernel points used for the convolution. The main contribution of this paper is to benefit from years of improvement in perspective optical flow estimation and to transfer it to equirectangular images. Comparison tests are performed on spherical datasets to demonstrate the performance of the new network, OmniFLowNet. These datasets were built with Blender and are the second contribution of this paper. The last contribution of this article is the evaluation of optical flow estimation network on real equirectangular images.

The remainder of the paper is organized as follows. Section II reviews the optical flow estimation process with CNNs, especially FlowNet. The proposed solution, OmniFlowNet, is described in Section III. Section IV compares the performances of the new network and its initial version on generated datasets. Finally, Section V presents the results of OmniFlowNet on real equirectangular images.

II. OPTICAL FLOW ESTIMATION

In the last decades, CNNs have shown their superiority in many image processing tasks. Especially, these networks are on top of the evaluation rankings of famous optical flow perspective datasets [START_REF] Butler | A naturalistic open source movie for optical flow evaluation[END_REF], [START_REF] Menze | Object Scene Flow for Autonomous Vehicles[END_REF]. One of the earliest CNN precursors is FlowNet [START_REF] Dosovitskiy | FlowNet: Learning Optical Flow with Convolutional Networks[END_REF]. It outperforms traditional methods of optical flow estimation and many recent networks are based on it [START_REF] Ilg | FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks[END_REF]- [START_REF]A Lightweight Optical Flow CNN -Revisiting Data Fidelity and Regularization[END_REF].

A. FlowNet

In the first version, two architectures called FlowNet-S and FlowNet-C are proposed and compared. FlowNet-S is the simplest network that directly stacks the two input images and transmits them via a CNN architecture. On the other hand, FlowNet-C first extracts the features of the two input images separately and then combines them in a correlation layer before transmitting to the CNN network. These convolutional neural networks are trained end-to-end using a large data set of image pairs and ground truth optical flow. The input of the network is a pair of successive perspective images, and the output is an estimate of the flow fields (u f , v f ), as shown in Fig. 2. FlowNet architectures contain a contracting and an expanding part, both using back-propagation. In the expanding part, several "up-convolutional" layers increase the resolution of the feature maps. The contracting and expanding parts are connected to preserve both high-level information of the coarser feature maps and low-level information of the contracting part.

B. FlowNet2

In FlowNet2, several networks (FlowNet-S/FlowNet-C) are stacked to form a deeper network. At the end of each network, the second image I 2 is warped to the first image I 1 using the optical flow estimated by the previous network. This way, the next network can only focus on the remaining optical flow between I 1 and I 2 after warping. Each network in the stack has a particular architecture to perform a specialized task: some focus more on large displacements, others on small ones. Each network is trained solo while keeping the other CNNs weights fixed during the process to ensure better performance.

Due to its great performances in 2017, FlowNet2 is one of the most cited networks and has given birth to an important family. Thus, LiteFlowNet2, the most recent network of the FlowNet family available on GitHub2 , is selected as baseline in this paper to obtain the best performance for optical flow estimation. Our solution is developed to be as universal as possible, easily implemented on every network. Based on FlowNet networks, this paper proposes to build OmniFlowNet: a CNN trained on perspective images and used on spherical images for optical flow estimation. This strategy avoids generating large and complex spherical datasets for training and is robust against equirectangular distortions. The overall architecture and weights of OmniFlownet come from FlowNet models trained on perspective images and groundtruth optical flow, as shown in the first row of Fig. 3. The standard convolution is then replaced by the new distortionaware convolution to deal with equirectangular inputs. Our solution is compatible with every size of kernel, stride, or padding. Once these code modifications realized, pairs of equirectangular images can be used in OmniFlowNet as input. The second row of Fig. 3 presents the testing process of the network.

III. PROPOSED SOLUTION

A. Equirectangular convolution

Inspired by [START_REF] Fernandez | Corners for Layout: End-to-End Layout Recovery from 360 Images[END_REF], this section presents the convolution adaptation. The usual perspective kernel is modified to fit the equirectangular distortions. To build a kernel of resolution r and angular size α centered in a location p 00 = (u 00 , v 00 ) in the equirectangular image, the center coordinates are first transformed to spherical system p s,00 = (φ 00 , θ 00 ) using

φ 00 = u 00 - W 2 2π W ; θ 00 = -v 00 - H 2 π H , (1)
where W and H are respectively the width and the height of the equirectangular image in pixels. Each point of the kernel is defined by

pspher,ij =   xij ŷij ẑij   =   i j d   , (2)
where i and j are integers in the range -r-1 2 , r-1

2
and d is the distance from the center of the sphere to the kernel grid. In order to cover the field of view α, the distance is set to

d = r 2 tan( α 2 )
. The coordinates of these points are computed by normalizing and rotating them to align the kernel center on the sphere. Therefore:

p spher,ij =   x ij y ij z ij   = R y (φ 00 ) R x (θ 00 ) pspher,ij |p spher,ij | , (3)
where R a (β) stands for the rotation matrix of an angle β around the a axis. These coordinates are transformed to latitude and longitude in the spherical domain using

φ ij = arctan x ij z ij ; θ ij = arcsin (y ij ) ; (4)
and finally back-projected to the original 2D equirectangular image

u ij = φ ij 2π + 1 2 W ; v ij = - θ ij π + 1 2 H. (5)
In Fig. 4, some example of kernels at different latitude and longitude are presented. The blue point defines the center of the kernel p 00 = (u 00 , v 00 ). The red points are the positions of the elements p ij = (u ij , v ij ) in the adapted kernel, defined as previously. The green points are the positions of elements in a standard perspective kernel given by:

u persp,ij = u 00 + ir; v persp,ij = v 00 + jr. (6)

B. OmniFlowNet convolution

The OmniFlowNet architecture is based on the implementation of open-source software shared by the LiteFlowNet2 team [START_REF]A Lightweight Optical Flow CNN -Revisiting Data Fidelity and Regularization[END_REF]. It is based on the Caffe framework [START_REF] Jia | Caffe: Convolutional Architecture for Fast Feature Embedding[END_REF] and allows GPU acceleration for training and testing. In the Caffe engine, the function im2col contains the convolution operation. Updating this function with the previously proposed models and building the distribution allows OmniFlownet to compute the spherical convolution.

In im2col function, the coordinates of elements on each spatial patch are modified according to patch latitude. Therefore, the distortion of the output matrix is the same as the equirectangular image. In OmniFlowNet, the new distortionaware convolution replaces all standard convolutions in the contracting or expanding part of the array.

IV. RESULTS ON VIRTUAL DATASETS

First, the performance of OmniFlowNet and its initial versions are tested on equirectangular data sets with a ground truth optical flow, specially built for this study. To the best of our knowledge, all the famous datasets available with optical field truth flow, such as MPI Sintel [START_REF] Butler | A naturalistic open source movie for optical flow evaluation[END_REF], KITTI [START_REF] Menze | Object Scene Flow for Autonomous Vehicles[END_REF] or FlyingThings3D [START_REF] Mayer | A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation[END_REF] contains only perspective images. Therefore, omnidirectional datasets are constructed to compare networks.

A. Equirectangular datasets 1) Scene creation: Inspired by the realization of the MPI Sintel dataset, Blender is used. This tool is a free and opensource 3D computer graphics software for the creation of animated films, visual effects, 3D printed models, interactive 3D applications, and video games. Three different scenes called Cartoon Tree, Forest and LoyPolyModel are generated with free 3D models available online (Fig. 5). An equirectangular camera, simulated by Blender, moves in these fixed scenes with different orientations given by Euler angles (roll φ, pitch θ, yaw ψ). The sets are shown in 2) Ground truth extraction: The Vector Pass given by the Blender Cycles Renderer is used to extract the ground truth optical flow, as presented in [START_REF] Ranjan | Learning Multi-Human Optical Flow[END_REF]. This render pass is usually helpful to produce motion blur by giving the motion of every pixel in the image. Here, the Vector Pass returns the pixel displacement in the horizontal and vertical directions perpendicular to the camera axis, the ground truth optical flow. Fig. 6 presents an example of images generated by the scene Forest and their ground truth optical flow.

B. Evaluation 1) Metrics:

To evaluate OmniFlowNet, two classical metrics are used [START_REF] Baker | A Database and Evaluation Methodology for Optical Flow[END_REF]: the average Angle Error (AE) and the average Endpoint Error (EE). The AE between a flow vector (u f , v f ) and the ground-truth flow (u f gt , v f gt ) is the angle in 3D space between (u f , v f , 1.0) and (u f gt , v f gt , 1.0). This error is computed by taking the dot product of the vectors, dividing by the product of their lengths, and then taking the inverse cosine:

AE = cos -1   1 + u f • u f gt + v f • v f gt 1 + u 2 f + v 2 f 1 + u f gt + v f gt   (7)
The goal of the AE is to provide a relative measure of performance that avoids the "divide by zero" problem for zero flows. Errors in large flows are penalized less in AE than errors in small flows. We also compute an absolute error, the EE:

EE = 1 N N (u f gt -u f ) 2 + (v f gt -v f ) 2 (8)
Although the use of AE is common, the EE measure is also appropriate to understand the magnitude of flow errors. For ease of reading and comparison, the AE difference is set by comparing the AE values between FlowNet and OmniFlowNet. The result is an angle, positive if OmniFlowNet performs better than FlowNet, negative if not.

AE dif f = AE F lowN et -AE OmniF lownet . (9)
The EE difference is also defined as the difference between the EE of FlowNet and OmniFlowNet. If OmniFlowNet performs better than FlowNet, the gain is a positive number of pixels, otherwise negative.

EE dif f = EE F lowN et -EE OmniF lownet . (10)
2) Results and Analysis: Network weights come directly from LiteFlowNet2 [START_REF]A Lightweight Optical Flow CNN -Revisiting Data Fidelity and Regularization[END_REF]. The SINTEL set [START_REF] Butler | A naturalistic open source movie for optical flow evaluation[END_REF] is chosen to fit the synthetic images coming from the Blender rendering. From this trained model, the OmniFlowNet network is created by adapting the equirectangular convolution, as described in Fig. 3.

Both networks are tested on previously constructed equirectangular datasets. The angular error and the final error are compared for all results. Table I presents all AE (Eq.7, degrees) and EE (Eq.8, pixels) results and their average on orientation cases for OmniFlowNet and its initial LiteFlowNet2 network on all scenes. Second part of Table I directly shows the differences in AE (Eq.9) and EE (Eq.9) between the networks.

In all averaged cases, OmniFlowNet outperforms Lite-FlowNet2 on the average angular error, which means that it measures better the direction of the motion vectors. As for the average endpoint error, OmniFlowNet has better results in almost all of them (small under-performance in 2 configurations in Forest). The overall gain expected, on average, is around 4.7 degrees in AE and 0.8 pixels in EE.

V. RESULTS ON REAL DATASETS

To the best of our knowledge, there is no omnidirectional dataset with optical flow ground truth available. So we compared the performances of LiteFlowNet2 and OmniFlowNet directly on real equirectangular images. These were taken with a RICOH THETA Z1. Two situations were mainly studied: one with a fixed camera in a moving scene and the other with a moving camera in an essentially rigid scene.

A. Field protocol

Different real indoor and outdoor scenes were filmed with the omnidirectional camera to observe motion in the polar regions. In the cases of Apple 1 and Ball 3, a sphere is moving next to a fixed camera in an indoor scene. In Ball 1 and Ball 2, a ball is also observed but in an outdoor scene. In Car 1 and Car 2, the camera is held outside a moving car while passing under an archway. Once acquired, the equirectangular videos are reconstructed from the fisheye inputs using the RICOH THETA Movie Converter application 3 . Then, the videos are cut into multiple frames using

B. Optical flow results

The same networks as in the virtual dataset evaluation were used: LiteFlowNet2 with SINTEL weights and its Om-niFlowNet version. A video compiles5 all the experimental results and shows the comparison between the two networks.

In all cases, the optical flow estimated in the equatorial region by LiteFlowNet2 and OmniFlowNet are similar as shown in Fig. 7. Our network behaves as its perspective version in low distorted areas. In polar regions, the optical flow estimated by OmniFlowNet is smoother and, all the time, more coherent than the network in perspective, as shown in Fig. 8. Whereas the ball motion estimated by LiteFlowNet2 is a shredded mark, OmniFlowNet predicts a complete ball with coherent motion. In Fig. 9, the arm moving above the north pole of the camera has a smoother predicted optical flow by the spherical network than the perspective one. The equirectangular convolution helps the network to better understand and calculate motion in highly distorted areas. The south polar region often presents optical flow noise in the OmniFlowNet estimation. This is related to the artifacts created by the tripod holding the camera. When reconstructing the equirectangular image from the two fish-eye lenses, this highly distorted region certainly brings some noise to the final image. While LiteFlowNet interprets it as white noise, OmniFlowNet with the equirectangular convolution probably detects coherent motion. The Fig. 10 shows that by blackening the entire tripod region, the induced noise is reduced.

Overall, on several diverse real data sets, OmniFlowNet outperforms its perspective version in estimating better and smoother optical flow in polar regions and shows the same performance in the equatorial region. Due to space limitation, only brief but most representative parts of total results are presented in the video 6 .

VI. CONCLUSION

In this paper, we present OmniFlowNet, a CNN for optical flow estimation on omnidirectional images. Built on a proven reference CNN, our network benefits from many previous studies on optical flow estimation. It adapts to equirectangular distortions using distortion-aware convolution. The high-end methods for optical flow estimation on perspective images are applied locally on the sphere. Thus it only uses pretrained weights on perspective images and does not require extra training on equirectangular datasets. OmniFlowNet outperforms its original network on omnidirectional sets built with Blender and on real equirectangular sets with various scenes and motion. Our code implementation and omnidirectional dataset is available on GitHub 7 .

Fig. 2 .

 2 Fig.2. Two main parts of FlowNet models: the contracting part and the expanding part.[START_REF] Dosovitskiy | FlowNet: Learning Optical Flow with Convolutional Networks[END_REF]

Fig. 3 .

 3 Fig. 3. Training: FlowNet is trained with perspective images and ground truth. Testing: OmniFlowNet is built with the architecture and weights of the previously trained FlowNet network and adapted with the equirectangular convolution.

Fig. 4 .

 4 Fig. 4. Example of kernels with different latitude and longitude. In blue is the center of the kernel, in green the perspective kernel and in red the equirectangular one. Wide distortions are visible near the poles.

Fig. 5 .

 5 Fig. 5. 3D models used to generated Forest datasets.

Fig. 6 .

 6 Fig. 6. Example of two image pairs and their optical flow ground truth generated by the scene Forest with different camera orientations.

Fig. 7 .

 7 Fig. 7. Car 1 case: RGB input image (top) and optical flow computed by LiteFlowNet2 (bottom-left) and OmniFlowNet (bottom-right). The optical flow estimated in the equatorial region is similar. The two networks behaves in the same way in low distortion areas.

  FFmpeg

4

 4

  .

  3 https://support.theta360.com/en/download/ 4 https://ffmpeg.org/

Fig. 8 .

 8 Fig. 8. Ball 1 case: RGB input image (top) and optical flow computed by LiteFlowNet2 (bottom-left) and OmniFlowNet (bottom-right). OmniFlowNet better estimates the optical flow in the top polar region is better. The ball is clearly more visible and smoother in that case.

Fig. 9 .

 9 Fig. 9. Apple 1 case: RGB input image (top) and optical flow computed by LiteFlowNet2 (bottom-left) and OmniFlowNet (bottom-right). OmniFlowNet better estimates the optical flow in the top polar region is better. The arm is smoother.

Fig. 10 .

 10 Fig. 10. Ball 1 case with blackened south pole: original RGB input image (top-left), new RGB input image with blackened south pole (top-right) and respective optical flow computed by OmniFlowNet (bottom-left) and Omni-FlowNet (bottom-right). The optical flow computed with the blackened images shows less noise in the south pole. Thus camera tripod and equirectangular reconstruction add noise to the optical flow estimation.

TABLE II .

 II The image resolution is 768 × 384 to maintain a width to height ratio of 2.

TABLE I AE

 I (DEFINED IN EQ.7, IN DEGREES) AND EE (DEFINED IN EQ.8 COMPUTED FOR BOTH LITEFLOWNET2 AND OMNIFLOWNET NETWORKS ON THE THREE SCENES. THE RESULTS ARE AVERAGED ON CASES.

		Cartoon Tree			Forest		Low Poly Model
		LiteFlowNet2 OmniFlowNet	LiteFlowNet2	OmniFlowNet	LiteFlowNet2 OmniFlowNet
		AE	EE	AE	EE	AE	EE	AE	EE	AE	EE	AE	EE
	Case 1	61.67 3.76 53.87	3.05	58.81 16.56 54.47 15.65	58.22 7.29 54.62	6.87
	Case 2	62.65 4.22 56.07	3.16	57.91	8.23	53.99	7.37	57.66 6.89 53.54	6.41
	Case 3	64.16 7.41 55.89	6.03	57.73	8.47	56.32	7.64	57.55 7.51 55.42	6.99
	Case 4	63.78 7.01 53.14	5.7	57.99	9.18	55.42	8.23	58.68 8.96 57.25	8.65
	Average on Cases	63.07 5.60 54.74	4.49	58.11 10.61 55.05	9.72	58.03 7.66 55.21	7.23
		AE diff	EE diff	AE diff	EE diff	AE diff	EE diff
	Average on Cases	8.32		1.11		3.06		0.89		2.82		0.43

https://www.blender.org

https://github.com/twhui/LiteFlowNet2

http://www.i3s.unice.fr/ ∼ allibert/Videos/icpr20 video.mp4

Case


• Hand movements and speech can be coordinated [2].

• Body movements and speech each require a specific adaptation of breathing [3,4,5,6].

• SALAMMBO project : interactions speech, breathing, movement 4

• Our aim is to investigate speech-breathing and limb interactions : how do the three systems behave in spontaneous speech tasks ? • First acoustic results [7] : less pauses and longer speech intervals when moving.

Question

• How does breathing interact with legs vs. arms movements when it is shared with speech ? • Is the breathing-arms coupling greater while speaking than the breathing-legs one [2] ?

References Movements shorten breathing cycle in case of quiet breathing, this effect is not present when speech occurs.

è When limbs movements are effortless, breathing seems to mainly adapt to speech. è What about effortful movements ?

We have unexpected results related to theories of hand-mouth coupling [2]: arms and legs cycles have the same behavior. Is there a Specific coupling?

Correlation between duration of arms and legs cycles

Analysis

Conclusion

Results

Duration of breathing cycles according to the condition

Pearson coefficient:0.78

Breathing cycle duration / movement cycle duration (ratio)

Breathing and movement coordination

  the French National Research Agency (ANR-17-FRAL-0005-01) and the German Research Foundation (DFG) as part of the SALAMMBO project (Spoken Language in Motions: Learning and Adaptation of Speech Communication in the context of Body Motions) Speaking during rhythmic motions of feet vs hands:how do speech and limb movements affect each other?

  [1] Oppezzo, M., & Schwartz, D. L. (2014). Give your ideas some legs: The positive effect of walking on creative thinking. Journal of experimental psychology: learning, memory, and cognition, 40(4), 1142. [2] Gentilucci, M., Benuzzi, F., Gangitano, M., & Grimaldi, S. (2001). Grasp with hand and mouth: A kinematic study on healthy subjects. Journal of Neurophysiology, 86, 1685-1699. Gentilucci, M., Benuzzi, F., Gangitano, M., & Grimaldi, S. (2001). Grasp with hand and mouth: A kinematic study on healthy subjects. Journal of Neurophysiology, 86, 1685-1699.[3] Pouw, W., Harrison, S. J., & Dixon, J. A. (2018). Gesture-Speech Physics: The Biomechanical Basis of Gesture-Speech Synchrony. Preprint PsyArxiv doi, 10. [4] Bardy, B. G., Hoffmann, C. P., Moens, B., Leman, M., & Dalla Bella, S. (2015). Sound-induced stabilization of breathing and moving. Annals of the New York Academy of Sciences, 1337(1), 94-100. [5] Fuchs, S., Reichel, U. D., & Rochet-Capellan, A. (2015). Changes in speech and breathing rate while speaking and biking.[6] Fuchs, S., Petrone, C., Rochet-Capellan, A., Reichel, U. D., & Koenig, L. L. (2015). Assessing respiratory contributions to f0 declination in German across varying speech tasks and respiratory demands. Journal of Phonetics, 52, 35-45. [7] Heather Weston & Susanne Fuchs (2019), Rhythmic limb movements affect speech output differently in individuals.

  Onset of movement cycles relative to the breathing cycles

			1.	Univ. Grenoble Alpes, CNRS, Grenoble INP, GIPSA-lab, 38000 Grenoble, France
			2.	Leibniz-Centre General Linguistics (ZAS)
			3.	Humboldt-Universität zu Berlin, Berlin, Germany
	Quiet/Speech	Quiet/Speech	Quiet/Speech
	only	+ leg cycling	+ arm cycling
	(Q/S)	(QL/SL)	(QA/SA)

Notes 4. Spoken lAnguage in Motions:Learning and Adaptation of speech coMMunication in the context of BOdy motions Imagine you have to go to a lonely island and you could bring only five of these ten items. Which ones would you choose? Bring them in order, and motivate your choice. Why wouldn't you take the other five items?

Acknowledgments This research is jointly supported by