Mathieu Danoy

Yannick Tauran

Stéphane Poulain

Rachid Jellali

Johanna Bruce

Marjorie Leduc

Morgane Le Gall

Francoise Gilard

Taketomo Kido

Hiroshi Arakawa

Karin Araya

Daiki Mori

Yukio Kato

Hiroyuki Kusuhara

Charles Plessy

Atsushi Miyajima

Yasuyuki Sakai

Eric Leclerc

Multi-omics analysis of hiPSCs-derived HLCs matured onchip revealed patterns typical of liver regeneration

Keywords: iPSCs, hepatocytes, liver, organ-on-chip, differentiation, nanoCAGE, transcriptomics, metabolomics, proteomics

Maturation of human-induced pluripotent stem cells (hiPSCs)-derived hepatocytes-like cells (HLCs) toward a complete hepatocyte phenotype remains a challenge as primitiveness patterns are still commonly observed. In this work, we propose a modified differentiation protocol for those cells which includes a prematuration in Petri dishes and a maturation in microfluidic biochip. For the first time, a large range of biomolecular families has been extracted from the same sample to combine transcriptomic, proteomic, and metabolomic analysis. After integration, these datasets revealed specific molecular patterns and highlighted the hepatic regeneration profile in biochips. Overall, biochips exhibited processes of cell proliferation and inflammation (via TGFB1) coupled with anti-fibrotic signaling (via angiotensin 1-7, ATR-2, and MASR). Moreover, cultures in this condition displayed physiological lipidcarbohydrate homeostasis (notably via PPAR, cholesterol metabolism, and bile synthesis) coupled with cell respiration through advanced oxidative phosphorylation (through the over-expression of proteins from the third and fourth complex). The results presented provide an original overview of the complex mechanisms involved in liver regeneration using an advanced in-vitro organ-on-chip technology.

Introduction

Despite significant efforts toward an enhanced maturation hiPSCs-derived (HLCs), primitiveness patterns are still largely observed [START_REF] Hay | Efficient differentiation of hepatocytes from human embryonic stem cells exhibiting markers recapitulating liver development in vivo[END_REF][START_REF] Touboul | Generation of functional hepatocytes from human embryonic stem cells under chemically defined conditions that recapitulate liver development[END_REF][START_REF] Si-Tayeb | Highly efficient generation of human hepatocyte-like cells from induced pluripotent stem cells[END_REF][START_REF] Kido | CPM is a useful cell surface marker to isolate expandable bi-potential liver progenitor cells derived from human iPS cells. Stem cell reportsliu[END_REF], Lereau Bernier et al., 2019).

Consequently, hiPSCs-derived HLCs are still far from being in a state which would allow them to be used for in-vitro studies in the industry. As a complement to conventional culture models, strategies based on tissue engineering using 3D reconstruction and organoids [START_REF] Takebe | Massive and reproducible production of liver buds entirely from human pluripotent stem cells[END_REF] or microscale bioreactors [START_REF] Giobbe | Functional differentiation of human pluripotent stem cells on a chip[END_REF] have been identified as a potential source of improvement for the differentiation of hiPSCs into hepatic cells.

Among those, organ-on-chips, aimed at reproducing the complex human physiology using micro and nanotechnologies (Sosa Hernandez et al., 2018), have allowed proposing a new generation of microfluidic liver models [START_REF] Kang | Liver sinusoid on a chip: Long-term layered co-culture of primary rat hepatocytes and endothelial cells in microfluidic platforms[END_REF][START_REF] Rennert | A microfluidically perfused three dimensional human liver model[END_REF][START_REF] Jellali | Long-term human primary hepatocyte cultures in a microfluidic liver biochip show maintenance of mRNA levels and higher drug metabolism compared with petri cultures[END_REF]. In that regard, the enhancement of the maturation of hiPSCs-derived hepatic cells in a microfluidic biochip has been demonstrated [START_REF] Danoy | Optimized protocol for the hepatic differentiation of induced pluripotent stem cells in a fluidic microenvironment[END_REF]. Previous datasets have shown that 7 days of maturation in the biochips were insufficient to obtain a quantifiable and inducible CYP450 activity [START_REF] Danoy | Optimized protocol for the hepatic differentiation of induced pluripotent stem cells in a fluidic microenvironment[END_REF]. Nevertheless, the culture in microfluidic biochips exhibited, when compared to Petri dishes, a higher activity of typical liver transcription factors such as ESRRA, HNF1A, HNF4A, TCF4, and CEBPA, as well as a different modulation of pathways related to the extracellular matrix, tissue reorganization, HIF and glycolysis which shows promise for the technology.

In this work, we present an extended maturation of hiPSCs-derived HLCs in microfluidic biochip for 14 days under perfusion and stimulation with OSM. Additionally, as we observed that the 5cm-long microfluidic biochip that was used in our previous experiment would lead to a largely hypoxic culture condition (Danoy et al., Submitted), we decided to shorten the microfluidic biochip to 1cm. This design has shown its suitability for primary liver human hepatic cultures [START_REF] Jellali | Long-term human primary hepatocyte cultures in a microfluidic liver biochip show maintenance of mRNA levels and higher drug metabolism compared with petri cultures[END_REF] and allows the reduction of the required number of inoculated cells, which make it more suitable for larger-scale experiments. However, despite a limited amount of biological material in these culture conditions (Around 1.01e5 cells per biochip), a large range of different biomolecular family (transcriptome, metabolome, and proteome) have been successfully extracted in a single sample.

The multi-omics approach is essential to understand the key molecular events involved in the stem cell plasticity [START_REF] Abazova | Advances in stem cell proteomics[END_REF], but until now, the few studies of hepatic differentiation from hiPSCs have mainly focused either on metabolome/transcriptome [START_REF] Tauran | Analysis of the transcription factors and their regulatory roles during a step-by-step differentiation of induced pluripotent stem cells into hepatocyte-like cells[END_REF] or more recently on proteomics alone (Hurrel et al., 2019). Here, the multiple datasets have been integrated to identify more accurately the main molecular changes that led eventually to a more mature hepatocyte profile. Finally, a scheme that revealed the biomolecular patterns (promotors, mRNA, proteins, and metabolites) has been proposed. These act as a trigger to specific biological pathways which were found either activated (glycolysis, cell proliferation, inflammation, SMAD, etc.) or inhibited (fibrosis, OXPHO respiration, etc.) under the unique 3D micro-environment offered by the microfluidic biochip culture as compared to the classical 2D Petri dishes.

Results

Cell morphology and tissue characterization

The cultures performed in both Petri dishes (Fig. 1A) and biochips (Fig. 1B) led to the formation of a tissue comprising cells exhibiting a typical hepatocyte morphology with a large nucleus. In biochips, binucleated cells were notably spotted in the middle of the chambers while cells with elongated morphologies were observed close to the walls. The albumin production, both biochips and Petri dishes reached a similar level around 3500 ng/mL/10 6 cells after 14 days of maturation (Fig. 1C). mRNA levels in Petri dishes and biochips revealed lower levels of ALB in biochips but similar levels of AFP, CYP3A4, and HNF4A in both conditions. When compared to samples issued from Liver Total RNA, both Petri dishes and biochips still exhibited lower levels of maturation notably in terms of CYP3A4 (Fig. 1D).

Immunostainings displayed a higher expression of AFP in Petri dishes when compared to biochips. Most of the cells were found to be positive for CK19 in Petri dishes while only cells on the top of the microstructure were positive in biochips (Fig. 1E). More intense staining for Albumin was observed in biochips when compared to Petri dishes. Expression of BSEP (bile salt export pump, involved in the bile transport and normally located at the canaliculi membranes) and of MRP3, located on the membrane of hepatocytes was also confirmed in the biochip cultures (Fig. 1F).

Functional assays

General capabilities of the tissue in terms of bile acid metabolism were found to be largely enhanced by the culture in microfluidic biochips or by an extended perfusion duration of 14 days (Fig. 2A). Consumption of Cholic Acid (CA) was detected in both Petri dishes and biochips but was found to be significantly more important after 14 days of perfusion in the microfluidic biochips. Resulting productions of Glycocholic Acid (GCA) and Taurocholic Acid (TCA) were only significantly detected in biochips.

Interestingly, consumption of Chenodeoxycholic Acid (CDCA) was detected in Petri dishes while a decreasing production from 7 days of perfusion to 14 days was detected in biochips. A production of Glycochenodeoxycholic Acid (GCDCA) was detected on day 14 in both culture modes but was found to be much higher in biochips. Finally, significant consumption of Taurochenodeoxycholic Acid (TCDCA) was observed in biochips after 7 and 14 days of perfusion.

CYP3A4 assay has shown higher activity in biochips when compared to Petri dishes but inducibility with rifampicin was found to be only effective in Petri dishes (Fig. 2B). Metabolisms of phenacetin via CYP1A2, of diclofenac via CYP2C9 and of coumarin via CYP2A6 was confirmed in both Petri dishes and biochips (Fig. 2C).

However, the activities of CYP1A2 and CYP2C9 were higher in biochips than in Petri dishes. The extraction of the urea levels from the metabolomic analysis indicated a similar production in both Petri dishes and biochips (Fig. 2D). PAS staining has confirmed the glycogen storage capabilities of the tissue in both Petri dishes and biochips (Fig. 2E).

The profile of cytokines and pro-angiogenic factors produced was evaluated by performing the measurement in biochips, in Petri dishes, and the culture medium (Fig. 2F, detailed results in Supp. Fig. 1). Those results seemed to indicate the consumption of proteins related to the formation of blood vessels and to angiogenesis (ANG, ANGPT2, PECAM-1, and TIE-2) as lower concentrations were found in biochips when compared to what was observed in the culture medium. In parallel, several proteins and cytokines related to angiogenesis (EGF, bFGF, MMP-9, endostatin, and UPAR) and inflammation (ENA-78, GRO, GM-CSF, MCP-3 IL-8, IL-1 beta, and IL-4) were found in higher concentrations in Petri dishes.

Transcriptome analysis

NanoCAGE [START_REF] Plessy | Linking promoters to functional transcripts in small samples with nanoCAGE and CAGEscan[END_REF][START_REF] Poulain | NanoCAGE: a method for the analysis of coding and noncoding 50-capped transcriptomes[END_REF]) RNA-Seq profiles of cell samples differentiated either in Petri dishes (n=3) or biochips experiments (n=4) have been compared by iDEP8.1 processing [START_REF] Ge | iDEP: an integrated web application for differential expression and pathway analysis of RNA-Seq data[END_REF]. Differential Gene Expression (DGE) analysis revealed 169 and 236 genes upregulated in Petri dishes and biochips respectively (False Discovery Rate cutoff of 0,1; minimum Fold Change of 2, Supp.

File 1). Notably, Transcription Factors (TFs) genes related to CYP1A1, to various transporters such as SLC12A7, SLC4A7, SLC 12A8, SLC 19A2, SLC 25A37 or SLC 2A3, to JUN, MYC, BMP6, ATF3, NFATC1, NFATC2, HCFC1, FOXO3, CTNNB1 (βcatenin) or NR4A1 were found to be upregulated in biochips. Several transcripts coding for TFs related to inflammation such as NFKB1A, NFKB1, NFKB2, NFKBIZ, CCL2, CXCL2, TNFRSF10D, TNFAIP3, TGFBI, TGFB2, MAP3K8 were upregulated in biochips. Meanwhile, mRNA levels of TFs such as LYVE1, PECAM1, or ANGPLT3 (related to the endothelium), LAMB1, COL16A1 (related to the extracellular matrix and the GP6 signaling) were upregulated in Petri dishes.

By raising the False Discovery Rate cutoff to 0,2 in the DGE analysis, a more restricted list of differentially expressed genes has been obtained for which the heatmap is represented in Supp. Fig. 2A. Pathway Gene Set Enrichment Analysis on this new list using the GO_biological_process geneset allowed to highlight pathways related to liver development, cellular response to oxidative stress, response to lipid, response to chronic inflammation and the regulation of the cell migration involved in angiogenesis sprouting (Supp. Fig. 2B). Also, the highlighted GO_KEGG pathways were NFKB, TNF, TGF-beta as well as the p53 signaling (Supp. Fig. 2C). Interestingly, HIF and Hepatitis B signaling were also highlighted by the analysis. Finally, the processes highlighted with the GO_molecular function category were mainly the upregulation of the transcription factor activity as well as processes related to DNA and RNA (Supp. Fig. 2D).

Motif Activity Response Analysis

NanoCAGE transcriptomic data were processed for Motif Activity Response Analysis (MARA) [START_REF] Balwierz | ISMARA: automated modeling of genomic signals as a democracy of regulatory motifs[END_REF]. The top 10 TFs with the highest motif activity extracted from the analysis are listed in Table 1, and the corresponding target genes and signaling pathways are detailed in Supp-file 1. Among those, most were found to have higher activity in biochips (MECP2, TAF1, RELA, SIN3A, MYCN, NFKB1, ZFX, FOSL1, and ZNF711) and only ZEB1 was found to have higher activity in Petri dishes.

Based on these TFs, a regulatory network, describing the direct relationship between TFs and their target pathways and genes, could be established (Fig. 3). Notably, TFs related to inflammation were highlighted (MECP2, RELA, MYCN, NFKB1, ZFX, and FOSL1 via the TNFA, TGFB, and NFKB pathways). Besides, TAF1 and ZNF711 were also featured in the analysis and were linked to the HIF signaling and mitochondrial processing. While they could not be found in the list of the top 10 TFs with the highest activity, PML and PATZ1 were found to be underlying nodes which contributed to bridging the network to important liver functions such as HNF1A and CTNNB1 (Beta Catenin) while associations with the Wnt, HNF3B and NFAT signaling could also be established. Finally, the analysis also revealed a link between MECP2 and several endothelial markers such as ANGPLT4 and VEZF1.

Proteomic analysis

The analysis allowed to extract 380 proteins that were differentially expressed in Petri dishes and biochip with p_value below 0,05 among the total of 3186 identified (Fig. 4A, Supp. File 2). Among those, AFP, several collagen proteins, and fibrinogens were found to be over-expressed in Petri dishes while surprisingly, no significant difference could be detected in the expression of CYP450. Amid the top 50 most differentially expressed proteins (Heatmap in Fig. 4B), TIMP1, COL3A1, COL14A1, COL15A1, LG, EMILIN-1 (related to the extracellular remolding), APOA2 and PLTP (linked to lipid metabolism) were found to be over-expressed in Petri dishes. On the other hand, MAP3K11, IKBIP, ASCC3 (related to NFKB activation), ABLIM3, FSCN1, (actin-related), AKR1D1 (bile metabolism), GBA (glycolipid metabolism), PLIN2 (perilipin 2, lipid droplet) were found to be over-expressed in biochips. Interestingly, proteins related to the Krebs cycle (ACO1, IDH2, SDHA, SDHB, PCK2) were also found to be under-expressed in biochips.

By inputting the list of 380 proteins in the Ingenuity Pathway Analysis (Supp.

File 3), the modulation of pathways related to the activation of LXR/RXR and FXR/RXR, to the acute phase response signaling, to the coagulation system and the inhibition of the matrix metalloproteases could be extracted. The top upstream regulators revealed were TGFβ1, iL-6, and molecules acting on the androgen receptors (dexamethasone, dihydrotestosterone, metribolone). Using the KEGG pathway analysis (Supp. File 2), pathways related to cancer, cell cycle (CDKN1C, MCM2 up-regulated in Petri dishes and CHEK2, ORC3, SKP2 up-regulated in biochips), lipid metabolism (PPAR signaling, cholesterol metabolism, and bile synthesis), shear stress, cellular remolding and reorganization (focal adhesion, ECM-receptor interactions, and tight junctions,) oxidative phosphorylation, iron metabolism (up-regulation of FTL and FTH1 in biochips).

Metabolomics analysis

Based on the OPLS-DA S-plots, 47 metabolites were found to be differentially expressed between Petri dishes and biochips among the 87 identified ones (Fig. 4D-F, with P < 0.05 and VIP > 1, Supp. File 4). Notably, higher levels of glycerol, cholesterol, malic acid, fumaric acid, succinic acid, aspartic acid (Krebs cycle intermediates), lactic acid and putrescine were found in Petri dishes while higher levels of lipids (lauric acid, oleic acid, caprylic acid, and palmitoleic acid), glucose and amino acids (serine, threonine, lysine, ornithine, phenylalanine, leucine and isoleucine) were observed in biochips.

Those metabolites were mapped with MetaboAnalyst to identify the related metabolic pathways. The analysis revealed that pathways related to the Aminoacyl-tRNA biosynthesis, the alanine aspartate and glutamate metabolism, the fatty acid biosynthesis, the phenylalanine metabolism, and the arginine proline metabolism were modulated (Supp. File 4). The enrichment analysis highlighted the beta-oxidation, the urea cycle, the spermidine and spermine metabolism, the fatty acid biosynthesis, and the glutathione Metabolism as the top 5 regulated pathways (Supp. File 4).

Discussion

In the present study, the maturation of hiPSCs-derived HLCs was led in a microfluidic environment for 14 days. The protocol has allowed detecting a CYP3A4 activity as well as an improved activity of several cytochromes over Petri dishes and over previously published protocols using biochips [START_REF] Danoy | Optimized protocol for the hepatic differentiation of induced pluripotent stem cells in a fluidic microenvironment[END_REF]Danoy et al., 2020). The multi-omics analysis performed in the biochips revealed a signature, typical of a liver regenerative process, illustrated in Fig. 5.

Analysis of the metabolome in Petri dishes revealed an aerobic respiration profile with important glycolysis as shown by the glucose/lactate ratios and by the overexpression of metabolites and proteins related to the cycle of Krebs in Petri dishes.

Notably, proteins from the first and the second protein complex of the respiratory chain were found to be over-expressed in Petri dishes while proteins from the third and the fourth (COX15, ATP6A1) were over-expressed in biochips. The over-expression of proteins from the first complex leads to a high production of ROS [START_REF] Murphy | How mitochondria produce reactive oxygen species[END_REF] which is also consistent with the high levels of pyroglutamate found in Petri dishes and can be linked to the glutathione metabolism [START_REF] Liu | Emerging regulatory paradigms in glutathione metabolism[END_REF]. Also, the over-expression of proteins from the second complex is consistent with the activation of the cycle of Krebs in Petri dishes (Martinez-Reyes et al., 2020) and with a potential response to hypoxia (Pflegger et al., 2015). Finally, the over-expression of proteins from the third and the fourth complex has been related to a higher OXPHOS capability [START_REF] Machida | Pluripotency transcription factors and metabolic reprogramming of mitochondria in tumor-initiating stem-like cells[END_REF] which might be linked to a profile of higher maturation in biochips.

In both culture modes, the tissue was exposed to high concentrations of insulin and glucose in what is considered a lipogenic stimulation [START_REF] Schwarz | Hepatic de novo lipogenesis in normoinsulinemic and hyperinsulinemic subjects consuming high-fat, low-carbohydrate and low-fat, high-carbohydrate isoenergetic diets[END_REF]. In biochips, a production of glycerolipids and triglyceric acids, an intense metabolism of steroids, and a synthesis of bile acids from cholesterol were observed. It has often been reported that the metabolism of lipids and the increase of bile acids are strongly related to a process of liver regeneration [START_REF] Rudnick | Functional Relationships between Lipid Metabolism and Liver Regeneration[END_REF][START_REF] Manco | Liver Regeneration: Different Sub-Populations of Parenchymal Cells at Play Choreographed by an Injury-Specific Microenvironment[END_REF][START_REF] Keitel | Endocrine and paracrine role of bile acids[END_REF]. After hepatectomy, the liver regeneration process is put in correlation with a transient apparition of a signature attributed to a fatty liver [START_REF] Schofield | Altered interactions between lipogenesis and fatty acid oxidation in regenerating rat liver[END_REF]Della Fazia et al., 2018) and is consistent with the overexpression of biomarkers such as PLIN2 [START_REF] Tsai | The constitutive lipid droplet protein PLIN2 regulates autophagy in liver[END_REF] and with the high levels bile acids [START_REF] Puri | The presence and severity of nonalcoholic steatohepatitis is associated with specific changes in circulating bile acids[END_REF][START_REF] Chashmniam | Metabolomic Biomarkers in the Diagnosis of Non-Alcoholic Fatty Liver Disease[END_REF] observed in the biochips.

A major factor of dissociation between the cultures in biochips and Petri dishes was the presence of vascular-related markers and collagen-related proteins in the later (Fig. 5). Meanwhile, AGT (angiotensin), a precursor of angiotensin as well as the production of angiotensin were detected in Petri dishes. The resulting consumption of angiotensin or the low levels of angiotensinogen observed in biochips may be related to either a vasoconstriction phenomenon (Consistent with the high levels of mannitol observed, [START_REF] Kido | CPM is a useful cell surface marker to isolate expandable bi-potential liver progenitor cells derived from human iPS cells. Stem cell reportsliu[END_REF]Shakat et al., 2012), to the transport of specific solutes (K + , Na +, and Ca 2+ , as SLC12A7, SLC4A7 and SLC12A8 were found to be upregulated in biochips) or to the regulation of a potential fibrotic state (Pro-fibrotic via angiotensin 2, ATR1 and TGFB1; anti-fibrotic via angiotensin 1-7, ATR2 and MASR; [START_REF] Shim | Role of the renin-angiotensin system in hepatic fibrosis and portal hypertension[END_REF]. It was previously reported that the activation of MAS receptors would contribute to the reduction of the levels of hydroxyproline and collagen A1 in rats and the increase of vasodilatation [START_REF] Shim | Role of the renin-angiotensin system in hepatic fibrosis and portal hypertension[END_REF]. Those observations would be consistent with the lower levels of trans-4-hydroxy-L-proline and with a potential response to the vasoconstriction phenomenon in biochips. As further confirmations of the observed phenomena, thrombospondin-1 was found to be upregulated in biochips.

This protein is known to be a regulator of blood pressure with a vasopressor activity [START_REF] Isenberg | Thrombospondin-1 and CD47 Regulate Blood Pressure and Cardiac Responses to Vasoactive Stress[END_REF] and induced by the shear stress which may be resulting from the perfused culture in biochips [START_REF] Gomes | Shear Stress Induced Release of Von Willebrand Factor and Thrombospondin-1 in Uvec Extracellular Matrix Enhances Breast Tumour Cell Adhesion[END_REF]. Besides, angiotensin 2 is known to activate SMAD [START_REF] Wang | Essential Role of Smad3 in Angiotensin II-Induced Vascular Fibrosis[END_REF] which might be consistent with the modulation of SMAD3 observed in biochips. As signaling attributed to fibrotic and angiogenetic pathways were reported to be mostly interconnected in the liver regeneration process [START_REF] Poisson | Liver sinusoidal endothelial cells: Physiology and role in liver diseases[END_REF], the separation between the two phenomena is ambiguous in the biochips. Finally, as angiotensin 2 is a pro-fibrotic factor [START_REF] Shim | Role of the renin-angiotensin system in hepatic fibrosis and portal hypertension[END_REF], a potential fibrotic state of the tissue in Petri dishes cannot be excluded.

TGFβ1 was identified as an upstream regulator linking potential tissue inflammation, cell proliferation, and a state of liver regeneration. Indeed, the regenerative liver is characterized by sequential stimulations of the TGFβ signaling [START_REF] Apte | Enhanced liver regeneration following changes induced by hepatocyte-specific genetic ablation of integrin-linked kinase[END_REF][START_REF] Michalopoulos | Liver Regeneration after Partial Hepatectomy, Critical Analysis of Mechanistic Dilemmas[END_REF] but is also involved in inflammation processes with the production of TNFα and IL-6 [START_REF] Manco | Liver Regeneration: Different Sub-Populations of Parenchymal Cells at Play Choreographed by an Injury-Specific Microenvironment[END_REF] and the activation of NF-κB [START_REF] Yang | TNFα in liver fibrosis[END_REF][START_REF] Leist | Activation of the 55 kDa TNF receptor is necessary and sufficient for TNF-induced liver failure, hepatocyte apoptosis, and nitrite release[END_REF]. The latter is known for its crosstalk with the p53 signaling [START_REF] Schneider | Cross talk between stimulated NF-κB and the tumor suppressor p53[END_REF] which may indicate a development towards a tumoral state. This hypothesis might be further enforced as CDKN1C, an inhibitor of cell proliferation [START_REF] Marshall | Hepatocyte Expression of Minichromosome Maintenance Protein-2 Predicts Fibrosis Progression After Transplantation for Chronic Hepatitis C Virus: A Pilot Study[END_REF], was found to be over-expressed in Petri dishes. As patterns typical of the progression of the cell cycle were observed in biochips (overexpression of CDKN2B, CD82 mRNA, CHEK2, SKP2, CENPF, and ORC3), this led to the identification of a pathway related to cancer in the proteomics and transcriptomics datasets. However, the typical signature for tumors was not observed in biochips as neither aerobic respiration nor a Warburg effect was identified and as the typical hepatocellular carcinoma protein biomarkers such as GPC3, GP73, Annexin 2, Osteopontin [START_REF] Lou | Biomarkers for hepatocellular carcinoma[END_REF] were not detected. On contrary, lower levels of AFP were found in the transcriptomics and proteomics datasets while GPC3 was also found to under-expressed, which may suggest that the state of liver regeneration is predominant over the potential apparition of a tumoral development.

Conclusion

In the present manuscript, a protocol for the enhancement of the maturation of hiPSCs-derived HLCs has been proposed and combined with an extensive multi-omics analysis approach. These technical advances have allowed characterizing the tissue which has shown an advanced hepatic differentiation, and which was capable to perform the metabolism of several drugs. Production of urea and glycogen were detected while the metabolism of lipids and conjugation of bile acids was also confirmed. In biochips, analysis of the pathways indicated a possible development of the tissue toward a tumoral state but as the typical hepatocellular carcinoma biomarkers were not identified, the option could be excluded. At first sight, the multiomics analysis led to categorize the tissue in Petri dishes as being in a fibrotic state as shown by the intense production of collagen. However, further investigations revealed a process of liver regeneration in which anti-fibrotic mechanisms were identified. The in-depth analysis performed in this work has allowed a better understanding of the possible crosstalk between liver regeneration processes (including lipogenesis and angiogenesis) and inflammation (along with transient fibrosis and with a steatotic response) during the maturation of hiPSCs-derived HLCs in a microfluidic environment.

Future works including other omics analyses such as epigenetics or Post Translational

Modifications but also other approaches like temporal or spatial studies are expected to complete our comprehension of the dynamic processes in hepatocyte differentiation.

Table 1 :

 1 Top 10 TF motifs extracted in the comparison between biochips and Petri dishes by ISMARA, sorted by z-value. "TF motif" represents a set of transcription factors with similar target nucleotide sequence patterns identified by CAGE. "PCC" is the gene with strongest Pearson correlation coefficient for the identified TF motif. "Pattern" describes the nucleotide sequence pattern of the motif for the PCC.

	TF motif	Z-value	PCC	Up	Pattern	Function
		of motif
		activity
	MECP2	2,08	MECP2 Biochip CCCGGAG	NFKB, NFAT, FOXO
						and JNK pathways
	TAF1	1,93	TAF1	Biochip GCCGCCA	HIF, Beta-catenin,
					TCTT	Mictochondrial
						processes,
						glucokinase regulation
	ZEB1	1,72	ZEB1	Petri	CTCACCTG	EMT
	RELA	1,49	RELA	Biochip GGGAATTT	NFKB, MAPK, JNK and
					CCC	STAT3 pathways,
						inflammation
	SIN3A_CHD1	1,41	SIN3A	Biochip GGAGGCG	E cadherin stabilisation,
					GAGGTGG	Wnt, SMAD2 and NFAT
					GAGAG	pathways,
						cell defense
	MXI1_MYC_MY	1,24	MXI1	Biochip CCACGTG	SMAD2 and MYC
	CN					pathways,
						glucose, fatty acid
						reactomes
	NFKB1	1,19	NFKB1 Biochip GGGGAAT	NFKB response
					CCCC
	ZFX	1,18	ZFX	Biochip GGGGCCG	HDAC, VEGF, Wnt and
					AGGCCTG	IL2 pathways,
						NFKB complex
	FOSL1	1,17	FOSL1	Biochip GTGAGTC	Response to
					A	inflammation (TNF,
						MAPK), response to
						nitrogen level
	ZNF711_TFAP2	1,17	ZNF711 Biochip AGGCCTA	HIF and SMAD2
	A_TFAP2D				G	pathway
						Interactions between
						Notch and Wnt

Acknowledgments

The project was supported by the iLite ANR-16-RHUS-0005. Stéphane Poulain by the JSPS Grant-in-aid for Scientific Research (S) 16H06328 and Charles Plessy by a grant to RIKEN CLST (DGT) from the MEXT, Japan. We acknowledge the 3P5 proteomic platform (Université de Paris, Institut Cochin, INSERM, U1016, CNRS, UMR8104, F-75014 PARIS, France) and François Guillonneau for their precious advice on the proteomic analysis and Cédric Broussard for monitoring the project.

Conflict of interest statement

The authors declare no conflict of interest.

Ethical issue

No ethical approval was required for the work presented in this study.

Online Methods

Complete details on the protocol and corresponding references are given in Supp. File 5.


Anass Bakour
email: bakour.anass@gmail.com

Omar Bajjou

Mimouna Baitoul

Florian Massuyeau

Jany Wéry

Malik Maaza

Eric Faulques

A study of the temperature effect on photoluminescence of the P3HT/MWNT nanocomposites

In spite of the strong competition from recently developed nanomaterials such as graphene, carbon nanotubes continue to be promising materials for photovoltaic applications thanks to their structural advantages and cheap prices of synthesis especially the multi-walled carbon nanotubes (MWNT). Mixing of MWNT with a semiconducting polymer such as the P3HT leads to a composite material characterized with enhanced mechanical, thermal and optical properties. In order to investigate the impact of the insertion of different wt% of MWNTs on the luminescence properties of the P3HT we conducted a series of measurements by means of steady state PL and time transient PL for different samples, as well as the study of the temperature effect on these properties. The results obtained confirm the luminescence quenching effect played by the MWNTs, and prove that the lifetime of photo-generated excitons in the P3HT matrix undergoes a modification after the insertion of the MWNTs which means that a charge separation process takes place at the P3HT/MWNT interface.

Introduction

This work is devoted to the exploration of light energy conversion at the interface between carbon nanotubes (CNTs) and semi-conducting polymer poly(3-hexylthiophene) which have proven to have an interesting optical characteristic. In this context, photo-conversion processes comprise light absorption, photo-induced charge transfer, exciton dissociation and photo-carriers transport as a result of absorbed light energy, which are basic photo-physical principles providing functionality of optoelectronic devices such as photo-detectors, OLED and solar cells. It has been found that the insertion of nanomaterials such as C 60 , CdSe, graphene and their derivatives [START_REF] Huynh | Hybrid Nanorod-Polymer Solar Cells[END_REF][START_REF] Cowan | Recombination in polymer-fullerene bulk heterojunction solar cells[END_REF][START_REF] Saito | Electronic structure of chiral graphene tubules[END_REF] with a higher electron affinity into conjugated polymers [START_REF] Barford | Electronic and Optical Properties of Conjugated Polymers[END_REF][START_REF] Bajjou | Charge carrier dynamics and pH effect on optical properties of anionic and cationic porphyrin-graphene oxide composites[END_REF][START_REF] Bajjou | Synthesis and opto-structural characterization of reduced graphene oxide and meso-tetrakis(4phenylsulfonic-acid) porphyrin composites[END_REF] facilitates the charge separation of photo-generated electron-hole pairs at an interface between the polymer and the electron-accepting material. In the bulk hetero-junction efficient charge separation in the semiconducting polymer/carbon nanotubes composites can be achieved due to the existence of many interfaces for charge transfer, but the performance of photovoltaic applications is limited by the poor carrier transport in composites after charge separation. Thus, the carbon nanotubes provide an enhanced charge transport path that is required to achieve efficient electronic conduction, thanks to their one dimensional conducting properties and hence offering direct pathways for electric conduction [START_REF] Collins | Engineering carbon nanotubes and nanotube circuits using electrical breakdown[END_REF]. In this study, we investigated carrier dynamics with respect to charge separation and transport in conjugated P3HT/Multiwalled carbon nanotubes composites by temperature dependent steady state photoluminescence (PL) and time-resolved photoluminescence (TRPL) spectroscopy.

Material and methods

The regioregular P3HT (RR-P3HT) used in this study (provided from Rieke) has the following characteristics: Average molecular weight MW = 37.680 g mol-1, PDI = 1.48 (polydispersity) and regio-regularity HT (head-tail)> 98.5%. Multiwalled carbon nanotubes used in this work were obtained Arkema synthesized by CVD and then purified by HCl.

In our work we used the mixing process in solution, the polymer matrix is first placed in solution in chloroform. The nanotubes on the other hand are dispersed in the same solvent under ultrasonic agitation. The CNT solution is then added to the polymer matrix solubilized under stirring. The nanotubes in this way can be introduced between the macromolecular chains of the polymer. The composite solution is then drop casted on silica substrate and left to dry at 50° C.

The Steady-state in photoluminescence measurements were performed at different temperatures using a Jobin-Yvon Fluorolog 3 spectrometer using a Xenon lamp (500 W) and a CCD detector, temperature was controlled by a cryostat coupled with the Fluorolog cooled by liquid nitrogen. Time-resolved photoluminescence (TR-PL) experiments were performed with a regenerative amplified femtosecond Ti:Sapphire laser system (Spectra Physics Hurricane X). This setup generates 100 fs pulses at 800 nm with a repetitive rate of 1 kHz and a nominal power of 1 W. The laser line is frequency-doubled with a thin BBO crystal to obtain an excitation  exc =400 nm. The pump energy pulse is controlled by means of neutral density filters (0.1 mJ with diameter laser beam of 5 mm falling on the sample) to ensure that the excitation density on the sample do not exceed 1017cm -3 and to avoid bimolecular annihilation process and sample photo-degradation. Emission spectra are temporally resolved with a high dynamic range Hamamatsu C7700 streak camera coupled to an imaging spectrograph with a temporal resolution less than 20 ps and processed using the HPDTA Hamamatsu software.

Results and discussions 3.1. Steady state photoluminescence

The effect of temperature on the steady state PL is studied because of the strong relation between the electronic band gap of P3HT/MWNT composites and determining the role of the electron-phonon interactions [START_REF] Capaz | Temperature dependence of the band gap of semiconducting carbon nanotubes[END_REF].

To investigate the transport and luminescence mechanism of the P3HT/MWNT composite, steady state photoluminescence was measured as a function of temperature within the range of [90°K, 300°K], by a difference of 40°K between each measure controlled by means of a cryostat with liquid nitrogen, at an excitation wavelength of 420 nm. P3HT light absorption is an initial step leading to the generation of bound excitons. In order to convert light energy into an electrical signal, the excitons should be separated into free charge carriers (electrons and holes), at the P3HT/MWNT interface. Theoretically, the absorption and emission spectra are symmetrical with respect to the energy of the zero-phonon band for which light emission takes place at the highest energy [START_REF] Guillet | Emission of a single conjugated polymer chain isolated in its single crystal monomer matrix[END_REF]. The P3HT photoluminescence spectrum at room temperature showed in Fig. 1(a) exhibits usually two peaks. In the literature, these two peaks are commonly assigned to the transitions between the vibrational energy levels (0-0) and (0-1). The (0-0) peak at 664 nm (1.86 eV) is often less intense than the (0-1) peak centered at 715 nm (1.73 eV). How-ever, the zero-phonon line is usually in real systems the most intense line and it is consistent with results obtained by other researchers [START_REF] Clark | Determining exciton bandwidth and film microstructure in polythiophene films using linear absorption spectroscopy[END_REF]. In Fig. 1(a) we notice an overall gradual redshift in the spectrum of P3HT, especially in the (0-1) peak that shows a strong shift to 740 nm when the temperature decreases to 90°K, and a less pronounce redshift in the (0-0) peak to 674 nm. Along with this shift, we note a change in the relative intensity of the two characteristic peaks while varying the temperature gradually.

In order to give a better insight of this effect we exhibit in the Fig. 1(c) the intensity ratio of the 0-0 band to the (0-1) band as a function of the temperature, this ratio is related to the effective Huang-Rhys parameter of the emitting state [START_REF] Barford | Theory of exciton transfer and diffusion in conjugated polymers[END_REF][START_REF] Spano | Vibronic coupling in J-aggregates and beyond: a direct means of determining the exciton coherence length from the photoluminescence spectrum[END_REF], in this plot we can see that the value of I (0-0) /I (0-1) decreases from 0.75 at a 90°K to 0.47 at room temperature.

In Fig. 1(b) we show the progress of the P3HT/MWNT composite film PL spectrum at 2 wt% of MWNT, as a function of temperature in the same range used for the pristine P3HT film. The overall trend of the spectrum is to move toward lower energy confirming the redshift already noted for the P3HT sample. The (0-0) peak at room temperature shows a slight shift to 657 nm when compared to the P3HT spectrum, this can be attributed to an increased percentage of short segments over long segments in the matrix owing to the wrapping effect of the P3HT over the MWNTs [START_REF] Baghgar | Effect of polymer chain folding on the transition from H-to J-aggregate behavior in P3HT nanofibers[END_REF][START_REF] Khenfouch | Optical properties and dynamics excitation relaxation in reduced graphene oxide functionalized with nanostructured porphyrins[END_REF]. Although, this shift doesn't persist when cooling the samples, we notice that the two bands start to have the same respective positions for both samples due to the reduction in thermal disorder in predominant component which is the polymer. The main peaks intensity ratio for the P3HT/MWNT 2 wt% is shown in Fig. 1(d) exhibiting a decrease from 0.74 at 90°K to 0.51 at room temperature confirming the tendency of a more ordered regions and enhanced conjugation length of the polymer chains by reducing the conformational disorder. The excited states are therefore delocalized over the more conjugated polymer chain, giving lower-energy emission.

It is observed that the normalized PL spectra of P3HT and composite films P3HT/MWNT at 2 wt% of MWNT show a slight shift when compared on the concentration base at room temperature, except the effect of quenching confirming the role of the MWNT in favoring the exciton dissociation at the P3HT/MWNT interface. The extinction of the photoluminescence could also be due to the presence of aggregates in the composite material, or due to the existence of some defects on the surface of carbon nanotubes. This effect is also observed when cooling the samples and thus confirming the elimination of the phonons contributing to the non radiative recombination in the samples.

3.2.Time Transient Photoluminescence

The luminescence decay of the excited states involves two com-peting processes, radiative and non-radiative processes. The rela-tive proportion of these processes determines the efficiency of luminescence. If the non-radiative channel is very fast, a significant portion of photogenerated excitations will decline through that pathway before having the time of issuing radiatively. The results found in the Steady state PL hereinbefore suggesting enhanced electronic delocalization in the composite films can be further investigated by the time-resolved PL analysis. Fig. 2 Shows the time resolved PL of the P3HT and P3HT/MWNT 2 wt% composite films at room temperature, at an excitation wave-length of 420 nm. The fitting of the decay patterns of the P3HT and P3HT/MWNT 2 wt% composite is shown in Fig. 2(b) and (c) respectively, using a bi-exponential decay model [START_REF] Massuyeau | Steady state and transient photoluminescence in poly-p-phenylene vinylene films and nanofibers[END_REF][START_REF] Bakour | Effects of single-walled carbon nanotubes on the optical and photo-conductive properties of their composite films with regio-regular poly(3hexylthiophene)[END_REF]. It is observed in Table 1 that the mean lifetime τ mean exhibits a decrease when inserting a 2 wt% concentration of nanotubes in the P3HT, suggesting that the decline is slightly faster for the 2 wt% composite. The decreasing PL lifetime is attributed to efficient charge separation from the increase of non-radiative pathway created by the insertion of MWNT. The remaining non recombined excitons which contribute to the remaining luminescence in polymer will migrate over more conjugated region before recombination [START_REF] Goutam | Photoluminescence quenching of poly(3hexylthiophene) by carbon nanotubes[END_REF]. We can therefore conclude that there is indeed a notable dissociation of charges due to the presence of nanotubes that causes the appearance of a greater probability of non-radiative process. Moreover, the contribution of rapid exponential first became prominent in the decline and thus the lifetimes decrease. This separation is made possible by the migration of excitations from short segments to long segments. During this migration, a significant fraction of excitations hops on the MWNT network on which they dissociate non-radiatively [START_REF] Baghgar | Effect of polymer chain folding on the transition from H-to J-aggregate behavior in P3HT nanofibers[END_REF]. In future, it is intended to optimize the current system in view of validating its broad white luminescence and other potential application [START_REF] Khenfouch | Photoluminescence and dynamics of excitation relaxation in graphene oxide-porphyrin nanorods composite[END_REF][START_REF] Maaza | White photoluminescence from a grown ZnO nanorods/graphene hybrid nanostructure[END_REF][START_REF] Oseni | ZnO: CNT assisted charge transport in PTB7: PCBM blend organic solar cell[END_REF].

Sample

Conclusion

In this work we have conducted an investigation on the photo-luminescence and the charge separation processes in the P3HT and P3HT/MWNT composite as well as the effect of temperature on these properties. It has been demonstrated by means of steady state PL that the insertion of MWNT in the P3HT matrix leads to an efficient PL quenching due to the nucleant agent role played by MWNTs, as well as the phonon contribution to the luminescence that was investigated by varying measurements temperature. This result reveals is the characters of enhanced short segment conjugation over long segments in the polymer/MWNT composite systems. Exciton dissociation and migration due to enhanced electronic delocalization in polymer after adding MWNT was demonstrated by TRPL measurements and thus confirming the charge separation at the P3HT/MWNT interface. These effects are of crucial importance for the application of the studied composites in heterojunction organic photovoltaic cells.

Fig. 1 :

 1 Fig. 1: Temperature dependent Normalized PL spectra of : (a) P3HT film. (b) P3HT/MWNT 2wt% film. Intensity ratio of the (0-0) to the (0-1) peaks as a function of temperature of (c) P3HT, (d) P3HT/MWNT 2wt%

Fig.. 2

 2 Fig.. 2 TRPL emission decays of: (a) P3HT and P3HT/MWNT 2wt% composite film. And decay fitting of (b) P3HT, (c) P3HT/MWNT 2wt%

Acknowledgements

This research was partially supported by the Institute of Mate-rials Jean Rouxel Nantes, France. We thank our colleagues from IMN who provided Materials, insights and expertise that greatly assisted the research, and for their comments that greatly improved the manuscript.

Declaration of Competing Interest

The authors declare that they have no known competing finan-cial interests or personal relationships that could have appeared to influence the work reported in this paper.


Lina Achaji
email: lina.achaji@stellantis.com

Julien Moreau
email: julien.moreau@stellantis.com

Thibault Fouqueray
email: thibault.fouqueray@stellantis.com

Francois Aioun
email: francois.aioun@stellantis.com

Francois Charpillet
email: francois.charpillet@inria.fr

Is attention to bounding boxes all you need for pedestrian action prediction?

The human driver is no longer the only one concerned with the complexity of the driving scenarios. Autonomous vehicles (AV) are similarly becoming involved in the process. Nowadays, the development of AV in urban places underpins essential safety concerns for vulnerable road users (VRUs) such as pedestrians. Therefore, to make the roads safer, it is critical to classify and predict their future behavior. In this paper, we present a framework based on multiple variations of the Transformer models to reason attentively about the dynamic evolution of the pedestrians' past trajectory and predict its future actions of crossing or not crossing the street. We proved that using only bounding-boxes as input to our model can outperform the previous state-of-the-art models and reach a prediction accuracy of 91% and an F1-score of 0.83 on the PIE dataset up to two seconds ahead in the future. In addition, we introduced a large-size simulated dataset (CP2A) using CARLA for action prediction. Our model has similarly reached high accuracy (91 %) and F1-score (0.91) on this dataset. Interestingly, we showed that pre-training our Transformer model on the simulated dataset and then fine-tuning it on the real dataset can be very effective for the action prediction task.

I. INTRODUCTION

During every moment of our waking life, our brains are trying to predict what sights, sounds, and tactile sensations will be experienced next [START_REF] Hawkins | Sequence memory for prediction, inference and behaviour[END_REF]. During motion continuation, viewers observe the trajectory of a moving object and simultaneously simulate its future trajectory, predicting where it is likely to move next. Knowledge about whether an object moves quickly or slowly affects these predicted trajectories accordingly [2]. The shift in our mobility system to the autonomous driving era is often regarded as adding an artificial layer of cognitive intelligence on top of basic vehicle platforms [3]. This layer of intelligence should be capable of not only perceiving the world, but also predicting and analyzing its future states. The predictive processing can be applied everywhere, especially when interacting with vulnerable road users (VRU) such as pedestrians. When dealing with pedestrians, we can consider prediction as a forecasting of the future state or states. For instance, these states can be the future positions of the pedestrian in the case of trajectory prediction. Also, we can formulate it as a higher-level semantic prediction such as the early anticipation of the future action of the pedestrian, for example, walking, running, performing hand gestures, or most importantly crossing or not crossing the street in front of the AV. Recently, trajectory and action prediction solutions have been proposed based on sequential reasoning that mainly use algorithms built on recurrent neural networks (i.e., RNN, LSTM) [4, [START_REF] Ovidiu Pop | Multi-task deep learning for pedestrian detection, action recognition and time to cross prediction[END_REF][START_REF] Kotseruba | Benchmark for evaluating pedestrian action prediction[END_REF][START_REF] Poibrenski | M2p3: multimodal multi-pedestrian path prediction by self-driving cars with egocentric vision[END_REF]. However, it has recently become clear that LSTM lacks many capabilities to model sequential data. For instance, LSTM suffers from long-term prediction, often due to the vanishing gradient problem [START_REF] Sutskever | Sequence to sequence learning with neural networks[END_REF]. That leads to its inability to model the correlation between non-neighboring inputs in the sequence. Furthermore, during training, LSTM is not able to assign different weights to different tokens based on their relative importance to the output. This will force it to give equal attention to all inputs, even if the capacity of the model does not allow it. Hence, the attention coupled with LSTM [9] has enhanced the previously suggested solution by proposing a mathematical framework that can weigh each of the input tokens differently depending on their importance to the input sequence itself (self-attention) [START_REF] Lin | A structured self-attentive sentence embedding[END_REF] or the output sequence (cross-attention). Nevertheless, attention mechanisms coupled with LSTM have limited the potential of "Attention" itself. We can see that using only attention "Transformer architecture" [11] can lead to better results. Transformers have first revolutionized the natural language processing problems by outperforming all the previously proposed solutions [12,13]. However, it was only until recently that were proved equally efficient for non-NLP problems [14,[START_REF] Giuliari | Transformer networks for trajectory forecasting[END_REF][START_REF] Aksan | Attention, please: A spatio-temporal transformer for 3d human motion prediction[END_REF].

In this paper, we will apply Transformer Networks for the pedestrian action prediction task of crossing or notcrossing the street. This network will predict the crossingprobability of a pedestrian in front of the ego-vehicle using an observation sequence of 0.5 seconds with a prediction horizon varying between one and two seconds in the future. To our knowledge, we believe that we are the first to apply Transformers for this task. We will propose multiple variants of the Transformer architecture: the Encoder-alone architecture, the Encoder coupled with pooling-layers, and the Encoder-Decoder architecture. Our model will use only bounding boxes as input features. Moreover, it will predict the crossing probability with accuracy and F1-score that will outperform the previous benchmarks even when using more features as input [START_REF] Kotseruba | Benchmark for evaluating pedestrian action prediction[END_REF]. These results are evaluated on the PIE dataset [4], and on a simulated dataset (CP2A) that we generated using the CARLA simulator. The latter dataset is larger in size than PIE, and will equally confirm the high efficiency of our models. Additionally, we will show that the pre-trained weights on the CP2A dataset will increase the efficiency of our fine-tuned models on the PIE dataset.

II. RELATED WORK

In this section, we review the recent advances in pedestrian action and trajectory prediction. Then, we discuss the work done using Transformers in various field of applications and the effect of Transfer learning on Transformer models. Pedestrian Action Anticipation is a highly important problem for autonomous cars. Where the objective is to anticipate in advance if a pedestrian will cross the road in front of the ego-vehicle. This problem was addressed using multiple approaches. For instance, [START_REF] Ovidiu Pop | Multi-task deep learning for pedestrian detection, action recognition and time to cross prediction[END_REF] used a network of convolutional neural networks and LSTM to detect the pedestrians and predict their actions up to 1.3 seconds ahead in the future and their estimated time-to-cross. [START_REF] Rasouli | Pedestrian action anticipation using contextual feature fusion in stacked rnns[END_REF] used a stacked GRU network composed of five GRUs each of which processes a concatenation of different features (i.e., the pedestrian appearance, the surrounding context, the skeleton poses, the coordinates of the bounding boxes, and ego-vehicle speed). [18] converted the human pose skeleton sequences into 2D image-like Spatio-temporal representations and then applied CNN-based models. [START_REF] Kotseruba | Benchmark for evaluating pedestrian action prediction[END_REF] has published a benchmark to evaluate action prediction on the PIE [4] and JAAD [START_REF] Bahdanau | Neural machine translation by jointly learning to align and translate[END_REF] datasets. Additionally, they proposed a new state-of-the-art model that uses attention mechanisms to combine implicit and explicit features. Pedestrian Trajectory Prediction is a closely related task to the action prediction. In contrast, the output sequence is a set of predicted positions in the future. In recent works, we see that intention and action prediction can be critical for trajectory prediction [4]. The M2P3 model [START_REF] Poibrenski | M2p3: multimodal multi-pedestrian path prediction by self-driving cars with egocentric vision[END_REF] used an Encoder-Decoder RNN along with conditional variational auto-encoder (CVAE) to predict multi-modal trajectories from an ego-centric vehicle view. GRIP++ [20] builds a dynamic GNN (Graph Neural Network) to model the interaction between agents in the scene. The Trajectron [21] combines elements from CVAE, LSTM, and dynamic Spatio-temporal graphical structures to produce multimodal trajectories. Recently, a Transformer model was proposed by [START_REF] Giuliari | Transformer networks for trajectory forecasting[END_REF] to predict the future trajectories of the pedestrians conditioning on the previous displacement of each pedestrian in the scene. The transformer used has the same architecture as the Vanilla Transformer proposed in [11]. Transformers Networks are self-attention-based models proposed by [11] for machine translation and have since become the state-of-the-art method in many NLP tasks. Here we focus on the development of Transformers for modeling actions, motion dynamics, and visual context. For instance, [START_REF] Girdhar | Video action transformer network[END_REF] introduced an Action Transformer model for recognizing and localizing human actions in video clips. [START_REF] Aksan | Attention, please: A spatio-temporal transformer for 3d human motion prediction[END_REF] proposed a Spatio-temporal transformer for 3D human motion modeling by learning the evolution of skeleton joints embeddings through space and time. Also, [23] proposed the use of transformer models for the prediction of dynamical systems representative of physical phenomena. Recently, [START_REF] Bertasius | Is space-time attention all you need for video understanding?[END_REF] and [START_REF] Arnab | Vivit: A video vision transformer[END_REF] applied a Spatio-temporal transformer for video action recognition. Similar to the NLP-oriented Transformers, where Large Transformer-based models are often pre-trained on large corpora and then fine-tuned for a particular task, [START_REF] Bertasius | Is space-time attention all you need for video understanding?[END_REF][START_REF] Arnab | Vivit: A video vision transformer[END_REF] used a pre-trained Image Transformer [START_REF] Dosovitskiy | An image is worth 16x16 words: Transformers for image recognition at scale[END_REF] weights, bootstrap the weights, and then fine-tuned the model on a very large dataset [START_REF] Carreira | Quo vadis, action recognition? a new model and the kinetics dataset[END_REF] for action recognition. This training on the large dataset has remarkably increased the performances of Transformers for visual action recognition.

III. METHOD

The prediction and analysis block in autonomous car architectures receives its inputs from the perception block. We assume that the sets of observations of every agent are arranged in sequences. Every sequence describes the states on a timeordered scale from a specific time-step t = m, to another time-step t = M : O m:M = {O m , . . . , O M }. Conditioning on the whole history of observations, the prediction block will answer the question of "what will be the future state (or states) at the time t = A". More formally, the objective is to calculate the following probability distribution:

p( P a:A | O m:M )
Where P a:A = {P a , . . . , P A }, and P t is the predicted state of a particular object or agent at time t. For the case of trajectory prediction, a is equal to the next time-step M + 1 of the end-frame in the observation sequence. For the action early anticipation, a is equal to A, where A is the time that the pedestrian starts to cross, or the last frame the pedestrian is observable in case no crossing takes place. The time between the last observation frame M in the observation sequence and the critical time A is called the Time-To-Event (TTE).

The input to our model is composed of the bounding box coordinates of each pedestrian in the scene. The bounding box is defined as the four coordinates (x_up, y_left, x_down, y_right) around the pedestrian, without the use of images or visual context. Certainly, these bounding boxes were first extracted from images, either manually (i.e., in the case of annotated datasets) or using a state-of-the-art object detector.

To predict the future states, we shall first build a representative model of the world or the environment around the vehicle. In the following section, we will present our Transformer Network based model.

A. Input Embedding

Before feeding the input O m:M = X T ∈ R T ×4 to our encoder, we first project the 4-dimensional bounding boxes into a D-dimensional space via a linear layer.

E T = W e X T + b e Where E T ∈ R T ×D . W e ∈ R 4×D
and b e ∈ R D correspond to the learnable weight and bias that project the bounding boxes vector into the embedding space.

Unlike RNNs or CNNs, the Transformer has no notion of ordering within a sequence. Following [11], we inject sinusoidal positional encoding into the input embeddings.

B. Attention Layer

Given a set of items, the attention mechanism estimates the relevance of one item to other items. Basically, an attention layer updates each component of a sequence by aggregating global information from the complete input sequence. This is done by defining three learnable weight matrices to transform Queries (W Q ∈ R D×dq ), Keys (W K ∈ R D×d k ), and Values (W Q ∈ R D×dv ). In our model,

d q = d k = d v = D.
The output of the attention layer is then given by:

Attention(Q, K, V, M ) = sof tmax(QK T / (d k ) + M )V
Where the mask M prevents information leaking from future steps.

Following [11], we use a multi-head attention (MHA) mechanism to project the D-dimensional representation into subspaces calculated by different attention heads i ∈ {1, . . . , H}. We concatenate all the output vectors to get the final MHA output:

M HA(Q i , K i , V i ) = Concat(head 1 , . . . , head h )W O Where Q i , K i , and V i ∈ R T ×F . We set F = D/H. The head i = Attention(QW Q i , KW K i , V W V i ), and W O ∈ R D×D .

C. Transformer Architecture

We proposed to apply multiple variations of the Transformer model:

1) Encoder-Only Architecture: The encoder-only model takes the X T as input, proceeds into the embedding layer, and then goes into a stack of N encoder layers. Each encoder layer consists of a multi-head self-attention layer and pointwise feed-forward neural networks (P-FFN). Both layers are wrapped by a residual connection and layer normalization. We define h n ∈ R T ×D as the output of the n-th encoder layer, n ∈ {1, . . . , N }:

h n = LayerN orm(h n-1 +M HA(Q = h n-1 , KV = h n-1 )) h n = LayerN orm(h n + P -F F N (h n ))
The output sequence length of the encoder layers remains the same as the input ∈ R T ×D . However, to predict the action of crossing or not crossing the street, we compress the output sequence in the temporal domain by applying a global average pooling layer at the top of the encoder stacks. Then we apply another embedding layer followed by a sigmoid function to get the action crossing probability.

2) Encoder-Pooling Architecture: Instead of applying the global average pooling at the top of the encoder stack of layers directly, we proposed pooling intermediate layers between the encoder blocks (Fig. 1 middle). We see this solution as a compact way to reduce the observation sequence length gradually from the input to the output as the layer gets deeper. This solution will prevent the network from directly transforming the last layer embedding size from R T ×D to R 1×D to predict the one-dimensional action. Instead, the final sequence length will be T , a reduced transformation of the initial sequence size T . In fact, pooling layers have been used in CNN architectures to reduce the dimensions of the feature maps. Similarly, [START_REF] Dai | Funnel-transformer: Filtering out sequential redundancy for efficient language processing[END_REF] proposed to adapt the pooling layers to achieve representation compression and computation reduction in transformer networks. At the output of each encoder layer, we will apply a strided mean pooling layer on the query vector to reduce its sequence length:

h n-1 = pooling(h n-1 ),
where h ∈ R T ×D for T < T , and the unpooled sequence h serves the role of the key and value vectors:

h n = LayerN orm(h n-1 +M HA(Q = h n-1 , KV = h n-1 ))
3) Encoder-Decoder Architecture: The final architecture is the encoder-decoder. The encoder block is identical to the encoder-only architecture with the classification head on the top of the encoder stacks. For the decoder architecture, we input the target sequence T a:A = Y C ∈ R C×4 , where a is equal to the next-observation time frame M + 1, and A is equal to the critical time TTE. Each decoder layer consists of a masked multi-head self-attention layer, a masked multihead cross-attention layer, and point-wise feed-forward neural networks (P-FFN). All layers are wrapped by a residual connection and layer normalization. The cross-attention layer takes the query from the previous self-attention layer plus the keys and the values from the encoder memory vectors:

h decn = LayerN orm(h decn-1 + M HA(Q = h decn-1 , KV = h enc N ))
The output of the decoder block is the target input shifted by one representing the future trajectory of the pedestrian from time step a + 1 to A + 1. This architecture will jointly learn to classify the pedestrian's future action as well as its future trajectory. In the following sections, we will see that jointly learning to predict the action and the trajectory is beneficial to both parties, where knowledge of the trajectory prediction increases the performance of the action prediction and vice versa.

D. Training and Inference

We train the encoder-only and the encoder-pooling models by optimizing the classification loss between the predicted and target class. For the encoder-decoder model, we use a combined weighted loss of the classification binary cross entropy (BCE) and the l 2 distance between the predicted and target sequence:

L = λ cls BCE + λ reg l 2 (Y C+1 , ŶC+1 )
Where λ cls and λ reg are the classification and regression hyperparameters to be tuned.

IV. EXPERIMENTS

In this section, We evaluate our proposed models on two datasets following the evaluation protocols in [START_REF] Kotseruba | Benchmark for evaluating pedestrian action prediction[END_REF]. We compare our model results with the baselines based on different features choices. We fix The observation length for all models at 16 frames (i.e., 0.5 seconds) and the Time-To-Event (TTE) between 30 and 60 frames (i.e., 1 and 2 seconds). We examine different settings of model structure (ablation studies) and explore the effect of changing the temporal prediction horizon on the results. Also, we investigate the effect of transfer learning on our models. The dataset represents a wide variety of pedestrian behaviors at the crossing point and includes locations with high pedestrian traffic and narrow streets. For each pedestrian who can potentially interact with the ego vehicle driver, it provides the coordinates of the bounding boxes, the critical time where each pedestrian will cross, as well as their actions of crossing or not crossing the street.

2) CARLA Pedestrian Action Anticipation (CP2A) dataset: In this paper, we present a new simulated dataset for pedestrian action anticipation collected using the CARLA simulator [31]. Indeed, Transformer networks have shown an advantage over other types of algorithms when a large amount of training data is available [START_REF] Dosovitskiy | An image is worth 16x16 words: Transformers for image recognition at scale[END_REF]. Moreover, pre-training Transformer networks on a large corpus of data and then fine-tune it on pre-text tasks has proven to be very beneficial [START_REF] Bertasius | Is space-time attention all you need for video understanding?[END_REF]. To this end, we have simulated a very large dataset (see Table I) that can be automatically and freely labeled using the simulator itself. Using such a dataset, we can first evaluate our model and then improve the overall performance using transfer learning on the real collected scenarios such as the PIE dataset. To generate this dataset, we place a camera sensor on our ego-vehicle and set the hyper-parameters to those of the camera used to record the PIE dataset (i.e., 1920x1080, 110°FOV). Then, we compute bounding boxes for each pedestrian interacting with the ego vehicle as seen through the camera's field of view. We stored the bounding boxes information, along with the current semantic label of the pedestrians, and their critical time to cross the street. We generated the data in two urban environments available in the CARLA simulator: Town02 and Town03. Samples from the CP2A dataset are shown in (Fig. 2). We used an Nvidia 1080 Ti GPU for data simulation with a generation rate of about 600 pedestrian sequences in one hour, which corresponds to the size of the PIE dataset in 2 hours.

B. Baseline Models

Recently, [START_REF] Kotseruba | Benchmark for evaluating pedestrian action prediction[END_REF] have published a benchmark on the PIE dataset for evaluating pedestrian action prediction. They unified multiple baseline models under the same evaluation procedures and analyzed their performance with respect to various properties of the data. Also, they introduced a new 2) Multi-stream RNN (MultiRNN) [33]: it is composed of separate GRU streams independently processing the following feature types: Bounding boxes coordinates, the pose skeletons, the ego-vehicle speed, and the local box frame cropped around the bounding box that is processed by a VGG16 backend.

3) Inflated 3D (I3D) network [START_REF] Carreira | Quo vadis, action recognition? a new model and the kinetics dataset[END_REF]: it takes as input a stack of RGB frames or Optical Flow and generates final prediction using a fully connected layer.

4) PCPA [START_REF] Kotseruba | Benchmark for evaluating pedestrian action prediction[END_REF]: it is composed of multiple RNN branches to encode non-visual features (e.g., bounding boxes coordinates, the pose skeletons, and the ego-vehicle speed) along with a C3D network to encode each pedestrian's local context. The outputs of the branches are then fed into modality attention layers.

C. Results on PIE dataset

Table II shows the comparison of the results of our proposed models with the state-of-the-art baselines on the PIE dataset. Our Transformer models TEO (Transformer Encoder-only), TEP (Transformer Encoder-Pooling), and TED (Transformer Encoder-Decoder) based only on Bounding Boxes features (BB) outperformer the state-of-the-art baselines that use BB, PoseSkeletons (P), Ego-vehicle speed (S), and RGB images features in term of accuracy. The TED model is the best in terms of ACC (92 %), F1-score (0.86), and AUC (0.9). This result exceeds the baselines' results by a large margin without using any of the other features. It emphasizes the impact of learning to predict the future position states and actions jointly and end-to-end. Additionally, we show outstanding results when using Transformers by comparing the PCPA model based on attention modules coupled with LSTM layers using only the BB features. In CP2A** Trans BB 0.9 0.9 0.9 the latter case, the PCPA model results will drop dramatically and prevent the model from learning any significant pattern about the action prediction task.

D. Ablations

Figure 3 shows the performance of our TEO model when trained with different numbers of layers and heads. We obtained the best performance for the 4-layer and 8-head configurations (shown in Table II) and 8-layer, 2-head configurations. For the TEP and TED, the best model goes for using the 8-layer and 8-head settings. We experiment with different hyperparameters for the regression and classification values in the TED loss function settings. We obtained the best model (Fig. 4) using the (1.8, 0.8) combination for the regression and classification parameters respectively. In our experiments, we use an embedding dimension D of 128, a , we noticed that using stochastic optimizers for our Transformer models was not stable, and the results will change remarkably from one experiment to another using the same hyper-parameters.

We study how the prediction performance changes when we vary the temporal prediction horizon (equivalent to TTE time). We set the length of the observation sequence to 16 frames (i.e., 0.5 s) and varied the TTE time from 0.5 seconds to 3 seconds in the future (90 frames). These results were reported on the test set by taking the same models that were trained on the original 30-60 frame TTE interval. The graphs (Fig. 5) show that our models reach their upper limit when predicting the upcoming 1 to 1.3 s interval with an accuracy of 93% for the TED model. Furthermore, the model still performs reasonably well even when all prediction horizons are 2 to 2.3 seconds, with an accuracy of about 80% for all three models. We performed all our experiments on an Nvidia 1080 Ti GPU. We reported the inference time (Table III) of the three proposed models using their best hyperparameters scenarios. We should note here that we can use only the encoder block of the TED model in the inference phase. This strategy reduces the inference time since we predict the future action without the future pedestrian trajectories.

E. Results on CP2A

Previously presented results on using Transformer models with only bounding boxes as features and outperforming the models even using other features have been controversial because, in general, action prediction is related to understanding the visual context. The first assumption we can make here is that the PIE dataset is biased. A biased dataset is one where we can estimate the output independently of the input but rather based on some constant patterns that are uniquely related to that dataset. Although previous models were not able to detect these patterns, Transformers was able to find this bias. To test this hypothesis, we trained our models on the CP2A dataset, which is 43x larger than the PIE dataset and is highly unlikely to have the same bias that can be exhibited in the PIE dataset. The raw CP2A in Table II shows the performance of the 8-layer, 8-head TEO model on the simulated dataset. It achieved 90% accuracy and 0.9 for AUC and F1-score, demonstrating the effectiveness of using transformers with bounding boxes, regardless of the choice of dataset. 1) Effect of Transfer Learning: Interestingly, another result we obtained is that using the pre-trained weights from the CP2A model and fine-tuning them on the TEO model using the PIE dataset also has an advantage over the PIE results (FTEO model in Table II). In particular, we can see the improvement in terms of F1 score, where we reached 0.88, which outperforms the TED model trained from scratch on PIE, and slightly outperforms the baseline TEO model in terms of accuracy and AUC. In brief, the transfer of knowledge about action prediction from CP2A to PIE was effective. The results obtained here are consistent with the advantage observed in computer vision [START_REF] Bertasius | Is space-time attention all you need for video understanding?[END_REF] when applying transfer learning from larger datasets.

V. CONCLUSION

In this paper, we presented transformer-based models for predicting pedestrians' action to cross or not cross the road in front of vehicles from an egocentric perspective. We have shown that using a simple and lightweight type of inputs (i.e., bounding boxes) with Transformers networks achieves high performance and outperforms the state-of-theart in predicting the future up to 2 seconds in advance. The Encoder-decoder Transformer was the best model in terms of accuracy (91%) and F1-score (0.83) when we train from scratch on the PIE dataset. These findings make it clear that jointly predicting action and trajectory can be beneficial for both parties. We also introduced the CP2A simulated dataset that confirmed our results on the PIE dataset with accuracy and an F1-score of 0.9. Also, applying Transfer learning on the CP2A dataset from simulated scenarios to real cases improved our models by increasing the F1-score to 0.88 in the Encoder-alone architecture. In future work, we will apply Transformer models to visual context directly instead of bounding boxes. The goal of using visual features will not be to improve model performance in terms of accuracy, AUC, and F1-score, as it is difficult to beat the results obtained. However, it will help us focus on other types of criteria such as the explainability and interpretation of the model.

Fig. 1 :

 1 Fig. 1: (Left) Encoder Alone Architecture; (Middle) Pooled-Encoder Architecture; (Right) Encoder-Decoder Architecture.

Fig. 2 :

 2 Fig. 2: Samples from the simulated CP2A dataset in different scenarios.

Fig. 3 :

 3 Fig. 3: Performance of the TEO model when trained with different number of attention heads and attention layers.

Fig. 4 :

 4 Fig. 4: Performance of the TED model when trained with different regression and classification hyper-parameters.

Fig. 5 :

 5 Fig. 5: Changing the prediction time horizon range for the models trained on the [30, 60] frames TTE interval.

TABLE I :

 I Comparison of the CP2A dataset with other pedestrian action prediction datasets.

	Dataset	Running Time	# Peds	S/R
	PIE [4]	6 hours	740k	Real
	JAAD [29]	1.5 hours	391k	Real
	STIP [30]	15.4 hours	3.5M	Real
	CP2A (ours)	5.5 days	232.7M Simulated
	A. Datasets
	1) Pedestrian Intention Estimation (PIE) dataset: The
	PIE dataset [4] provides 6 hours of continuous footage
	recorded at 30 frames per second (FPS) in clear weather con-
	ditions.

TABLE II :

 II Accuracy (ACC), F1-score, and Area Under Curve (AUC) comparison with baseline models. TEO: Transformer Encoder-only, TEP: Transformer Encoder-Pooling, TED: Transformer Encoder-Decoder, FTEO: Fine-tuned Transformer Encoder-only, BB: Bounding Boxes, P: Pose-Skeletons, S: Ego-vehicle Speed. *: Ours. **: Trained on the CP2A dataset.

	Model Name	Architecture	Features	ACC AUC F1-score
	Static I3D	CNN	RGB Optical flow	0.71 0.80 0.81	0.60 0.73 0.83	0.41 0.62 0.72
	MultiRNN PCPA	GRU LSTM + ATT	BB, P, S, RGB	0.83 0.87	0.8 0.86	0.71 0.77
	PCPA	LSTM + ATT		0.48	0.42	0.57
	TEO* TEP*	Trans	BB	0.88 0.88	0.85 0.87	0.77 0.77
	TED*			0.91	0.91	0.83
	FTEO*	Fine-tuned Trans	BB	0.89	0.89	0.88

TABLE III :

 III Inference Time on Nvidia 1080 Ti GPU.

	Model Name Number of Layers Inference Time (ms)
	TEO	4	1.63
	TEP	8	2.85
	TED	8	2.76

ACKNOWLEDGEMENT

This work was carried out in the framework of the Open-Lab "Artificial Intelligence" in the context of a partnership between INRIA institute and Stellantis company.