Alphy John

Matteo Rauzi
email: matteo.rauzi@univ-cotedazur.fr

Composite morphogenesis during embryo development

Morphogenesis drives the formation of functional living shapes. Gene expression patterns and signaling pathways define the body plans of the animal and control the morphogenetic processes shaping the embryonic tissues. During embryogenesis, a tissue can undergo composite morphogenesis resulting from multiple concomitant shape changes. While previous studies have unraveled the mechanisms that drive simple morphogenetic processes, how a tissue can undergo multiple and simultaneous changes in shape is still not known and not much explored. In this chapter, we focus on the process of concomitant tissue folding and extension that is vital for the animal since it is key for embryo gastrulation and neurulation. Recent pioneering studies focus on this problem highlighting the roles of different spatially coordinated cell mechanisms or of the synergy between different patterns of gene expression to drive composite morphogenesis.

Introduction

During embryo development, a cluster of cells can be shaped into a plethora of different living forms. Each form is designed to provide specific features and functionalities to each animal, thus contributing to the great diversity and beauty of the animal kingdom. While the number of living shapes is innumerable, 7 is the number of fundamental epithelial transformations that are required to mold the embryo into a living form. Tissues (i) grow, (ii) shrink, (iii) thicken, (iv) thin, (v) twist, (vi) converge-extend and (vii) fold to give shape to the mature animal. Epithelial convergence-extension and folding are two fundamental tissue shape changes that often occur at the onset of embryogenesis. While extension can, for instance, elongate the animal body separating the anterior (where the head and mouth are located) from the posterior (where the anus is located), folding can translocate cells into the interior separating the outside from the inside of the animal. Over the last century, these two morphogenetic processes have been intensively studied since they are primordial for the emergence of multicellular life. Folding and extension have often been studied separately as two distinct and uncoupled processes (i.e., two 'simple' morphogenetic processes) controlled by distinct gene expression patterns and signaling pathways known to specify animal body plans [START_REF] Gilmour | From morphogen to morphogenesis and back[END_REF]. Nevertheless, folding and extension often occur simultaneously (e.g., during gastrulation and neurulation) resulting in a 'composite' morphogenetic process [START_REF] Nishimura | Planar cell polarity links axes of spatial dynamics in neural-tube closure[END_REF]. How composite morphogenesis is controlled and driven is still a question that has not been thoroughly considered and that deserves great attention.

Before addressing the problem of composite morphogenesis, in the first part of this chapter we review the common origin of the gene expression patterns responsible for different body plan specification. We then present how distinct gene expression patterns, extending along different body axes, can be uncoupled from each other to independently control simple morphogenetic processes. Finally, we present the process of composite morphogenesis resulting from multiple and concomitant changes in the shape of a tissue.

More specifically, we focus on the process of concomitant tissue folding and extension that is key during embryo gastrulation and neurulation.. Embryonic axis specification is under the control of a common organizer Metazoans originate from a single cell, the zygote. The zygotic cell undergoes stereotypic divisions to give rise to the embryonic blastula. Cells eventually remodel their shape, divide, die, translocate and rearrange to give shape to the mature animal. During this process, the blastoderm cells are guided by instructive signals that control body axis orientation, morphogenesis and cell specification leading to the formation of the three fundamental germ layers. For instance, Gurken-mediated EGFR signaling is necessary for anterior-posterior (AP) and dorsal-ventral (DV) axis establishment during oogenesis in Drosophila [START_REF] Gavis | Pattern formation. Gurken meets torpedo for the first time[END_REF][START_REF] Ray | Intercellular signaling and the polarization of body axes during Drosophila oogenesis[END_REF]. Gurken (Grk) was firstly identified in 1993 as a DV axis generator with gurken mRNA accumulating anterior-dorsally at a position juxtaposed to the oocyte nucleus [START_REF] Neuman-Silberberg | The Drosophila dorsoventral patterning gene gurken produces a dorsally localized RNA and encodes a TGF alpha-like protein[END_REF] (Figure 1 A). Gurken is a secreted ligand signaling to follicle cells near the oocyte nucleus. Embryos derived from grk mutants were reported to exhibit ventralization [START_REF] Schupbach | Germ line and soma cooperate during oogenesis to establish the dorsoventral pattern of egg shell and embryo in Drosophila melanogaster[END_REF]. In 1995 Gonzalez-Reyes et al. and Roth et al. showed that Gurken/EGFR also signals to posterior follicle cells to establish the AP axis. Therefore, mutants for grk, egfr/torpedo (top) or cornichon (Cni, a protein involved in Gurken secretion [START_REF] Bokel | Drosophila Cornichon acts as cargo receptor for ER export of the TGFalphalike growth factor Gurken[END_REF][START_REF] Castro | Cornichon regulates transport and secretion of TGFalpha-related proteins in metazoan cells[END_REF]) show both DV and AP defects [START_REF] Gonzalez-Reyes | Polarization of both major body axes in Drosophila by gurken-torpedo signalling[END_REF][START_REF] Roth | cornichon and the EGF receptor signaling process are necessary for both anterior-posterior and dorsal-ventral pattern formation in Drosophila[END_REF]. bcd mRNA localizes to the anterior and osk mRNA to the posterior poles of the wild type oocyte. In grk mutants, bcd mRNA accumulates at both anterior and posterior poles and osk mRNA is mislocalized to the center of the oocyte [START_REF] Gonzalez-Reyes | Polarization of both major body axes in Drosophila by gurken-torpedo signalling[END_REF]. Therefore, EGFR signaling between the developing oocyte and somatic follicle cells acts as a common organizer for both AP and DV gene patterning in Drosophila.

The planarian worm is a powerful model system for studying axes specification due to its extensive ability to regenerate: a small portion excised from any part of the worm body can regenerate to form a new and fully functional worm [START_REF] Ivankovic | Model systems for regeneration: planarians[END_REF]. In planarians the midline blastema cells function as an organizer for both AP and DV gene patterning [START_REF] Sureda-Gomez | Planarian organizers[END_REF]. The existence of an organizer, defining the animal body axes, has also been reported during vertebrate embryogenesis [START_REF] Martinez Arias | On the nature and function of organizers[END_REF]. It was dubbed Spemann's organizer in salamander [START_REF] Spemann | Induction of embryonic primordia by implantation of organizers from a different species[END_REF] and Xenopus [START_REF] Cho | Molecular nature of Spemann's organizer: the role of the Xenopus homeobox gene goosecoid[END_REF][START_REF] Stewart | The anterior extent of dorsal development of the Xenopus embryonic axis depends on the quantity of organizer in the late blastula[END_REF][START_REF] De Robertis | Spemann's organizer and self-regulation in amphibian embryos[END_REF], Hensen's node in chick [START_REF] Boettger | The avian organizer[END_REF][START_REF] Waddington | Induction by heteroplastic grafts of the primitive streak in birds[END_REF] and the 'embryonic shield' in zebrafish [START_REF] Feldman | Zebrafish organizer development and germ-layer formation require nodal-related signals[END_REF][START_REF] Schier | The zebrafish organizer[END_REF][START_REF] Shih | Characterizing the zebrafish organizer: microsurgical analysis at the early-shield stage[END_REF][START_REF] Zhang | Positional cloning identifies zebrafish one-eyed pinhead as a permissive EGF-related ligand required during gastrulation[END_REF][START_REF] Saude | Axis-inducing activities and cell fates of the zebrafish organizer[END_REF]. By implementing tissue grafting techniques, Spemann and Mangold discovered that a piece of dorsal lip from a donor salamander embryo generated a complete secondary axis when grafted to the recipient embryo (seemingly similar to Siamese twins, Figure 1 B) [START_REF] Spemann | Induction of embryonic primordia by implantation of organizers from a different species[END_REF].

Spemann's organizer potency for axes specification depends on the number of organizer cells [START_REF] Stewart | The anterior extent of dorsal development of the Xenopus embryonic axis depends on the quantity of organizer in the late blastula[END_REF]. The molecular identity of Spemann's organizer was later identified. One of the molecular factors participating to this organizer is, for instance, the homeobox-family protein Goosecoid (Gsc). Microinjection of gsc mRNA at the ventral region is sufficient to induce twinned axes [START_REF] Cho | Molecular nature of Spemann's organizer: the role of the Xenopus homeobox gene goosecoid[END_REF]. Furthermore, the organizer secretes a set of BMP and Wnt antagonists (e.g., Noggin, Chordin, and Frzb under the control of Nodal and β-Catenin) to counteract the ventrally produced BMP and Wnt ligands [START_REF] De Robertis | Dorsal-ventral patterning and neural induction in Xenopus embryos[END_REF][START_REF] De Robertis | Molecular mechanisms of cell-cell signaling by the Spemann-Mangold organizer[END_REF]. Recently, an organizer with a molecular signature similar to Spemann's organizer was identified in echinoderms. Ectopic Nodal expression was sufficient to induce Siamese twin formation in the sea urchin pluteus [START_REF] Lapraz | A deuterostome origin of the Spemann organiser suggested by Nodal and ADMPs functions in Echinoderms[END_REF].

Similarly, cells showing organizer activity and that provide positional cues to the other cells of the embryo were also identified during organ formation. Examples have been shown in the notochord during nervous system development [START_REF] Stemple | Structure and function of the notochord: an essential organ for chordate development[END_REF], in the zone of polarizing activity (ZPA) of the limb bud [START_REF] Mcqueen | Establishing the pattern of the vertebrate limb[END_REF], in the zonula limitans intrathalamica (ZLI) of the vertebrate diencephalon [START_REF] Kiecker | Hedgehog signaling from the ZLI regulates diencephalic regional identity[END_REF][START_REF] Scholpp | Hedgehog signalling from the zona limitans intrathalamica orchestrates patterning of the zebrafish diencephalon[END_REF], and in the mid-hindbrain boundary [START_REF] Rhinn | The midbrain--hindbrain boundary organizer[END_REF]. Finally, while in insects the oocyte secretes molecules that function as organizers to determine body axes, in vertebrates the secretion of organizer molecules is performed by a specific group of cells in the embryo.

Another tier of regulation of gene expression happens during the initial phase of embryo development. At the onset of embryogenesis, when the zygotic genome is quiescent, the early developmental processes rely on the protein and RNA pool supplied by the mother. It is only during the maternal-to-zygotic transition that the zygotic genome is activated by pioneer transcription factors that boost transcription [START_REF] Tadros | The maternal-to-zygotic transition: a play in two acts[END_REF][START_REF] Schulz | Mechanisms regulating zygotic genome activation[END_REF]. A wellcharacterized pioneer factor in Drosophila is Zelda [START_REF] Rushlow | Temporal dynamics, spatial range, and transcriptional interpretation of the Dorsal morphogen gradient[END_REF][START_REF] Liang | The zinc-finger protein Zelda is a key activator of the early zygotic genome in Drosophila[END_REF][START_REF] Schulz | Zelda is differentially required for chromatin accessibility, transcription factor binding, and gene expression in the early Drosophila embryo[END_REF]. Zelda binds to DNA sequences, enhancing gene expression [START_REF] Foo | Zelda potentiates morphogen activity by increasing chromatin accessibility[END_REF][START_REF] Harrison | Zelda binding in the early Drosophila melanogaster embryo marks regions subsequently activated at the maternal-to-zygotic transition[END_REF][START_REF] Yamada | The Drosophila Pioneer Factor Zelda Modulates the Nuclear Microenvironment of a Dorsal Target Enhancer to Potentiate Transcriptional Output[END_REF]. Recent work from McDaniel and colleagues, using optogenetic tools to block Zelda activity, has shown that Zelda is required for zygotic genome activation. Blocking Zelda activity reduces the expression of gene components acting along both the AP (e.g., eve, ftz, odd, etc.) and the DV (e.g., sog, zen, sna, twi, etc.) body axes [START_REF] Mcdaniel | Continued Activity of the Pioneer Factor Zelda Is Required to Drive Zygotic Genome Activation[END_REF]. Transcription factors with pioneering activity have also been identified in other model systems, for instance, Oct4 in mouse [START_REF] Foygel | A novel and critical role for Oct4 as a regulator of the maternal-embryonic transition[END_REF] and Pou5f1, Nanog, and Sox1 in zebrafish [START_REF] Lee | Nanog, Pou5f1 and SoxB1 activate zygotic gene expression during the maternal-to-zygotic transition[END_REF][START_REF] Leichsenring | Pou5f1 Transcription Factor Controls Zygotic Gene Activation In Vertebrates[END_REF].

Finally, a centralized signaling hub, working as a positional organizer, plays a pivotal role to establish gene patterning and cell specification along the different embryonic axes.

Gene expression patterns along different axes can be uncoupled from each other acting autonomously

While different gene expression patterns that control distinct body axes specifications originate from the same signal (provided by the organizer or from pioneer factors), the downstream signaling pathways that specify individual axes can act independently.

Mutations that result in loss or gain of function in downstream components produce specific patterning defects, generating phenotypes along one body plan without altering cell specification and patterning along another body plan. One classic example is the signaling pathway responsible for the specification of body plans in Drosophila [START_REF] St Johnston | The origin of pattern and polarity in the Drosophila embryo[END_REF]. The AP body plan specification in the fruit fly embryo is determined by the AP spatially regulated expression of gap genes (e.g.,huckebein,tailless,giant,hunchback,Kruppel,) under the control of upstream morphogens (e.g., Bicoid at anterior and Nanos at posterior sides [START_REF] Driever | The bicoid protein determines position in the Drosophila embryo in a concentration-dependent manner[END_REF][START_REF] Driever | A gradient of bicoid protein in Drosophila embryos[END_REF][START_REF] Lehmann | The maternal gene nanos has a central role in posterior pattern formation of the Drosophila embryo[END_REF][START_REF] Nusslein-Volhard | Determination of anteroposterior polarity in Drosophila[END_REF][START_REF] Struhl | The gradient morphogen bicoid is a concentrationdependent transcriptional activator[END_REF][START_REF] Wang | Nanos is the localized posterior determinant in Drosophila[END_REF]). DV body axis specification occurs at the same time and originates from the nuclear localization gradient of the transcription factor Dorsal that controls the transcription of twist and snail in the ventral and of zen and dpp in the dorsal region of the embryo [START_REF] Roth | A gradient of nuclear localization of the dorsal protein determines dorsoventral pattern in the Drosophila embryo[END_REF][START_REF] Morisato | Signaling pathways that establish the dorsal-ventral pattern of the Drosophila embryo[END_REF][START_REF] Leptin | Cell shape changes during gastrulation in Drosophila[END_REF][START_REF] Rushlow | Maternal regulation of zerknullt: a homoeobox gene controlling differentiation of dorsal tissues in Drosophila[END_REF][START_REF] Thisse | The twist gene: isolation of a Drosophila zygotic gene necessary for the establishment of dorsoventral pattern[END_REF].

Interestingly, the founding studies (identifying maternal and zygotic mutations that alter gene expression patterns along different body axes in the fly embryo) report mutations that affect either AP or DV features of the embryo and not both simultaneously [START_REF] Nusslein-Volhard | Determination of anteroposterior polarity in Drosophila[END_REF][START_REF] Leptin | Twist and Snail as Positive and Negative Regulators during Drosophila Mesoderm Development[END_REF][START_REF] Irvine | Cell intercalation during Drosophila germband extension and its regulation by pair-rule segmentation genes[END_REF][START_REF] Roth | Mechanisms of dorsal-ventral axis determination in Drosophila embryos revealed by cytoplasmic transplantations[END_REF][START_REF] Huang | An anteroposterior dorsal gradient in the Drosophila embryo[END_REF][START_REF] Anderson | Establishment of dorsal-ventral polarity in the Drosophila embryo: genetic studies on the role of the Toll gene product[END_REF][START_REF] Schupbach | Germline autonomy of maternal-effect mutations altering the embryonic body pattern of Drosophila[END_REF][START_REF] Schupbach | Female sterile mutations on the second chromosome of Drosophila melanogaster. I. Maternal effect mutations[END_REF]. This suggests that AP and DV pathways are uncoupled and act independently. This idea is also supported by the evidence that transplanting anterior plasm from wild type embryos to the anterior pole of a bicoid mutant embryo is sufficient to rescue AP patterning defects [START_REF] Frohnhofer | Manipulating the anteroposterior pattern of the Drosophila embryo[END_REF].

Similarly, it is possible to rescue a posterior gene mutated embryo by transplanting posterior pole plasm from a wild type embryo [START_REF] Lehmann | The maternal gene nanos has a central role in posterior pattern formation of the Drosophila embryo[END_REF][START_REF] Lehmann | Abdominal segmentation, pole cell formation, and embryonic polarity require the localized activity of oskar, a maternal gene in Drosophila[END_REF]. Siegfried Roth further tested the relationship between orthogonal gene expression patterns in 1993. Toll (a transmembrane receptor activated by the ventrally produced ligand Spatzle) induces the nuclear localization of Dorsal, thereby initiating DV axis specification. A Toll -/-embryo shows no nuclear localization of Dorsal and consequently no ventral fate specification [START_REF] Moussian | Dorsoventral axis formation in the Drosophila embryo--shaping and transducing a morphogen gradient[END_REF]. Interestingly, the seven Evenskipped (Eve) stripes formed along the AP axis in a wild type embryo [START_REF] Frasch | Characterization and localization of the even-skipped protein of Drosophila[END_REF] are unaltered in the Toll -/-embryo [START_REF] Hu | Multimerization and interaction of Toll and Spatzle in Drosophila[END_REF]. Transplantation of wild type plasm induces the expression of twi near the injection site. Injection of wild type cytoplasm at the posterior end of a Toll -/-embryo results in the posterior expression of twi. Remarkably, the posterior expression of twi does not alter any of the seven Eve stripes (Figure 2 A). This further supports the idea that, at this signaling level, the AP and DV signaling pathways are apparently uncoupled in the Drosophila embryo [START_REF] Roth | Mechanisms of dorsal-ventral axis determination in Drosophila embryos revealed by cytoplasmic transplantations[END_REF].

The planarian worm exhibits uncoupling between pathways that control the specification of different body axes. The Wnt/β-Catenin and BMP pathways have been shown to control AP and DV axis specification, respectively [START_REF] Gurley | Beta-catenin defines head versus tail identity during planarian regeneration and homeostasis[END_REF][START_REF] Molina | The BMP pathway is essential for re-specification and maintenance of the dorsoventral axis in regenerating and intact planarians[END_REF][START_REF] Molina | Organizing the DV axis during planarian regeneration[END_REF][START_REF] Petersen | Smed-betacatenin-1 is required for anteroposterior blastema polarity in planarian regeneration[END_REF][START_REF] Yazawa | Planarian Hedgehog/Patched establishes anterior-posterior polarity by regulating Wnt signaling[END_REF][START_REF] Iglesias | Silencing of Smed-betacatenin1 generates radial-like hypercephalized planarians[END_REF]. Downregulation of βcatenin results in hypercephalized flatworms and unaltered expression of dorsal and ventral specification genes (e.g., septin and eye53, respectively) [START_REF] Iglesias | Silencing of Smed-betacatenin1 generates radial-like hypercephalized planarians[END_REF]. In a similar experiment, downregulation of admp or noggin results in a dorsalized planarian (shown by the expression of septin on the ventral side). These experiments, using RNA interference, result in regeneration defects along the DV axis, whereas structures along the AP axis remain unaffected and regenerate normally [START_REF] Gavino | A Bmp/Admp regulatory circuit controls maintenance and regeneration of dorsal-ventral polarity in planarians[END_REF][START_REF] Molina | Noggin and noggin-like genes control dorsoventral axis regeneration in planarians[END_REF] (Figure 2 B). This provides further evidence for the uncoupling of AP and DV patterning and signaling in planarians.

A certain level of uncoupling between pathways that regulate AP and DV body plans is also shown in vertebrate embryos. Wnt, FGF, and retinoic acid (RA) signaling mediate vertebrate AP [START_REF] Gamse | Vertebrate anteroposterior patterning: the Xenopus neurectoderm as a paradigm[END_REF][START_REF] Yamaguchi | Heads or tails: Wnts and anterior-posterior patterning[END_REF][START_REF] Hikasa | Wnt signaling in vertebrate axis specification[END_REF] while BMP signaling mediates DV [START_REF] Langdon | Maternal and zygotic control of zebrafish dorsoventral axial patterning[END_REF] axis specification. Downregulation of Wnt signaling in the Xenopus laevis embryo, using morpholinos against β-cat, produces embryo ventralization, while treatment with Lithium Chloride (LiCl) induces embryo dorsalization (LiCl treatment was shown to induce radial expression of BMP antagonists such as Chordin [START_REF] Oelgeschlager | The pro-BMP activity of Twisted gastrulation is independent of BMP binding[END_REF]). Conversely, blocking BMP signaling by triple knockdown of bmp2, bmp4 and bmp7 results in a tail-less embryo, whereas blocking BMP and Wnt signaling together generates a head-like embryo [START_REF] Reversade | Depletion of Bmp2, Bmp4, Bmp7 and Spemann organizer signals induces massive brain formation in Xenopus embryos[END_REF]. Similarly, Varga et al. showed that accurate AP patterning of the neuroectoderm is achieved in an ichabod (ich) mutant zebrafish embryo with bmp2 morpholino that lacks DV polarity [START_REF] Varga | Correct anteroposterior patterning of the zebrafish neurectoderm in the absence of the early dorsal organizer[END_REF]. Uncoupling between gene-patterning along different body axes is reported during vertebrate limb morphogenesis.

The AP patterning defines the digit identity whereas the proximal-distal (PD) patterning defines the axis of the upper to lower arm and hand. The AP axis of vertebrate limbs is defined by Sonic hedgehog (Shh) signaling [START_REF] Tickle | Sonic Hedgehog Signaling in Limb Development[END_REF] while the PD axis is defined by FGF and RA signaling [START_REF] Delgado | Coordination of limb development by crosstalk among axial patterning pathways[END_REF]. Chick embryos mutant for Shh signaling (as in the case of the oligozeugodactyly mutant) or with reduced Shh activity (by applying pharmacological inhibitors like cyclopamine) show patterning defects along the AP axis while PD pattern formation remains unaltered [START_REF] Towers | Integration of growth and specification in chick wing digit-patterning[END_REF].

Finally, examples from different model systems show that the patterns of gene expression, imposing cell fate specification and defining the different body plans of the animal while stemming from the same organizing center, can be uncoupled from each other to autonomously control specific morphogenetic processes along specific body axis. This may be the consequence of specialized signaling factors that activate ad hoc pathways devoid of cross-talk.

Concomitant morphogenesis results from multiple simultaneous changes in the shape of a tissue

The form of an organism is sculpted by the cellular and epithelial changes in shape during embryo development. Cells actively change shape and position during embryogenesis under the control of specifying signaling pathways. This process relies on the gene patterning systems established along the different body plans providing positional coordinates of gene expression. Therefore, gene expression patterns instruct cells to undergo specific shape changes and movements driving morphogenesis.

The 7 simple and fundamental epithelial transformations can take place sequentially.

Nevertheless, this is not always the case. Multiple and concomitant simple morphogenetic processes can lead to the composite change in the shape of a tissue. While the mechanisms driving simple morphogenetic processes have been thoroughly studied, how composite morphogenetic processes are controlled and driven remained, until recently, unexplored.

The remainder of this chapter will focus on the process of concomitant folding and extension that is key during embryo gastrulation and neurulation. We will highlight two modes of concomitant folding-extension: (i) uniaxial and (ii) orthogonal both leading to the formation of an epithelial tube. In the former mode, folding and extension take place along the same axis synergistically driving tissue budding. This composite morphogenetic process may emerge from the coordination of multiple radially patterned cell mechanisms. In the latter mode, folding and extension take place along orthogonal axes, separately driving tissue wrapping and elongation. A recent pioneer study shows that this composite morphogenetic process is driven by a cell combo-mechanism under the synergistic control of orthogonal gene patterning signals.

Uniaxial tissue folding and extension

The process of simultaneous tissue folding and extension can occur along one same axis driving tissue in-pocketing (also referred to as tissue budding) and eventually the formation of an epithelial tube. Examples of this are, for instance, the processes of salivary gland [START_REF] Cheshire | Kinetic and mechanical analysis of live tube morphogenesis[END_REF][START_REF] Myat | Fork head prevents apoptosis and promotes cell shape change during formation of the Drosophila salivary glands[END_REF][START_REF] Myat | Organ shape in the Drosophila salivary gland is controlled by regulated, sequential internalization of the primordia[END_REF], trachea [START_REF] Brodu | The RhoGAP crossveinless-c links trachealess and EGFR signaling to cell shape remodeling in Drosophila tracheal invagination[END_REF][START_REF] Llimargas | EGF signalling regulates cell invagination as well as cell migration during formation of tracheal system in Drosophila[END_REF][START_REF] Nishimura | A wave of EGFR signaling determines cell alignment and intercalation in the Drosophila tracheal placode[END_REF], and intestine [START_REF] Iwaki | bowl, and lines are required for patterning and cell rearrangement in the Drosophila embryonic hindgut[END_REF] formation, or endoderm/mesoderm internalization in early embryogenesis during gastrulation [START_REF] Hardin | The Mechanisms and Mechanics of Archenteron Elongation during Sea-Urchin Gastrulation[END_REF][START_REF] Nakajima | The initial phase of gastrulation in sea urchins is accompanied by the formation of bottle cells[END_REF][START_REF] Keller | An experimental analysis of the role of bottle cells and the deep marginal zone in gastrulation of Xenopus laevis[END_REF][100]. Tissue in-pocketing results from the extension of a planar epithelium into the third dimension to form a fold. During epithelial inpocketing, tissue extension and folding synergistically cooperate to form a budding tube. As a consequence, the cell mechanisms known to drive tissue extension (e.g., polarized cell stretching, intercalation, cell division, migration) may cooperate with mechanisms known to drive tissue folding (e.g., apical constriction, apical-basal shortening, basal expansion). The planar cell polarized signals and mechanisms driving out-of-plane tissue extension are patterned differently from what is usually reported for the canonical planar cell polarity (PCP) patterns driving in-plane tissue extension [101,102]. For budding tissues, the signaling factors and the cell shape and topology changes follow a circular geometry centered on the tissue budding region and develop radially from this point, resulting in out-of-plane tissue extension [START_REF] Myat | Fork head prevents apoptosis and promotes cell shape change during formation of the Drosophila salivary glands[END_REF][START_REF] Brodu | The RhoGAP crossveinless-c links trachealess and EGFR signaling to cell shape remodeling in Drosophila tracheal invagination[END_REF][START_REF] Llimargas | EGF signalling regulates cell invagination as well as cell migration during formation of tracheal system in Drosophila[END_REF][START_REF] Nakajima | The initial phase of gastrulation in sea urchins is accompanied by the formation of bottle cells[END_REF][103][104][105]. Therefore, epithelial in-pocketing is a process that results from the combination of different spatio-temporally coordinated cell mechanisms that are radially distributed and polarized in the plane of the tissue. In the remaining part of this section, we present two examples of epithelial in-pocketing (i) during salivary gland formation in the developing Drosophila embryo and (ii) during archenteron formation in the sea urchin gastrula.

Salivary gland placode folding and extension

A key function of the Drosophila salivary glands (SGs) is to produce and eventually release the secretory glue needed to affix the de novo formed puparium, within which the larva will undergo metamorphosis, to a substrate. During SG formation, the SG placode (an epithelial tissue formed of around 100 cells) in-pockets, forming an epithelial tube [106]. Under determined mutated conditions, the epithelial tube can evaginate. This shows that the process of tube formation in the SG placode does not require contact with internal tissues [107]. Surprisingly, cells in evaginated tubes invert their apical-basal polarity, resulting in the apical side facing the inside of the tube devoid of luminal space. This supports the idea that the process of SG tube formation takes place in a tissue-autonomous fashion and can be decoupled from the process of tube internalization. Before tissue budding, an actomyosin cable forms around the SG placode. Katja Röper showed in 2012 that this cable is under tension and contributes to the in-pocketing of the placode. The actomyosin cable drives tissue centripetal convergence towards the budding locus, working as a contractile pursestring [108]. At the onset of tissue budding, cell shape and topology changes take place following a circular symmetry: while cells in the budding locus apically constrict, the crown of cells around the bud intercalate following a radial pattern (tangential junctions shrink while new junctions form along a radial direction) [104]. While apical constriction initiates tissue bending, extending the SG placode in the third dimension, radially polarized cell intercalation promotes centripetal tissue extension and circumferential tissue convergence (the latter reinforced by the actomyosin circumferentially contracting cable) (Figure 3). The distribution of molecular force generators mirrors the radial cell shape and topology changes. Cells located in the center accumulate medial-apical fiber actin (F-actin) and MyoII that generate contractile forces that constrict the apical surface area of cells (seemingly similar to what is reported during mesoderm invagination [109]). The transcription factor Fork head (Fkh) is necessary for SG tube formation [START_REF] Myat | Fork head prevents apoptosis and promotes cell shape change during formation of the Drosophila salivary glands[END_REF] and is required for medial-apical MyoII accumulation in apically constricting cells promoting Rho kinase (Rok) apical recruitment via Fog [107]. While cell apical constriction is necessary for the correct SG tube shape, surprisingly cell apical constriction is not necessary for tube formation per se in contrast to what is reported to occur for mesoderm fold formation [110]. In fog-embryos, in which apical constriction is strongly perturbed, the SG tube is intact, closed and correctly polarized with the cell apical side facing towards the inside of the tube. Nevertheless, fog-embryos show a great variety of tube shapes and topologies: tubes can form with normal shape, can twist or, in more than 1/3 of the cases, can evaginate and extend outside the embryonic body. These observations show that apical constriction is necessary to pre-sculpt the shape of the central budding region of the SG placode (resulting in a concave cup shape) to impose directionality during the tube extension phase providing robustness to the overall process [111]. Cells located in the crown region, surrounding the patch of cells that apically constrict, show a planar polarized distribution of MyoII with tangential junctions showing higher levels of MyoII than radial junctions. Cortical actomyosin has previously been shown to increase tension to initiate junction shortening [112].This is the first step of cell intercalation in simple epithelia [113]. MyoII junctional accumulation is controlled by the transmembrane protein Crumbs, which is planar cell polarized and which activates the Par6/Cdc42/Pak1 pathway, modulating the dissociation rate (koff) of Rok [108,114]. In fkh-embryos, in which apical constriction of the central SG placode is abolished, junction remodeling still takes place but is no longer polarized. Junctions with higher levels of MyoII (the motor protein is no longer planar cell polarized in an fkh-background) shrink to a 'four-way junction' but new junctions do not form, resulting in an incomplete non-polarized cell intercalation process. This evidence supports the idea that, while junction shrinkage is a cell-autonomous process driven by junctional actomyosin contraction, new junction formation and extension may be a passive response to the centripetal pulling force exerted by the patch of cells that apically constrict. This observation further supports the idea that the transcription factor Fkh controls MyoII radial PCP. In a circular geometry, PCP could be achieved, for instance, by a radially graded distribution of signaling factors (e.g., Fkh) from a source point outwards, following the same principle of uniaxial PCP that is controlled by a unidirectionally oriented gradient (e.g., the AP distribution of Bicoid in the early developing Drosophila embryo). Finally, tube formation and extension is powered by (i) cells in the central region of the SG placode that apically constrict to locally bend the tissue inward to bias the direction of bud extension and (ii) the actomyosin cable (surrounding the SG placode) accompanied by radially polarized intercalations of cells located in the crown region surrounding the central patch of apically constricting cells.

Vegetal plate folding and extension during sea urchin gastrulation

The sea urchin is historically among the first model systems used to study embryo gastrulation [115][116][117]. The sea urchin gastrula combines a number of outstanding features making this model system a unique opportunity to study the mechanisms and mechanics of tissue folding: [START_REF] Gilmour | From morphogen to morphogenesis and back[END_REF] this embryo is a very simple thus appealing system for experimentation and mathematical modelling since it is constituted of about 1000 cells forming a hollow spherical monolayer epithelium surrounded and filled with water; (2) all 1000 cells can be imaged and 3D segmented over time since the sea urchin gastrula is transparent; providing a plethora of sea urchin model systems that can be compared.

During sea urchin gastrulation, the vegetal plate in-pockets by folding inwards and eventually by extending to form a tubular epithelium (also known as archenteron) that constitutes the primordial gut of the future sea urchin pluteus. The formation of the archenteron is divided into two phases: (i) a first phase named 'primary invagination' that results in tissue bending (forming a concave cup shape) and (ii) a second phase named 'secondary invagination', during which the folded vegetal plate extends towards the interior of the embryo crossing the inner hollow space (the blastocoel, Figure 4). In Lytechinus pictus, if the vegetal pole is isolated microsurgically from the rest of the blastoderm, the vegetal plate still folds and partially extends, supporting the idea that the forces responsible for primary invagination are generated by the cells located in the vegetal 1/3 of the blastoderm [123].

During primary invagination, a ring of cells (the prospective secondary mesenchymal cells) apically constricts around a group of quiescent cells (the small micromeres) located in the center of the vegetal plate. The process of cell apical constriction is accompanied by a reorganization of the F-actin network that becomes denser in these cells. Embryos treated with a calcium channel activator or blocker show initiation or inhibition, respectively, of apical constriction and primary invagination [START_REF] Nakajima | The initial phase of gastrulation in sea urchins is accompanied by the formation of bottle cells[END_REF]. If apically constricting cells are removed by laser ablation, the primary invagination of the vegetal plate is delayed suggesting that these cells play a role in the folding process [124].

Filopodia are formed at the basal side of cells located at the vegetal plate. Protrusion activity begins during primary invagination and it is reinforced during secondary invagination when cells extend longer and more numerous protrusions [125]. During archenteron elongation, filopodia projected from the basal side of cells located at the vegetal plate, reach the basal side of cells located at the animal pole (on the opposite side) eventually exerting traction forces. Jeff Hardin in 1988 probed the role of cell filopodia by performing filopodia laser ablation [126]. These experiments show that traction forces exerted via filopodia contribute to archenteron elongation during secondary but not primary invagination. In a filopodium ablated condition, the archenteron extension is reduced by 30%. This supports the idea that other filopodium-independent processes still account for 70% of archenteron elongation. This idea is also supported by the fact that in both Lytechinus pictus and Strongylocentrotus purpuratus exogastrulae (i.e., embryos forming an archenteron that extends outwards) the archenteron still extends by 70% [START_REF] Hardin | The Mechanisms and Mechanics of Archenteron Elongation during Sea-Urchin Gastrulation[END_REF]115,123,126]. What other mechanism could thus drive archenteron elongation? Radial cell intercalation of vegetal plate cells has been suggested to be another mechanism driving archenteron extension during secondary invagination [START_REF] Hardin | The Mechanisms and Mechanics of Archenteron Elongation during Sea-Urchin Gastrulation[END_REF][127][128][129]. Cells, by exchanging neighbors in a radially polarized fashion, would reduce and increase the width and length of the archenteron, respectively. A study by Jeff Hardin and Michael Weliky suggests that filopodium traction forces could also contribute to cell rearrangement in the late phase of convergenceextension of the archenteron [128]. Other mechanisms have been proposed to play a role in vegetal plate folding and extension as, for instance, cell tractoring [130], spatio-temporally coordinated swelling of the extracellular matrix [START_REF] Nakajima | The initial phase of gastrulation in sea urchins is accompanied by the formation of bottle cells[END_REF]131] and modulation of blastocoel osmotic pressure [132]. Some of these mechanisms have been tested by implementing mathematical models [START_REF] Hardin | The Mechanisms and Mechanics of Archenteron Elongation during Sea-Urchin Gastrulation[END_REF][133][134][135].

Several signaling pathways driving vegetal plate folding and extension have been identified in Lytechinus variegatus and Paracentrotus lividus: (i) a non-canonical Wnt/PCP pathway transduced via the Frizzled5/8 receptor is required for primary invagination [103].

The Wnt/PCP small RhoGTPases, RhoA and Cdc42, have also been shown to be key players during primary invagination since RhoA downregulation inhibits primary invagination while RhoA overexpression induces precocious invagination [136]. (ii) The canonical Wnt/βcatenin pathway via the Frizzled1/2/7 receptor is necessary for secondary invagination [137-140] and (iii) FGF signaling is necessary to drive the invagination process in a timely fashion [141]. A renewed effort is now needed to unravel how the different signaling pathways control the mechanisms and mechanics responsible for cell shape and topology changes that drive folding and extension and result in vegetal plate in-pocketing.

Concomitant tissue folding and extension along orthogonal axes

Simultaneous tissue folding and extension can take place along orthogonal axes. For example, during neurulation in tetrapods, the anterior-dorsal tissue folds to form the neural tube while extending (together with the rest of the animal body) to separate the head from the anus [142-145].. Another example has been recently highlighted in the study by John and Rauzi showing that the prospective mesoderm undergoes simultaneous and orthogonal tissue folding and extension on the ventral region of the Drosophila embryo during gastrulation [102]. Since folding and extension in these processes are orthogonal, thus a priori independent, cells may need to perform multiple independent morphological and topological transformations simultaneously.

How does composite orthogonal morphogenesis take place? One hypothesis could be that simultaneous and orthogonal morphogenetic processes could result from the simple addition of independent signaling pathways or cell mechanisms driving different cell shape and topology changes. Specific changes in cell shape and topology are driven by specialized mechanisms finely tuning the cell cytoskeleton that is eventually remodeled to generate mechanical forces at the cell cortex. Even if upstream signaling pathways were instructive for distinct morphogenetic processes, downstream (at the cytoskeleton level) all pathways are intended to cross-talk or synergize since the nature of the forces and of the force generating factors is common (Figure 8). Therefore, the naïve hypothesis that composite orthogonal morphogenesis could result from the simple addition of independent signaling pathways driving different cellular mechanisms is to be excluded. How multiple specialized cell mechanisms coexist and act simultaneously in a coordinated fashion to remodel the same cortical cytoskeletal substrate is not clear. Signals, which are under the control of distinct gene expression patterns, may synergize at different levels of the signaling cascade (Figure 8 -gray side bars) resulting in the emergence of novel combined mechanisms driving composite morphogenesis along orthogonal directions.

In the following, we review recent pioneering advances in the study of primary neurulation and mesoderm invagination highlighting our current understanding of how composite and orthogonal morphogenetic processes are controlled and driven.

Neural plate orthogonal folding and extension during primary neurulation

The formation of the neural tube is thought to rely on both extrinsic and intrinsic forces [146].

Removal of the stratified tissue (composed of the non-neural ectoderm, the mesoderm and the endoderm stacked on top of each other), located laterally to the neural-ectoderm, inhibits The neural plate, during folding, converges towards the dorsal midline and extends along the AP axis of the embryo. By using distortion diagrams, Elul and colleagues in 1997 concluded that the part of the neural ectoderm closer to the dorsal midline undergoes greater convergence-extension than the more laterally located part [156]. In addition, neuralectoderm explants, taken from regions closer to the dorsal midline, undergo enhanced convergence-extension. This suggests that the medial portion of the neural tissue may contribute more to convergent extension than the lateral portion that would offer resistance.

A dynamic analysis at the cellular scale shows that cells located in the midline undergo planar polarized cell intercalation converging medio-laterally and separating anteroposteriorly [157]. Intercalation can take place via junction remodeling (as shown in the mouse and chick, [START_REF] Nishimura | Planar cell polarity links axes of spatial dynamics in neural-tube closure[END_REF]158]) or via polarized protrusive activity (for instance, as shown in the Xenopus [143,159,160]).

Changes in cell shape and topology are accompanied by cytoskeletal remodeling.

For example, actomyosin accumulates along junctions and in the medio-apical cortex of neuroepithelial cells [161][162][163][164]. In the Xenopus laevis neural epithelium, a reduction in apical intermediate filaments and an increase in basal-to-apical microtubule arrays is also reported [165]. During neural plate folding, nuclei migrate from a more apical to a more basal position.

This process was first named 'interkinetic nuclear migration ' in 1978 [166] and was reported in avian [167] and amphibian [168] neurulation and in other epithelial folding processes (e.g., in Drosophila mesoderm invagination [START_REF] Leptin | Cell shape changes during gastrulation in Drosophila[END_REF]). While the actomyosin network is known to be a key player in the generation of mechanical forces, little is known on the role of other cytoskeletal components and on the movement of intracellular organelles (e.g., the nucleus) in neural tube formation.

A great number of mutations, affecting the regulators of the actomyosin cytoskeleton, result in neural tube defects. Key actomyosin regulators are cofilin [169,170], Rac1 [171],

RhoA/ROCK [START_REF] Nishimura | Planar cell polarity links axes of spatial dynamics in neural-tube closure[END_REF]172], ENA/WAP [173] and Shroom [174][175][176][177]. In addition, because neural plate convergence-extension relies upon planar cell polarized mechanisms, the inhibition of pathways controlling planar and apical-basal cell polarity impairs neurulation [178][179][180].

Nishimura and colleagues showed in 2012 that polarized short-range MyoII cables form medio-laterally at cellular junctions in the neural plate [START_REF] Nishimura | Planar cell polarity links axes of spatial dynamics in neural-tube closure[END_REF] (Figure 5). Since the neural plate folds anisotropically (along the medial-lateral but not the AP axis), Nishimura and colleagues proposed a model in which MyoII cables work both to initiate cell intercalation (seemingly similar to [112]) and to promote anisotropic neural plate bending. Nevertheless, the idea that cell polarized junctional cables are necessary for anisotropic folding is misleading because folding anisotropy can emerge simply from the anisotropic distribution of apically constricting cells [181]. In addition, if MyoII cables trigger cell neighbor exchange, the process of cell intercalation would reduce medial-apical tension, thus disfavoring anisotropic tissue bending.

How polarized junctional actomyosin networks could simultaneously drive cell intercalation and neural plate apical constriction and thus trigger bending of the neural plate is thus not clear. Cell intercalation, driven by actomyosin planar cell polarity, is more likely to play a role in the closure of the tube rather than triggering its bending. Polarized cell neighbor exchange results in tube narrowing, which brings the neural plate lips closer to one another. This is in agreement with siRNA-mediated depletion of RhoGEF or upstream signaling factors (e.g., siRNA-mediated depletion of Celsr1) resulting in a neural plate that can bend and fold but that fails to close [START_REF] Nishimura | Planar cell polarity links axes of spatial dynamics in neural-tube closure[END_REF]. Finally, while important advances have been made in the recent years to unravel the signaling pathways and some of the cell mechanisms playing a role in primary neurulation, further work is necessary to understand how these govern the mechanics of neural tube formation.

Orthogonal folding and extension of the prospective mesoderm during Drosophila embryo gastrulation

The ventral tissue of the developing Drosophila embryo constitutes the future mesoderm of the larva and is known to fold during early gastrulation [START_REF] Leptin | Cell shape changes during gastrulation in Drosophila[END_REF]182]. Tissue folding is under the control of the Twist and Snail transcription factors that are part of the embryo DV gene patterning [START_REF] Leptin | Cell shape changes during gastrulation in Drosophila[END_REF]. Apical surface area reduction of prospective mesoderm cells is a key mechanism that initiates tissue furrowing, eventually leading to epithelial folding. Cell apical backed up by cell lateral contractility [102,187], to drive tissue buckling. While apical constriction is thought to be necessary to drive tissue furrowing, it is still not clear if it is sufficient per se. Finally, embryo-scale accommodation of neighboring lateral tissues is necessary to allow the involution of the prospective mesoderm [188].

A recent study shows that the mesoderm not only furrows (and eventually folds forming an epithelial tube) but also extends along the AP axis (i.e., along a direction orthogonal to folding ̶ Figure 6 A and B) [102]. How are concomitant orthogonal folding and extension achieved? Cells in the prospective mesoderm undergo planar cell polarized intercalation while apically constricting. Seemingly similar to ectoderm cells [START_REF] Irvine | Cell intercalation during Drosophila germband extension and its regulation by pair-rule segmentation genes[END_REF], mesoderm cells also intercalate in such a way that AP neighbors separate while new DV neighbors come into contact. This finding, while apparently surprising, is consistent with the fact that both prospective lateral ectoderm and mesoderm express genes that control planar polarized cell intercalation and that both tissues are part of the germ band that is known to extend. Both apical constriction and cell intercalation are cell processes that result from specialized activities of the actomyosin cytoskeleton coupled to adherens junctions [109,189]. How can both cellular processes occur concomitantly in a cell? Cells located in the ventral region of the Drosophila embryo adopt a remarkable strategy never described before:

they establish a two-tier system of adherens junctions. While the first tier is located apically and mediates apical cell constriction, the second tier is located 10 µm from the cell apical surface and initiates cell intercalation [102]. The first junctional tier originates from sub-apical adherence junctions that are established in the Drosophila blastoderm during cellularization under the control of the apical-basal polarity Par3/Bazooka signaling pathway [190]. In mesoderm cells, sub-apical junctions are then relocalized apically [191] under the action of tethered actomyosin networks [192]. In this way, junctions that would otherwise be dismantled by the action of Snail signaling [192,193] extends. This shows that junction extension is a cell autonomous process in the prospective ectoderm [195] while in the mesoderm it results from polarized tissue-scale forces (similar to those reported for salivary gland formation). In the future it will be important to investigate (for instance by entirely isolating a group of ectoderm cells using laser based cauterization [196]) whether converging forces from more dorsally located tissues may also contribute to new junction extension in the ectoderm.

How is concomitant folding and extension controlled in the prospective mesoderm?

Mutations affecting the DV patterning (e.g., mutations for the twist and snail genes) result in a strong reduction of cell intercalation in the mesoderm (but not in the ectoderm where twist and snail are not expressed) and in a consequent inhibition of mesoderm extension. More specifically, in a snail mutant embryo, MyoII only localizes at subapical cell junctions. This

shows that Snail is necessary to establish the two-tier junctional system. AP gene patterning is key to controlling cell intercalation in the ectoderm [START_REF] Irvine | Cell intercalation during Drosophila germband extension and its regulation by pair-rule segmentation genes[END_REF]. Surprisingly, mutations affecting the AP patterning (e.g., a triple mutation for the bicoid, nanos and torso genes, bnt-) affect neither the number nor the polarity of cell intercalation in the mesoderm. Nevertheless, in bnt-embryos mesoderm extension is compromised. Therefore, the phenotype shown by bntembryos raises two paradoxes that underlie the following two key questions: (i) how can extension be impaired if intercalation density and polarity are not affected? And (ii) how can cell intercalation still be planar cell polarized if no instructive signal is present along the PCP axis? A closer look at the dynamics of cell intercalation in bnt-embryos sheds light on these apparent paradoxes. In bnt-embryos, the T1 transition occurs in an unusual fashion: the separating cell duo reduces in surface area while the new contacting cell duo expands in surface area, leading to a sudden exchange of cell neighbors. Given the peculiar cell conformation of the intercalating quartet, this mode of cell intercalation was dubbed bowtie intercalation (Figure 7). Because of the inhomogeneity in the surface area of the intercalating cell quartet, bowtie intercalation may result in less polarized displacement of cell center of masses (thus less tissue extension). In addition to the extension, the convergence of the mesoderm along the orthogonal axis is also compromised, resulting in a wider epithelial tube [102,197]. Remarkably in bnt-embryos, MyoII at the second tier is no longer planar cell polarized and forms ring patterns. This shows that AP patterning controls MyoII planar cell polarity at the second junctional tier to insure a regular T1 transition and net tissue extension. Signals emanating from the AP patterning are then not necessary to impose cell intercalation polarity which is instead oriented by tissue scale polarized forces. Schematic representation of a ventralized regenerated planarian worm after treatment interfering with BMP signaling using smad RNAi (bottom). Worms display patterning defects along the DV axis, while patterning along the AP axis remain unaltered. Dorsal markers are represented as green and ventral-specific cells as pink circles. Adapted from [START_REF] Molina | The BMP pathway is essential for re-specification and maintenance of the dorsoventral axis in regenerating and intact planarians[END_REF].

  (3) the signaling factors controlling sea urchin vegetal plate folding and extension are known and can be tuned to dissect their function; (4) the gastrula is a mechanically accessible tissue: it can be partitioned [118], cells can be transplanted [119], and micro-indentation and micropipetting techniques [120-122] can be applied to measure tissue mechanical properties. The sea urchin gastrula is thus a perfect playground for cell and developmental biologists and biophysicists. Sea urchins from different species and families are studied (e.g., Lytechinus pictus in the eastern Pacific Ocean, Lytechninus variegatus in the Caribbean Sea and in the western Atlantic Ocean, Paracentrotus lividus in the Mediterranean Sea and in the eastern Atlantic Ocean, Stronglyocentrotus purpuratus in the eastern Pacific Ocean, Stronglyocentrotus drobachiensis in the northern Pacific and Atlantic Oceans, Hemicentrotus pulcherrimus in the western Pacific Ocean, Arbacia punctulata in the Atlantic Ocean)

  surface reduction has been shown to be driven by a constricting force generated by a cell medial-apical actomyosin network coupled to adherens junctions [109]. Cellular networks are coupled with another, forming a supracellular cytoskeletal network that drives tissue-scale tension [183] and inward-directed hydrodynamic cytoplasmic flow [184]. Upregulation of cell basal MyoII blocks furrowing and eventual folding [185]. This supports the idea that cell apical and basal tension could work together to generate an active bending moment [186],

  are rescued and reinforced to withstand tissue scale tension[183]. The second junctional tier is then established 'on the fly' during tissue furrowing: lateral actomyosin contractions, under the control of the RhoGEF2 pathway, cluster E-cadherin proteins that engage with the actomyosin network via alpha-and beta-catenins to form spot-adherence junctions (Figure6C, inset). These lateral junctions, located 10 µm from the apical cell surface, are necessary to establish cell-cell contacts (a prerequisite to initiate junction remodeling and cell intercalation[113]). Similar to the process of cell intercalation in the ectoderm[189, 194], mesoderm cells show a planar cell polarized distribution of the motor protein MyoII which generates cortical tension that shortens the junction and initiates AP neighboring cell separation. Therefore, the T1 transition, though initiated apically in ectoderm cells, is initiated 10 µm inside the tissue in mesoderm cells (Figure6 C). Cell neighbor exchange is a topological cell transformation that proceeds in two steps: (i) a junction shortens between two neighboring cells and (ii) a new junction extends bringing two new neighboring cells into contact. While the mechanism of junction shortening is similar in mesoderm and ectoderm cells, the process of new junction extension is quite different. In ectoderm cells, the pulsatile medial-apical actomyosin network plays a key role in junction extension[195]. In mesoderm cells (in absence of a medial network at 10 µm inside the tissue) a new junction extends as a result of AP tissue-scale forces that are generated by the mesoderm apical supracellular network. When AP tension is downregulated in the mesoderm (by surgically ablating the apical supracellular network using an infra-red femtosecond laser), junctions enriched in MyoII shrink but no new junction

Finally,

  AP and DV gene patterning work synergistically during Drosophila gastrulation, providing signals that can cross-talk to control the emergence of combined cellular mechanisms which superposition drives and orients composite orthogonal morphogenesis. Outlook Functional living shapes are built during morphogenesis. Uncovering the mechanisms driving morphogenesis is thus understanding how multicellular life emerges. Dissecting embryo morphogenesis by studying simple morphogenetic processes is pivotal to understand the fundamental cell mechanisms driving tissue shape changes. Nevertheless, during body and organ formation of an animal, tissues undergo concomitant changes in shape resulting in composite morphogenetic processes that cannot be always reduced to the simple addition of signaling pathways and cell mechanisms responsible for simple tissue shape changes. Multiple and distinct cell mechanisms need to be spatio-temporally coordinated within different subgroups of cells or even within each cell eventually under the synergistic action of multiple gene patterning signals. A new avenue is now open to unravel the mechanisms driving composite morphogenesis and to uncover their origins.

FIGURE LEGEND Figure 1 :

 LEGEND1 FIGURE LEGEND

Figure 2 :

 2 Figure 2: Gene expression patterns are apparently uncoupled (A) Representative image adapted from[START_REF] Roth | Mechanisms of dorsal-ventral axis determination in Drosophila embryos revealed by cytoplasmic transplantations[END_REF] showing the seven Eve stripes (arrows) unaffected by transplantation of ventral plasm to posterior in Toll -/-embryos. Note that ventral plasm injection results in Twist expression (arrow heads) in the posterior cells. (B) Schematic representation of a ventralized regenerated planarian worm after treatment interfering with BMP signaling using smad RNAi (bottom). Worms display patterning defects along the DV axis, while patterning along the AP axis remain unaltered. Dorsal markers are represented as green and ventral-specific cells as pink circles. Adapted from[START_REF] Molina | The BMP pathway is essential for re-specification and maintenance of the dorsoventral axis in regenerating and intact planarians[END_REF].

Figure 3 :

 3 Figure 3: Tissue in-pocketing during salivary gland formation Schematic representation of the process of salivary gland placode folding and extension. Adapted from [104].

Figure 4 :

 4 Figure 4: Tissue in-pocketing during sea urchin embryo gastrulation Time lapse images of vegetal plate folding and archenteron elongation during sea urchin embryo gastrulation.

Figure 5 :

 5 Figure 5: Gene patterning synergy diagram Diagram showing the signaling pathways along DV and AP axes. Gray shaded side bars indicate possible synergy between DV and AP signaling patterns. Gray double-headed arrow indicates possible upstream synergy as presented in[198] and, most remarkably, in long shaped insects (e.g., honeybee embryos)[199]. Upstream synergy can also be mediated by coordinating factors[200, 201]. While upstream synergy involves ad hoc signaling factors, downstream synergy results from DV and AP signaling pathways merging towards common cellular factors/components and ultimately to physical forces of the same nature.

Figure 6 :

 6 Figure 6: Neurulation in the chick embryo (A) Schematic representation of the process of neurulation. (B) MyoII distribution in the neural plate during neuroepithelium folding and extension. Anterior (A), posterior (P), medial (M) and lateral (L). Adapted from [https://en.wikipedia.org/wiki/Neural_plate] and from [2].

Figure 7 :

 7 Figure 7: A two-tier junctional mechanism drives simultaneous folding and extension

Figure 8 :

 8 Figure 8: Bowtie intercalation in bnt-embryos (A) Cell intercalation in the prospective mesoderm in a wild type embryo. (B) Cell intercalation in the prospective mesoderm in a bnt-embryo. Adapted from [102].

  Figure1

Acknowledgments

We thank T. Lepage and S. Roth for critical reading of the manuscript. We thank B. Delorme, I. Bekri and R. Costanzo from the Rauzi lab for fruitful discussions and for sea urchin embryo images. AJ is supported by the scholarship Région SUD PACA -Enterprise (Bruker Luxendo) -UCA. This work was supported by the French government through the UCA JEDI Investments for the Future project managed by the National Research Agency (ANR-15-IDEX-01), the Investments for the Future LABEX SIGNALIFE (ANR-11-LABX-0028-01), the Tramplin-ERC program of the National Research Agency (ANR-16-TERC-0018-01), the ATIP-Avenir program of the CNRS and the Human Frontier Science Program (CDA00027/2017-C), the Region SUD PACA AAP RECHERCHE and ERC BOOSTER.


Jesus Bujalance
email: jesus.bujalance_martin@mines-paristech.fr

Fabien Moutarde

Reward Relabelling for combined Reinforcement and Imitation Learning on sparse-reward tasks Extended Abstract

Keywords: Reinforcement Learning, Imitation Learning, Robotic Manipulation

In the search for more sample-efficient reinforcement-learning (RL) algorithms, a promising direction is to leverage as much external off-policy data as possible. For instance, expert demonstrations. In the past, multiple ideas have been proposed to make good use of the demonstrations added to the replay buffer, such as pre-training on demonstrations only or minimizing additional cost functions. We present a new method, able to leverage both demonstrations and episodes collected online in any sparse-reward environment with any off-policy algorithm. Our method is based on a reward bonus given to demonstrations and successful episodes (via relabeling), encouraging expert imitation and self-imitation. Our experiments focus on several robotic-manipulation tasks across two different simulation environments. We show that our method based on reward relabeling improves the performance of the base algorithm (SAC) on these tasks.

INTRODUCTION

In this work we focus on off-policy RL, and present a way to leverage offline data in the form of expert demonstrations. Our method is based on the observation that, in hindsight, a successful episode of collected experience is in fact a demonstration, so it should receive the same treatment. In particular, we propose to add a reward bonus to transitions coming from both demonstrations and successful episodes. Our approach provides a simple way of tying positive rewards and desired behaviour. We instantiate our approach with Soft Actor-Critic (SAC) [4], and compare it to three other algorithms. Just like ours, these methods are task-agnostic and generic, in the sense that they can be applied to any continuous-action off-policy algorithm with some minor modifications. SAC-fD [START_REF] Vecerík | Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewards[END_REF] (originally DDPGfD based on DDPG) introduces transitions from demonstrations into the replay buffer, and uses a mix of 1-step and n-step return losses. SAC-BC [START_REF] Nair | Overcoming exploration in reinforcement learning with demonstrations[END_REF] (has no name and originally based on DDPG) also introduces transitions from demonstrations into a separate additional replay buffer, and uses an auxiliary behaviour-cloning loss.

SAC-SAIL [START_REF] Ferret | Self-Imitation Advantage Learning[END_REF] (originally SAIL based on Q-learning algorithms) uses an additional self-imitation loss based on advantage learning. We also introduce transitions from demonstrations into the replay buffer for a fair comparison with the other methods. We propose SAC-R 2 , acronym for SAC with Reward Relabeling, a straight-forward method that could be implemented to any other off-policy RL algorithm with sparse rewards. Let 𝑅 be the sparse reward from the environment, 𝑏 the reward bonus of our method, and 𝐿 the amount of transitions that will receive the bonus. First, we add demonstration data to the buffer: the last transition of each expert trajectory is given the sparse reward 𝑅, and the last nonfinal 𝐿 transitions are given a reward equal to 𝑏. Then, as the agent explores the environment during training, every new successful episode is relabeled the same way: the last 𝐿 transitions leading to the sparse reward are assigned a reward equal to 𝑏. Intuitively, our method helps propagate the signal of the sparse reward by explicitly turning zero rewards into positive rewards, rather than entirely relying on bootstrapping rare future rewards.

METHOD

Reward bonus to demonstrations. The first part of our algorithm is most similar to SQIL [START_REF] Reddy | {SQIL}: Imitation Learning via Reinforcement Learning with Sparse Rewards[END_REF], a pure IL algorithm where the replay buffer is initially filled with demonstrations with all rewards equal to 𝑟 = 1, and new experiences collected by the agent are added with reward 𝑟 = 0. Intuitively, it gives the agent an incentive to imitate the expert. One important difference is that our method learns from both the reward bonuses and the reward from the environment. Also, SQIL gives an incentive to avoid states that were not in the demonstration data, which could potentially be harmful if those states led to successful behaviour.

Reward bonus to successful episodes. The relabeling part of our algorithm tries to mitigate this issue and is most similar to self-imitation methods such as SIL [START_REF] Oh | Self-Imitation Learning[END_REF]. In our method, the selfimitation is achieved by effectively treating successful episodes as if they were demonstrations. In order to give a reward bonus to successful episodes, we need to wait for the episodes to end first. We follow the idea presented in HER [START_REF] Andrychowicz | Hindsight Experience Replay[END_REF] to relabel past experiences and modify the transitions' rewards.

Decay and Hyper-parameters

One issue with our method so far, is that it might change the optimal policy of the problem. Intuitively, the bonus encourages a policy that requires at least 𝐿 steps to reach the goal, which might not take the shortest path available. We want our method to converge to the optimal policy of the sparse reward problem.

As stated in [START_REF] Andrew Y Ng | Policy invariance under reward transformations: Theory and application to reward shaping[END_REF], in order to still converge to the optimal policy of the original problem one can only add a reward term such as, for a given transition between two states, the term is expressible as in the difference in value of an arbitrary potential function applied to those states. Our reward bonus cannot expressed as such since it depends on the time-step of the states. To ensure that our method converges to the optimal policy, we decide to use a decay that eventually causes the bonus to completely disappear. By doing so, our method operates in two steps: An initial RL training with reward bonuses that might not converge to the optimal policy, followed by a second RL training without any reward bonuses which should converge to the optimal policy. For our experiments, we use a simple linear decay that quickly turns the bonus to 0 when the success rate stops improving.

Our method has two hyper-parameters to tune, 𝑏 and 𝐿. 𝐿 represents how far the sparse reward should be propagated into the past, and is similar to other parameters in RL such as the 𝑛 of the n-step look-ahead in Q-learning.

How about 𝑏? We propose to adjust the value of 𝑏 during training based on the agent's recent success. This also acts as a more natural decay during the entire training process. Let's place ourselves in the episodic undiscounted scenario. Let 𝜁 be the fraction of successful episodes over the last 100 episodes, and 𝑟 the total reward obtained over a successful episode 𝜏. We propose 𝑟 (𝜏)

= 𝑅 + 𝐿 • 𝑏 • (1 -𝜁 ).
Intuitively, the more the agent struggles, the larger the bonus to help guide it. We can prove that this reward-shaping bonus doesn't affect the converged performance of the algorithm as long as 𝑅 prevails over the bonuses:

undiscounted 𝑏 • 𝐿 ≤ 𝑅 discounted 𝑏 𝐿-1 ∑ 𝑖=0 𝛾 𝑖 ≤ 𝑅 • 𝛾 𝐿 (1)

EXPERIMENTS

We test our method on two different simulation environments: RL-Bench [START_REF] James | Rlbench: The robot learning benchmark & learning environment[END_REF] and Meta-World [START_REF] Yu | Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning[END_REF]. The reward is fully sparse, and is equal to +100 if the robot solves the task and 0 otherwise. The state includes the robot proprioceptive state (joint angles, joint speeds, gripper pose) and the task-related information (3D coordinates of each target object). We evaluate our method on four RLBench tasks for a 6-degrees-of-freedom robot manipulator: reaching a ball, pushing a button, flipping a switch, and sliding a block to a target square, and four Meta-World tasks for a 7-degrees-of-freedom robot manipulator: reaching a ball, pressing a button, closing a drawer, and sliding a block to a target location. The RLBench demonstrations are generated with motion planning, while the Meta-World demonstrations come from a pre-trained RL agent. 1: Experimental results (averaged on three random seeds) for all 8 tasks. The values correspond to the final success rate and the amount of iterations needed to converge.

SAC-Demo

We compare our method SAC-R 2 to a simple baseline SAC-Demo, which we define as SAC with demonstrations in the buffer, as well as SAC-fD, SAC-BC and SAC-SAIL. The results are reported in Table 1 and show that our method has comparable results to the other baselines. Overall, the best methods are SAC-fD in the RLBench tasks, and SAC-BC in the Meta-World tasks. Hyper-parameters. Figure 2 shows the impact of different values of 𝐿 and 𝑏. For this RLBench task, there seems to exist an optimal value for both parameters. In the case of 𝑏, the optimal value is attained for the largest possible value that respects the condition 1.

DISCUSSION

We propose Reward Relabeling (R 2 ), a generic method that can be applied to any off-policy RL algorithm in any sparse-reward environment. It encourages two behaviours: imitate the expert demonstrations (if available), and imitate the past successful trajectories. From our experiments, our method SAC-R 2 had comparable results to previous works. In the full version of this paper [START_REF] Bujalance | STIR 2 : Reward Relabelling for combined Reinforcement and Imitation Learning on sparse-reward tasks[END_REF], we propose an additional algorithm that combines elements from SAC-R 2 , SAC-fD, SAC-BC and SAC-SAIL to outperform all baselines.

Figure 1 :

 1 Figure 1: Reward Relabeling procedure.

Figure 2 :

 2 Figure 2: For a given hyper-parameter value, we plot the number of training steps required for the average episode success to reach a certain threshold.

Proc. of the 22nd International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2023), A. Ricci, W. Yeoh, N. Agmon, B. An (eds.), May 29 -June 2, 2023, London, United Kingdom. © 2023 International Foundation for Autonomous Agents and Multiagent Systems (www.ifaamas.org). All rights reserved.


Adil Baigadilov

Stéfan Colombano

Sagyn Omirbekov

Maxime Cochennec

Dorian Davarzani

Fabien Lion

Laurent Oxarango

Hugues Bodiguel

Evaluation of stability and performance of foam composed of complex mixtures of surfactant, nanoparticles and polymer for in-situ remediation of LNAPL

Keywords: foam, foam stability, polymer, nanoparticles, soil remediation, LNAPL, porous media

Purpose of study:

Light Non-Aqueous Phase Liquids (LNAPL: gasoline, diesel, motor oil, etc.) -immiscible organic liquids which are one the most common source of groundwater pollution, particularly in France they represent 37.22% of polluted sites. The presence of LNAPL in subsurface mostly associated with accidental spills and leaks during storage and transportation. As a result of the low mass removal efficiency of the pump and treat in-situ environmental remediation (ER) methods are receiving more attention thanks to the lower risk of contaminant migration and higher sustainability. Moreover, due to the heterogeneity of the soil, even the more advanced recovery methods like in-situ surfactant flushing reach their technical limits. Injected fluid preferentially flows in highly permeable areas caused by differences in viscosity. Among the increasing interest in shear-thinning fluids for in-situ ER, foam (being 60-99% of gas in a continuous liquid phase) is a promising displacing agent. However, it is worth noting the main challenge of using foam for ER. When foam gets in contact with organic compounds, its stability deteriorates dramatically, which leads to poor sweeping efficiency. A current focus among many foam stability studies is on methods to enhance foam using a variety of additives, such as co-surfactants, polymers, and nanoparticles (NPs). It should be noted that the majority of these studies are focused on foam applications in EOR. Thus, our study focuses on (i) determining the destructive effect of LNPL on foam properties, (ii) examining the stability of foam generated by two or more additives in the presence of a contaminant, both in bulk form and in porous medium, (iii) evaluating the recovery yields of each of the foaming formulations.

Methodology:

All foaming components used in our study were environmentally friendly. Diesel oil was used as the source of LNAPL. Two biodegradable surfactants (sodium dodecyl sulfate (SDS) and cocamidopropyl hydroxysultaine (CAHS)), and their mixture were used for the main foaming components. Silica (SiO2) NPs and Xanthan Gum (XG) biopolymer were employed as foam-enhancing additives. All prepared foaming formulations were experimentally investigated for their foaming ability and stability (half-life) in the presence of diesel oil using the bulk and bubble scale foam screening method. The rheological properties of the foaming liquids were also studied using a rheometer. Two decimetric 1D columns filled with sand and connected in series (i.e. foam generator and highly permeable diesel saturated sandpack) were utilized to (i) generate a fully developed foam flow and (ii) assess foam Resistance Factor (RF) (i.e. foam strength) and removal efficiency of diesel initially at residual saturation. Each injection experiment involved monitoring the pressure drop along the sandpack and collecting the effluents.

Results:

Our results showed significant destruction effect of diesel oil on foam stability. For instance, the halflife of the bulk SDS foam decreased 8 times in the presence of LNAPL, while the Resistance Factor decreased by more than 2.2 times in the porous medium. However stability assessments clearly demonstrated improved performance in the presence of diesel oil both in bulk and porous medium, when complex foaming solutions were employed. The bulk foam analysis showed that in contrast to SDS alone, the mixture of SDS and CAHS (SC), SC+NPs and SC+XG enhanced foam stability more than 7.5 times. From 1D sand column experiments, we found foam enhancement of 2.2, 2.5, and 2.8 times for SC, SC+NPs, and SC+XG, respectively. Moreover, for all these complex foaming solutions there was a significant recovery efficiency of more than 90%.

Conclusion/contributions of study:

Additional studies are being carried out to deeply characterize how the presence of NPs and polymer impacts the diesel oil recovery. Nevertheless, based on our results we can conclude that advanced foam formulations clearly exhibited promising perspectives to develop an efficient in-situ remediation technique for highly permeable soils contaminated by LNAPL.