James F Peters
email: james.peters3@umanitoba.ca

Roberto Alfano
email: roberto.alfano@physics.org

AND Peter Smith
email: peter.smith@sunderland.ac.uk

Arturo Tozzi
email: tozziarturo@libero.it

Arturo Tozzi Discrete

Arturo Tozzi Dedicated

J H C Whitehead

S Lefschetz

DISCRETE AND CONTINUOUS CURVED SURFACE ANTIPODAL PATHS. EXTENSIONS OF THE BORSUK-ULAM THEOREM AND APPLICATION OF THE FEYNMAN PATH INTEGRAL AND WODEHOUSE CONTOUR INTEGRAL

Keywords: 2010 Mathematics Subject Classification. 05C38 (paths and cycles), 14H55 (Riemann surfaces . arc, cell, curvature, path, surface, vector

d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

This paper introduces a path-Borsuk-Ulam Theorem, stemming from three main forms of paths over curved surfaces that have been identified, namely, 1 o Poincaré Contour paths were introduced by Poincaré in 1892 in his analysis situs paper [START_REF] Poincaré | Sur l'analysis situs[END_REF]. In a contour path, each subpath is an infinitely small contour on a manifold [14, p. 240]. Recently, N.M.J. Woodhouse [START_REF] Woodhouse | Contour integrals for the ultrahyperbolic wave equation[END_REF] introduced contour integrals defined on twistor curves on a complex manifold. 2 o Whitehead Homotopic paths were introduced during the late 1940s by J.H.C. Whitehead [START_REF] Whitehead | Combinatorial homotopy. I[END_REF], [START_REF]Combinatorial homotopy. II[END_REF] and S. Lefschetz [START_REF]Introduction to topology[END_REF]. For Lefschetz, a homotopic path h in an arcwise connected space S is simply a mapping of a directed (= oriented) closed arc > v 0 , v 1 into R [6, p. 158]. A space is arcwise connected, provided every vector in the space S is on a path containing an initial vector and a terminal vector. 3 o Feynman paths were introduced by R.P. Feynman in his thesis completed in 1942 [3, p. xiv]. A Feynman path is a trace of the trajectory of a particle between fixed endpoints [3, p. xiv], providing a framework for a path integral, also introduced by Feynman [START_REF] Feynman | The principle of least action in quantum mechanics[END_REF] and elaborated by R.P. Feynman and A.R. Hibbs in [START_REF] Feynman | Quantum mechanics and path integrals[END_REF]. A Penrose path over a twistor curve (from R. Penrose's 1968 paper [START_REF] Penrose | Twistor quantization and curved space-time[END_REF]) and its refinement by R.S. Ward in his 1977 thesis [START_REF] Ward | Curved twistors spaces[END_REF] supervised by Penrose, is a form of Feynman path in which the trajectory of a particle is over a twistor curve.

The original BUT [START_REF]Drei sätze über die n-dimensionale euklidische sphäre[END_REF] from K. Borsuk in 1933 is given in terms of antipodal vectors ⃗ p, -⃗ p on the surface of an n-dimensional Euclidean sphere S n , defined by

S n = ⃗ p(x 1 , . . . , x n+1 ) ∈ R n+1 , n ≥ 2|x 2 1 + • • • + x 2 n+1 = 1 .
Points on the surface of a sphere are antipodal, provided the points are diametrically opposite each other. Examples of antipodal vectors are the poles on the surface of a planet.

In 1933, K. Borsuk introduced his Theorem 1.

BUT.

Theorem 1. (Borsuk-Ulam Theorem) [2, p. 178]. Let vector ⃗ p ∈ S n . For a continuous map f : S n → R n , there is an antipodal point -⃗ p such that f (⃗ p) = f (-⃗ p).
Remark 1. Theorem 1 is a translation from German, which is given by J. Matoussȇk [9,p. 21].

Remark 2.

The basis for Theorem 1 came from K. Borsuk's thesis completed in 1930 [START_REF] Borsuk | On retractions and related sets[END_REF]. Ulam is credited by Borsuk (in a footnote [2, p. 178]) with the idea codified in Theorem 1, which Ulam stated as a conjecture. In effect, Borsuk proved Ulam's conjecture in 1933. In 1930, L. Lusternik and S. Shnirel'man introduced the nonvoid intersection of sets of closed surface curves that have antipodal vectors in common.

LST.

Theorem 2. (Lusternik-Shnirel'man Theorem) [START_REF] Lusternik | man, Topological problems in variational problems (russisa), Issledowatelskii Institut Matematiki i Mechaniki pri[END_REF]. For any cover F 1 , . . . , F n+1 of the sphere S n by n + 1 closed sets, there is at least one set containing a pair of antipodal points common to

F i , -F i (i.e., F i ∩ -F i ̸ = 0). Remark 3.
Theorem 2 is a translation from Russian, which is given by J. Matoussȇk [9,p. 21].

Theorem 2 contrasts with Theorem 1. In the Lusternik-Shnirel'man Theorem 2, there is a closed set F i that is a cover of a sphere S n and that has an opposite set -F i , in which the sets F i , -F i contain antipodal points such that F i ∩-F i ̸ = ∅. This sharply contrasts with the Borsuk-Ulam Theorem, which asserts there is a continuous map f from S n into R n over a surface containing antipodal surface vectors ⃗ p, -⃗ p such that f (⃗ p) = f (-⃗ p). Also, Theorem 2 concludes with the observation that the intersection of F i , -F i is nonvoid but the values of the shared antipodal points are not given. In the LS theorem formulation, it is possible that the antipodal points in F i ∩ -F i have different values. By contrast, in the Theorem 1 formulation, it is asserted that the antipodal points map to the same value.

Let Id be a discrete form of a unit interval [0, 1] defined by i ∈ Z + natural numbers 1,2,3,….

X =

i.e., there is a gap between x i and x i+1

{X i ∈ S n | |x i+1 -x i | > 0} . Id = {0, 1} ∪ {X : X i ∈ S n } .
Example 1. Let X 0.00001 be a set of surface points on an n-dimensional Riemannian sphere S n , defined by X 0.00001 = i.e., there is a 0.00001 gap between x i and x i+1 .

{X i ∈ S n | |x i+1 -x i | = 0.00001} . Id = {0, 1} ∪ {X 0.1 : X i ∈ S n } .

Preliminaries

More recent versions of BUT [11, §68,p.405] ??. This path begins at vector ⃗ h(0) ∈ R n at ⃗ v 1 on the surface of S and ends at vector ⃗ h(1) ∈ R n , which is the value of antipode of ⃗ v 1 . The assumption made here is that ⃗ h(0) and ⃗ h [START_REF] Borsuk | On retractions and related sets[END_REF] have the same value such as identical temperature.

That is, a discrete path h : Id → S is a mapping from the discrete unit interval Id ⊂ I (for I = [0, 1]) to a bounded, simply connected surface S with non-zero curvature. Path h is discrete, since there gaps between all points ⃗ h(t) ∈ S between 0 and 1 in Id ⊂ [0, 1]. The surface S is simply connected, provided every path h has end points h(0), h(1) ∈ S and h has no self-loops.

Paths either lie entirely on a surface in the planar case or lie on a surface and, possibly, puncture a surface in the non-planar case. Paths that puncture a surface are called cross-cuts. A cross cut path P (also called an ideal arc [10, §3, p.11]) has both ends in P and path interior in the interior of S. [START_REF] Whitehead | Combinatorial homotopy. I[END_REF]. For Whitehead, a path h : [0, 1] → X is a continuous mapping from the unit interval to a cell complex X. In the pursuit of discrete paths in a curved space, the focus is on 0-cells (single points) and 1-cells (arcs) in an n-dimensional Riemannian space S. A single surface vector is a 0-cell. A Lefschetz arc is a curvilinear line seqment attached to a pair of 0-cells [START_REF] Lefschetz | Applications of algebraic topology. Graphs and networks, the picard-lefschetz theory and feynman integrals[END_REF].

Remark 4. Homotopic paths were introduced by J.H.C. Whitehead

Antipodal and Non-Antipodal Path Borsuk-Ulam Theorem

This section introduces results for paths on a Riemann surface. Theorem 3. The endpoints of a discrete Lefschetz arc can be the same.

Proof. Let > v 0 , v 1 be a discrete Lefschetz arc in which the endpoints v 0 = v 1 are the same. There is an arc between v 0 , v 1 , namely, itself. Hence, from Def. 3, the endpoints are path-connected.

Lemma 1. Every discrete path constructs a discrete arc. Proof. Every h(t) is a point on an arc

> h(0), h(1) between h(0) and h( 1) 1) is also discrete.

is a discrete path h : Id → S. Id is discrete. Hence, > h(0), h(
Theorem 4. Every discrete path constructs a vector field.

Proof. Let h : Id → S be a discrete path. From Lemma 1, h constructs a discrete arc > h(0), h(1) on a surface S. Consequently, each h(t) ∈ > h(0), h(1) has a location (x 1 , . . . ) ∈ S with its own magnitude and direction S, i.e., every h(t) is a vector in S. Hence, h constructs a vector field. Lemma 2. Let ⃗ v 1 , ⃗ v 2 be antipodal vectors on the surface of a sphere S. There exists a discrete path h with endpoints are antipodal on a surface S.

Proof. Let ⃗ v 1 , ⃗ v 2 be antipodal vectors on the surface of a sphere S. Then define a discrete path h :

Id → S with ⃗ h(0) = ⃗ v 1 and ⃗ h(1) = ⃗ v 2 .
Hence, a discrete path can be defined for every pair of antipodal points on S.

From what we have observed about discrete paths on the surface of a sphere, we obtain path-BUT. Observe that a path can be constructed between any pair of surface vectors. This observation leads to more general form of Theorem 5.

Theorem 5. (Path-Borsuk-Ulam Theorem). Let the discrete unit interval Id be an index set for vectors

v 0 , . . . , v t , . . . , v 1 , t ∈ Id in S n in a path h : S n → R n , i.e., a path discretely maps v 0 , . . . , v t , . . . , v 1 to a set of values {h(v 0 ), . . . , h(v t ), . . . , h(v 1 )} in R n . For a discrete path h : S n → R n and a vector ⃗ p ∈ S n , there is an antipodal vector ⃗ -p ∈ S n such that h(⃗ p) = h( ⃗ -p). Proof. From Lemma 1, a path h : S n → R n constructs

Theorem 6. (Non-antipodal path-BUT).

Let the discrete unit interval Id be an index set for vectors v 0 , . . . , v t , . . . , v 1 , t ∈ Id in S n in a path h : S n → R n . There is a discrete path h :

S n → R n with endpoints h(v 0 ), h(v 1 ) that are values in R n such that h(v 0 ) = h(v 1 ).
Proof. In the proof of Theorem 5, replace antipodal points on a Leftschetz arc with endpoints that are any pair of surface points. In addition, notice that the initial point and the ending point of any discrete path can be the same. Hence. the desired result follows.

Example 8.

An example of a discrete path h : S 2 → R 3 on a 3D Gomboc Riemannian surface is shown in Fig. 2. The same path is also depicted on a 2D slice of the 3D surface. In keeping with Theorem 6, each vector ⃗ h(v t ) is a signal value from the path h. For example, if we let the descrete path be an optical field flow containing a stream of photons reflected from a Riemannian surface, then there are number of possible signal values for ⃗ h(v t ), e.g.,

1 o wavelength of ⃗ h(v t ). 2 o frequency of ⃗ h(v t ). 3 o electron voltage of ⃗ h(v t ). 4 o lumens (luminosity) of ⃗ h(v t ).
5 o gradient of ⃗ h(v t ), t ∈ Id, which would be perpendicular to the surface at (x, y, z), defined by

grad( ⃗ h(v t )) = ∂ ⃗ h x i + ∂ ⃗ h y j + ∂ ⃗ h z k.

Feynman Trajectories of a Particle

This section introduces particle trajectories as continuous paths over the curvature of space-time, which leads to the counterpart of the discrete path results already given. The transition from discrete paths results from the geometry of space-time generated by quantum processes [START_REF] Mason | Twistors in curved space-time[END_REF], which is in keeping with the observation by R. Penrose [START_REF] Penrose | Twistor quantization and curved space-time[END_REF] that the link between space-time curvature and quantum processes such as those found in Feynman trajectory of a particle is supplied by the use of twistors. A twistor space is a complex manifold CM . For example, a Lefschetz arc in curved space-time is a R.S. Ward hypersurface S twistor [16, p.56], which is a complex curve ℓ in CS. x n ′ in a N.M.J. Woodhouse [START_REF] Woodhouse | Contour integrals for the ultrahyperbolic wave equation[END_REF] twistor space R 2 × S 2 with metric signature + + --. Remark 6. From Def. 4, the vectors in h({t xi }) are J.H.C. Wodehouse zero cells in an arcwise-connected space R × S 2 × R.

Example 9. A sample twistor curve ℓ ∈ CS is shown in Fig. 3, which is a geometric realization of a Feynman trajectory of a particle (see Def. 4), which leads to a space-time view of a Lefschetz arc (see Def. 4 and Lemma 3).

Definition 4. The trajectory of a particle in a 2-plane in curved space-time is a mapping

h : R 2 × S 2 → R 2 × S 2 defined by h( t tx i ) = > t tx 0 , t tx i ∪ > t tx i , t ′ t ′ x i ′ = ℓ, t, t ′ ∈ R, x i ∈ S 2 , i ∈ I, in which each > t tx i , t ′ x i ′ is

Definition 5. (Lefschetz arc).

A Lefschetz arc E is a curve ℓ attached between a pair of 0-cells p, p ′ . The curve ℓ is dense and the points in ℓ are path-connected, i.e., between every pair of points q, q ′ in ℓ, there is a sequence of sub-arcs traversable between q and q ′ .

Lemma 3. (Lefschetz trajectory arc).

A trajectory of a particle is realizable as a Lefschetz arc.

Proof. From Def. 4, a trajectory h is a curve ℓ that starts and ends with a 0-cell and is the union of subarcs in an arcwise-connected space. Hence, from Def. 5, the trajectory h is realizable as a Lefschetz arc.

Example 10. A sample trajectory of a particle as a Lefschetz arc over a twistor curve realized as a Lefschetz arc ℓ = > pc with endpoints (0-cells) ⃗

p, ⃗ q and which is the union of sub-arcs is shown in Fig. 3.

Definition 6. (Unit interval).

The unit I = [0, 1] ∈ R is the set of all real values in the closed interval with initial value 0 and ending 1 and an unbounded number of consecutive everywhere dense subintervals of real values between 0 and 1. That is, every real number x in a subinterval of A ⊂ I has another real number x ′ ∈ A that is arbitrarily close to x.

Lemma 4. The trajectory of a particle is continuous.

Proof. From Def. 6, I is dense and is the index set for the points in the trajectory of a particle. From L. Susskind and A. Friedman [15, p. 238], a paricle moving along the Lefschetz curve can be found at any real value in the unit interval I = [0, 1]. Let h be the trajectory of a particle. Then if i, i ′ ∈ I are close, then t xi , t x ′ i are close. Hence, the trajectory h is continuous.

Example 11. (Trajectory of a Particle).

Let g : I → R be defined by g(t i ) = t i , i ∈ I, g(t i ) ∈ R for each time t i in the trajectory of a particle. That is, I is a index set for the instants of time of occurrence of the points in the trajectory of a particle over a vector field. The map f is continuous, since, for every pair t i , t j , i ̸ = j, if i, j are arbitrarily close, then g(t i ), g(t j ) are arbitrarily close. Hence, the g is continuous.

Feynman Path Integral

In this section, it is observed that a Feynman path is continuous (Lemma 5), which leads to the results in Theorem 7 and Theorem 8 for Feynman paths, which are consequences of the Borsuk-Ulam Theorem. Definition 7. A Feynman path is a function h(t xi ) = x i for a particle at point x i at time t [4, p.31].

Lemma 5. (Feynman Path). Every Feynman path is continuous.

Proof. Let h : R 2 × R 2 → S 2 be a Feynman path, defined by h(t tx a ) = x a which is the trajectory of a particle at point x a at time t t . Let ℓ represent that a particle travels over during its trajectory and let h(t tx a ) = x a be a point in ℓ. For simplicity, the curve ℓ is referred to as the trajectory of a particle and p is point in ℓ. During the passage of a particle over ℓ, ℓ has no gaps in it. That is, for each point h(t tx a ) = x a ∈ ℓ at time t t , if x a is arbitrarily close to point h(t tx k+1 ) = x k+1 ∈ ℓ at time t t , then h(t tx a ) is arbitrarily close to h(t x k+1 ). Hence, a Feynman path h is continuous. Remark 7. In Lemma 5, the continuity of a Feynman path h is explained in terms of the closeness (nearness) paradigm from [12, §1.5, p. 8], instead of the abstract (less intuitive) ϵ -δ view of continuity. This approach befits the character of the trajectory of a particle over a curve ℓ, where both the trajectory of a particle and the curve ℓ (without gaps) traced by particle trajectory. Just as pairs of points in the curve ℓ can be arbitraily close, so too,from Lemma 4, the vectors h(t tx a )), h(t tx k+1 ) in the trajectory of a particle can be arbitrarily close.

The value of a path between points a and b on a curve ℓ (the positions of a particle trajectory at times t a , t b , respectively), is K(b, a), defined in a complex space CS with respect to Planck's constant by Feynman and Hibbs [4,p. The Feynman trajectory of a particle satisfies Borsuk-Ulam Theorem 1. Let h : S n → R n be the trajectory of a particle on the surface of sphere. There is at least one pair vectors ⃗ p, ⃗ p ∈ S n such that h(⃗ p) = h( ⃗ p ′ ).

Proof. From Lemma 5, a Feynman trajectory is continuous. Hence, from Theorem 1, we obtain the desired result for antipodal points ⃗ p, ⃗ p ∈ S n in Feynman tracjectory h.

Theorem 9. (Feynman Path Integral Theorem).

There exists a Feynman path with an initial path integral K(b k , x a ) for an initial vector ⃗ x a that equals the path integral K(b k , -x a ) for a later vector -⃗ x a , which may or may not be the antipode of vector ⃗ x a .

Proof. K(b k , x a ) are Feynman path integrals that resonate (have values) for a particle that has gradients on either two different surface curvatures along a surface curve ℓ or on the same surface curvature on a path ℓ ′ for a boomerang trajectory that follows a path that is a cycle. In either case, choose an intermediate point b k in the path between ⃗ x a and b k so that the two segments on ℓ have the same length. In that case, K(b k , x a ) = K(b k , -x a ) Remark 8. The significance of Theorem 9 is that the endpoints on a particle trajectory curve ℓ need not be antipodal points. That is, Theorem 9 is more general than Theorem 7.

Woodhouse Borsuk-Ulam Theorem

This section briefly introduces N.M.J. Woodhouse contour integrals [19, p. 198], defined with respect to real α-planes in a twistor space R 2 × S 1 in which S 1 represents the α-planes that lie in the null cone at ∞, over which we have ξ = x 1 + ix 2 and τ = t 1 + it 2 , representing α-planes as surfaces, with w = ξ + zr, w = ξ + zr, constant for z = e iθ . ϕ(ξ, τ, ξ, τ ) = 1 2π |z|=1 f (ξ + zτ, ξ + zτ , z) dz z .

Definition 8. (Contour Integral as a Smooth function).

The contour integral ϕ is a smooth function on a twistor space [START_REF] Woodhouse | Contour integrals for the ultrahyperbolic wave equation[END_REF]. That is, ϕ is continuous. Theorem 11. Let ϕ p , ϕ ′ p ′ be the Wodehouse contour integral over a twistor curve ℓ and let p, p ′ be any two distinct points on ℓ. Then there are ϕ, ϕ ′ such that ϕ p = ϕ ′ p ′ . Proof. Again, replace the Feynman path integral with the Wodehouse contour integral in the proof of Theorem 9, and the desired result follows. That is, we can always find a point q between p, p ′ on the twistor ℓ such that ϕ p = ϕ ′ p ′ . Remark 9. Theorem 11 covers a broader spectrum of twistor length measurements than Theorem 10. That is, for any pair of distinct vectors on a twistor curve, we can always find an intermediate vector so that the contour integrals over the resulting twistor sub-arcs have equal value.

Example 13. Sample contour integrals on sub-twistor curves

> ⃗ p, ⃗ q, > ⃗ p ′ , ⃗ q with end points ⃗ p, ⃗ p ′ that may or may not be non-antipodal are shown in Fig. 5.

Concluding Remarks

The Borsuk-Ulam Theorem is topological with an implicit surface geometry. That is, the focus in the Borsuk-Ulam Theorem is on a continuous map from the surface of a sphere S n to real values of antipodes in R n . The geometry underlying the Borsuk-Ulam Theorem looms up, when we consider where to look for antipodes such as the endpoints on an arc stretching over the curved surface of the Earth between a place such as the University of the Manitoba and another place at varying space-times with the same latitude and longitude.

The beauty of the Borsuk-Ulam Theorem is that it serves as a roadmap, telling us how to look for surface antipodes (with continuous maps), where to look (e.g., endpoints of twistor curves that are antipodal) and how to measure antipodal distances. In this paper, the Borsuk-Ulam Theorem is an emperor with new clothes to wear, namely, 1 o How to look (with either a discrete or continuous map).

2 o Where to look (endpoints of twistor curves that are either antipodal or nonantipodal). 3 o Length-of-arc measure (with either Feynman path integrals or Wodehouse contour integrals over arcs having antipodal endpoints).

Figure 1 .

 1 Figure 1. Discrete paths h i : Id → S with all t ∈ Id.

Definition 3 .Example 5 .

 35 (Path-Connected) A pair of vectors v 0 , v 1 are path-Connected, provided there is a sequences of arcs between the vectors. All vectors on the circle in Fig.?? are path-connected, since there is a sequence of arcs between each pair of vectors.

Figure 2 .Example 7 .

 27 Figure 2. 2D and 3D views of discrete paths on a Gombox Riemannian surface

Figure 3 .

 3 Figure 3. Trajectory of a particle over twistor curve realizable as the union of a sequence of sub-arcs on Lefschetz arc ℓ = > pq on a R.S. Ward hypersurface CS [16, p.62].

eTheorem 7 .Theorem 8 .

 78 45] by V (x, t) = Potential energy of particle with mass m. L = m 2 ẋ2 -V (x, y)(Lagragian for the system). ( i )S[b,a] Dx(t). (Feynman Path Theorem). Let h : S 3 × R 3 → S 3 be a Feynman path over a curved surface S 3 and let K(b k , x a ), K(-x a , b k ) be the value of path h containing points b k , x a in a segment > b k , x a in a curve ℓ starting at point x a and with endpoint b k attached to a segment > -x a , b k starting at point b k and with endpoint -x a . For each initial point x a , there exists an antipodal point -x a in path h such that K(b k , x a ) = K(-x a , b k ). Proof. From Lemma 5, a Feynman path h is continuous. Hence, from Theorem 1, we obtain the desired result for antipodal points x a , x b in path h. (Feynman Trajectory-of-Particle Theorem).

Figure 4 .Theorem 10 .Example 12 .

 41012 Figure 4. Wodehouse contour integrals on sub-twistor curve antipodes

Figure 5 .

 5 Figure 5. Wodehouse contour integrals on sub-twistor curves

Definition 1. Discrete map. Discretely close surface points

  require the map f : S n → R n to be continuous, i.e., the map f is continuous, provided, for each subset E ⊂ S

	n , if
	a point ⃗ p is arbitrarily close to E, then f (⃗ p) is arbitrarily close to E. However, in
	keeping with an interest in Riemannian surface surface points with gaps between
	them, we consider discrete maps.
	⃗ p, ⃗ q such as water molecules always have a
	minute gap between them. A map f : S n → R n is discrete, provided, for each
	subset E ⊂ S n , if a point ⃗ p is discretely close to E, then f (⃗ p) = ⃗ q is close to E.
	Example 2. Discretely close surface points ⃗ p, ⃗ q such as close water molecules
	always have a minute gap between them.

Example 3. The discrete unit interval Id is a collection Discretely close surface points

	⃗ p, ⃗ q ∈ Id such that |⃗ p -⃗ q| > 0.
	Definition 2. Discrete Path.
	Let S be a Riemannian surface. A discrete path h is a discrete mapping, defined
	by
	h : Id → S, such that
	h(0) ∈ S is the initial point in path h, h(1) ∈ S the endpoint of h and h(t) ∈ S for
	all t ∈ {x : x ∈ X ⊂ Id}.
	Example 4. A sample discrete path h : Id → S on a circular slide of a Riemannian
	sphere is shown in Fig.

  of values from vectors on an arc with endpoints ⃗ v, vecv ′ on S n . Let the endpoints of this arc be antipodal. Hence, from Lemma 2, the desired result follows.

	Remark 5. An immediate consequence of Theorem 5 is that, for any pair
	of antipodal surface points, we can always introduce a discrete path h that
	begins and ends at the antipodal points such as places that have same latitude

and longitude. For example, the antipode of Winnipeg, Manitoba, Canada with coordinates 49 • .53'N, 97 • .8'W is Port-aux-Français, Kerguelen, French Southern Territories. Example 6. An example of a discrete path that begins and ends at antipodal surface points is shown in Fig. ??.


Remya Sankar
email: remya.sankar@inria.fr

Arthur Leblois
email: arthur.leblois@u-bordeaux.fr

Nicolas P Rougier
email: nicolas.rougier@inria.fr

Dual pathway architecture underlying vocal learning in songbirds 1

Keywords: sensorimotor learning, reinforcement learning, birdsong, simulated annealing

Song acquisition and production in songbirds is governed by a dedicated neural circuitry that involves two parallel pathways: a motor pathway for the production and a basal ganglia (BG) pathway for the acquisition. Juveniles learn by imitating adult vocalizations and proceed by trial and error, errors being conveyed by a dopaminergic signal. The complex nature of the relationship between neural control and syrinx musculature makes song learning a complicated problem to solve. Reinforcement learning (RL) has been widely hypothesized to underlie such sensorimotor learning even though this can lead to sub-optimal solutions under uneven contours in continuous action spaces. In this article, we propose to re-interpret the role of a dual pathway architecture, underlying avian vocal learning, that helps overcome these limitations. We posit that the BG pathway conducts exploration by inducing large daily shifts in the vocal production while the motor pathway gradually consolidates this exploration. This process can be understood as a modified form of a simulated annealing process. Simulations on Gaussian performance landscapes and a syrinx-based performance landscape are demonstrated and compared with standard approaches. Taking behavioral constraints into account (60 days of learning, 1000 trials per day), the model allows to reach the global optimum in complex landscapes and thus provides a sound insight into the role of the dual pathway architecture underlying vocal learning.

I. INTRODUCTION

The acquisition of motor skills in vertebrates is widely believed to be governed by reinforcement learning (RL) wherein a skill is progressively learned through a series of trials and errors [START_REF] Boraud | A natural history of skills[END_REF]. Each trial in the motor space can be perceived in the sensory space to subsequently provide an evaluation of the performance, which will in turn generate a reward signal to the motor system and help rectify the action. In this view, learning to produce a given sensorimotor target involves finding the global optimum of the reward landscape across all possible motor commands. While previous work has shown how neural circuitry may implement RL (direct policy search ) through gradient ascent in the reward landscape to maximize the cumulative reward, such a policy is deemed to fail in the presence of local maxima. Other gradient-free This work was partially funded by grant ANR-17-CE24-0036 and ANR-16-CE37-0020-01.

† Equal contribution.

algorithms (e.g. simulated annealing) are more efficient in complex uneven reward landscapes, but the mechanisms of their implementation in brain circuits remains speculative [START_REF] Tsallis | Generalized simulated annealing[END_REF].

In order to look into the mechanisms underlying sensorimotor learning, we consider the paradigm of vocal learning in songbirds. Based on their behavior, brain anatomy and physiology, we hypothesise that motor learning is governed by a dual pathway architecture that allows for an efficient learning, mixing reinforcement learning and the regulation of noise. The point we want to make here is not about optimality but rather plausibility. To do so, we adopt an alternative to classical approaches: instead of trying to justify a posteriori the existence of one or the other critical feature (e.g. back-propagation) inside the brain, we start from the raw biological facts and explore different hypotheses as to how the neural circuitry could implement efficient vocal learning.

Juvenile songbirds learn to imitate the vocalisations of an adult tutor through vocal learning, a form of sensorimotor learning akin to human speech learning. During development, the juvenile's vocalizations progress from highly variable (vocal babbling) to highly stereotyped and accurate imitations of the tutor song through a trial-and-error process indicating reinforcement learning. In zebra finches, song learning lasts around 60 days during which a juvenile produces thousands of vocalisations per day [START_REF] Derégnaucourt | How sleep affects the developmental learning of bird song[END_REF]. Vocalisations undergo changes at multiple timescales. While rapid changes are observed across vocalisations produced in a single day, some changes are consolidated on a weekly timescale [START_REF] Derégnaucourt | How sleep affects the developmental learning of bird song[END_REF], [START_REF] Kollmorgen | Nearest neighbours reveal fast and slow components of motor learning[END_REF]. Moreover, sleep induces a rapid discontinuity in the produced vocalisations with increased variability post-sleep [START_REF] Derégnaucourt | How sleep affects the developmental learning of bird song[END_REF]. Overnight changes, daily fluctuations and weekly consolidations are only partially aligned, making the song learning an erratic process [START_REF] Kollmorgen | Nearest neighbours reveal fast and slow components of motor learning[END_REF].

Song acquisition and production is governed by a dedicated neural circuitry that involves two parallel pathways: a cortical (motor) pathway controlling vocal production in adults and a basal ganglia-thalamo-cortical (BG) pathway necessary for vocal learning and plasticity (Figure 1a). These pathways connect two main cortical nuclei: the premotor HVC (used as a proper name) generating song timing [START_REF] Hahnloser | Erratum: An ultra-sparse code underlies the generation of neural sequences in a songbird[END_REF], [START_REF] Long | Using temperature to analyse temporal dynamics in the songbird motor pathway[END_REF], and the robust nucleus of the arcopallium (RA), controlling the syringeal and tracheal musculature in order to produce vocalisations. Direct axonal projections along the motor pathway develop during the early phase of song acquisition and exert a growing influence on song production during learning. On the contrary, the earlymatured BG pathway drives initial vocalizations and variability in the subsequent song production, but has a reduced influence post learning [START_REF] Mooney | Waiting periods versus early innervation: the development of axonal connections in the zebra finch song system[END_REF]. The BG pathway receives a performance evaluation signal via strong dopaminergic projections from the midbrain and drives a motor bias that rectifies vocal errors [START_REF] Bottjer | Circuits, hormones, and learning: Vocal behavior in songbirds[END_REF]. This motor bias is then gradually consolidated within the cortical motor pathway. Activity-dependent synaptic plasticity is believed to allow RL to be implemented in the BG pathway (HVC-BG synapses [START_REF] Ding | Long-Term Potentiation in an Avian Basal Ganglia Nucleus Essential for Vocal Learning[END_REF]), and the consolidation of motor bias within the cortical motor pathway (HVC-RA synapses [START_REF] Mehaffey | Naturalistic stimulation drives opposing heterosynaptic plasticity at two inputs to songbird cortex[END_REF]). Following song acquisition, the motor pathway is capable of producing the learnt song without the input from the BG pathway as lesions in the BG pathway do not affect song quality, but reduce the residual song variability. During song learning, each timing signal in HVC must be associated to the proper muscle configuration to produce a given syllable, i.e. a vocalization with the desired acoustic features. This represents a complex problem to solve in a continuous action space, with a non-linear and redundant relationship between RA neural activity, muscle activation patterns and vocal acoustics [START_REF] Srivastava | Multifunctional and Context-Dependent Control of Vocal Acoustics by Individual Muscles[END_REF].

Here, we propose to draw insights from the dual pathway architecture underlying sensorimotor learning in birds, which offers a potential solution to the aforementioned limitations of direct gradient descent approaches. We explore the benefits of this architecture by simulating a simplified algorithmic implementation of the vocal learning process in the context of different reward landscapes, both randomly generated and biologically inspired. We demonstrate that the structural (delayed maturation of cortical pathway) and functional plasticity (activity-dependent synaptic plasticity [START_REF] Ding | Long-Term Potentiation in an Avian Basal Ganglia Nucleus Essential for Vocal Learning[END_REF], [START_REF] Mehaffey | Naturalistic stimulation drives opposing heterosynaptic plasticity at two inputs to songbird cortex[END_REF]) observed in a two-pathway framework together help overcome certain shortcomings of standard RL approaches and facilitate a convergence to the global optimum.

II. DUAL PATHWAY MODEL

The goal of the model is to implement a vocal learning process consistent with the behavioural, physiological and anatomical evidence collected in songbirds and thereby gain insight into sensorimotor learning. We theoretically investigate the interplay of the two parallel pathways within the song system and their role in song acquisition. In this manuscript we choose to define an abstraction based on a simplified two dimensional (2D) representation of the dual pathway architecture, in order to perform a more systematic study of its properties in a reduced system.

A. Architecture

Taking inspiration from the vocal learning circuitry found in songbirds, the model1 has been designed as a three layered architecture with two major parallel pathways (Figure 1b). The first layer (HVC) operates as an input layer which indicates the target syllable to be produced. The second layer (RA) generates a bounded 2D motor output. The third layer mimics the working of an avian syrinx and transforms the lowdimensional motor output from the RA layer into a syllable vocalisation. The HVC and RA layers are connected by two parallel circuits inspired by the song system, the cortical pathway (motor pathway) which drives motor output and the BG-thalamo-cortical circuit (RL pathway) which implements RL and provides a tutor signal to the motor pathway (Figure 1). The outputs of the motor and RL pathways are represented by two scalar values, µ mtr , and µ rl , respectively. These values are weighted by the influence of the two pathways, w mtr and w rl that reflect their respective contributions to RA output (Eq 1-2). RA output, P , is a summation of the contributions of the motor, P mtr , and RL, P rl , pathways (Eq 3). The output, P , is further transformed into a syllable vocalisation, as explained in section II-C.

P mtr = w mtr * µ mtr (1) P rl = w rl * (µ rl + ξ) (2)
P = P mtr + P rl (3)

B. Learning

The RL pathway, µ rl , is governed by reinforcement learning, following the REINFORCE rule (Eq 6) [START_REF] Williams | Simple statistical gradient-following algorithms for connectionist reinforcement learning[END_REF]. Local exploratory noise, ξ, is injected directly into the RL pathway, along with the performance (or reward) prediction error, P P E (Eq 2, 6). Performance prediction error, P P E, here corresponds to the difference between the performance evaluation at a given trial, R tr , and the expected performance evaluation, R tr (Eq 5). The motor pathway, µ mtr , gradually consolidates the drive from the BG-led exploration, by maintaining a slow trace of the BG contribution, P rl , and eventually learns to produce the desired vocalisation (Eq 4).

∆µ mtr = η mtr * P rl (4)

P P E = R tr -R tr (5) ∆µ rl = η rl * ξ * H(P P E) (6)
where tr denotes the current trial, η rl : learning rate within the RL pathway, η mtr : learning rate within the motor pathway, Rtr : running average of recently (100 trials) obtained performance evaluations and H(x) = 1 x > 0 0 x <= 0 .

We supplement the reinforcement learning implemented by the BG pathway with two mechanisms derived from empirical evidences underlying vocal learning in songbirds.

(i) Taking inspiration from the delayed growth of the cortical motor pathway, we simulated the increasing influence of the motor pathway, µ mtr , alongside the decreasing influence of the RL pathway, µ rl . The maturation of the motor pathway is modeled using an exponentially increasing contribution, w mtr , towards the RA (Eq 1,7) while the influence of the RL pathway, w rl is modeled using an exponentially decreasing contribution (Eq 2,8).

(ii) Drawing on evidences from the studies of [START_REF] Derégnaucourt | How sleep affects the developmental learning of bird song[END_REF] showing a daily post-sleep deterioration of song structure during the sensorimotor period (discussed in section I), the output of the RL pathway, µ rl , is shifted each morning. This shift is implemented as a random jitter, ϕ (Eq 10) added to the current BG output. These two factors are weighted as per the daily consolidation trace, w k , determined by the mean rectified performance prediction error experienced in the previous day k (Eq 9).

w mtr = e -.5 * c1/tr (7)

w rl = 1 -e -c1/tr (8)
w k = c 2 * f (P P E) k (9)
µ k+1 rl = w k * µ k rl + (1 -w k ) * ϕ (10)
where c i denotes scalar constants and f (x) = xH(x).

C. Performance landscapes

The transformation of RA motor output into produced vocalizations, that are then compared to an auditory template of a tutor song for performance evaluation, represents a complex and largely unsolved problem. We therefore test the model in two different contexts. First, we map the 2D motor space of RA output to a 1D performance space through an arbitrary non-monotonic function artificially denoting the performance quality, detailed below. Second, we generate a biologically realistic performance landscape using an artificial model of the syrinx driven by RA output, detailed in this section.

In order to evaluate the performance of the model, we first create continuous performance landscapes (analogous to reward profiles), by transforming a 2D motor space to a 1D performance evaluation: The performance landscape is a set of Gaussian distributions, giving rise to several local optima along with one global optimum. Each optimum is represented by a 2D Gaussian distribution, and the performance landscape is the maximum of these distributions. We generate several contours by changing the number, position and widths of local optima, awarding a maximum performance evaluation of 60% of that of the global optimum. We categorise these set of contours into three classes with low (1-5), medium (10-20) and high (30-50) number of local optima. Figure 2a-c shows an example performance landscape from each class. These randomly generated contours provide us with explicit control over the complexity of the performance landscape, in terms of the number of local and global optima and their relative heights and positions.

Further, we test the performance of the dual pathway model with a reward contour that has been generated using a model of the avian syrinx [START_REF] Amador | Elemental gesture dynamics are encoded by song premotor cortical neurons[END_REF]. The 2D scalar output of the RA layer, P , is transformed to form the input signals for the avian syrinx model as per Eq 11, 12 (Figure 2d). The syrinx layer receives two input signals from the RA layer, corresponding to the air-sac pressure, α(t), and the tension of syringeal labia, β(t). These input signals of pressure and tension lead to oscillations in the syrinx and trachea and generate a syllable vocalisation as a pressure wave of 50ms (T ) (Figure 2e) [START_REF] Amador | Elemental gesture dynamics are encoded by song premotor cortical neurons[END_REF]. We construct a spectrogram from this oscillatory pressure wave, and choose a target syllable which is similar to a commonly-occurring syllable in zebra finches (shown in Figure 2e). We proceed to simulate several vocalisations over a range of input parameters (as described in [START_REF] Amador | Elemental gesture dynamics are encoded by song premotor cortical neurons[END_REF]), and thereby, generate a performance landscape using a similarity metric, based on the correlation coefficient between the generated vocalisations and the target template. As shown in Figure 2f, the performance landscape has three global optima, as multiple configurations of the syrinx can produce the desired vocal output. Alongside these global optima, there are several [START_REF] Srivastava | Multifunctional and Context-Dependent Control of Vocal Acoustics by Individual Muscles[END_REF] shallow local optima across the sensorimotor range.

α(t) = P α + 0.04sin((T /2 + t) * π * 10) (11) β(t) = P β -0.2e -200t -10 -5 e 200t (12)
where the motor output of the RA layer, P ∝ [P α , P β ], T : duration of the vocalisation and t ∈ [0, T ].

D. Simulation

Each simulation of the model is run over 60 days, which is the typical duration of the sensorimotor phase for a zebra finch, with each day consisting of 1000 trials [START_REF] Derégnaucourt | How sleep affects the developmental learning of bird song[END_REF]. The noise level injected by the BG is initialised at 20% of the RA output range. As we reach the crystallisation stage, the RL noise level exponentially decreases to 10% of its initial value (=2%) which is comparable to the variance observed in the pitch of the vocalisations produced by adult birds [START_REF] Sober | Central Contributions to Acoustic Variation in Birdsong[END_REF].

E. Metrics

We measure the 'terminal performance' as the mean performance evaluation obtained during the last five days of a simulation. For the Gaussian-based landscape, the global optimum has an associated reward level of 1 while all other local optima have a maximum associated performance evaluation of 0.6. Therefore, we consider a simulation to be 'successful' if it achieves a performance metric above 0.6. For the syrinx-based performance landscape, we observed that the highest peak outside the global optimum has an associated performance evaluation of 0.55 while the global optima has an associated reward of 1. Therefore, we maintain 0.6 as the threshold above which a simulation is considered to be successful. Finally, the 'success rate' of the model for a given scenario is the proportion of successful simulations compared to the total number of simulations.

F. Benchmark algorithms

In order to compare the performance of the dual pathway model with established approaches, we build a framework with a single pathway architecture, implementing variants of RL. The single pathway injects exploratory noise, receives performance evaluation and governs motor output. In essence, we lesion the motor pathway allowing the RL pathway to be in sole control of the motor output.

First, we test the performance of the single pathway framework using the standard reinforcement learning approach (StdRL). Here, the learning rule is based on gradient descent, akin to the former RL pathway (Eq 6). Under this scenario, the variability of the output does not reduce over time as the influence of the BG remains intact due to the absence of a motor pathway. Second, we compare the performance of the dual pathway system with a modified standard RL approach, i.e. RL with decreasing noise (DevRL). Here, we exponentially reduce the noise injected into the RL pathway, as learning progresses (Eq 13).

ξ tr = ξ 0 (1 -e -c1/tr ) [START_REF] Amador | Elemental gesture dynamics are encoded by song premotor cortical neurons[END_REF] where ξ 0 denotes the initial noise level injected into the pathway. Third, we compare the performance of the dual pathway system with simulated annealing (SA), a probabilistic technique, used to find the global optimum in discrete search spaces [START_REF] Tsallis | Generalized simulated annealing[END_REF]. This technique uses an explicit acceptance function M , to control the exploitation-exploration trade-off, or more specifically, the probability to move to lower rewarding positions. This acceptance probability is determined by an exponentially-decreasing temperature parameter Γ, weighted by the difference in performance evaluation between successive iterations ∆R (Eq 14-16).

Γ tr = 1 -e -c1/tr (14) ∆R tr = R candidate -R current (15)
M (tr) = e ∆R tr Γ tr [START_REF] Holtmaat | Transient and Persistent Dendritic Spines in the Neocortex In Vivo[END_REF] where ∆R tr refers to the difference in performance evaluation between a randomly chosen candidate motor output, R candidate and the current motor output R current . The candidate motor output is chosen in a range corresponding to η rl * ξ (Eq 6) to maintain an equivalent step size with the previous algorithms.

III. RESULTS

In this section, we test the proposed data-driven algorithm on our simplified implementation of the dual pathway architecture. We verify the robustness of the model using different types of reward contours and modifying exploratory parameters. We, further, compare the performance of the model with established reinforcement learning approaches.

A. Sensorimotor learning by the dual pathway model

We simulate the vocal learning process using the algorithm described above governing the dual pathway architecture, on a Gaussian-based performance landscape with a 'medium' number of local optima [START_REF] Roberts | Rapid spine stabilization and synaptic enhancement at the onset of behavioural learning[END_REF] and 1 global optimum (Figure 3). Figure 3a shows that the motor output of the model P explores several local optima, including the global optimum. Each day, the RL pathway output is shifted, following which it performs gradient ascent to find the nearest local optimum, over the course of the day. Meanwhile, over the weekly timescale, the motor pathway consolidates more information from the motor outputs that are produced more often. Figure 3a-b shows that the motor pathway does follow the model output to certain local optima. However, it is able to successfully evade it and eventually converge at the global optimum.

Figure 3b shows that the output is governed increasingly by the contribution of the motor pathway as the model approaches the global optimum. The contribution of the RL pathway induces a strong bias and high variability in the early stages. 17) and 1 global optimum using 20% initial BG variability. A The cortical motor pathway, in brown, follows the BG-led exploration to several local optima on the performance landscape before coverging at the global optimum. The black dots denote the total motor output. B Initially, the contribution of the RL pathway P rl , in grey, drives a strong bias in the motor output P , in black. As the contribution of the motor pathway Pmtr, in brown, reaches the global optimum, the BG contribution recedes. C The range of BG-led exploration, around the motor pathway, shrinks with development. Each dot represents the bias driven by the RL contribution P rl at a given trial. D Performance evaluation, in purple, fluctuates over the coarse of learning on both daily (inset) and weekly timescales. The daily BG consolidation trace w k , in black, determines the shift on the following day.

As learning progresses, the target information is consolidated into the motor pathway and the bias and variability induced by the RL pathway recedes. This helps the model to converge at the chosen peak. Figure 3c shows that in the initial days of learning, BG-led exploration ranges over a large area of the sensorimotor space around the contribution of the motor pathway, P mtr . Eventually, the growth of the motor pathway curbs the influence of the RL pathway, resulting in a reduced range of exploration as well as increased exploitation tendency. Thus, despite the same amount of noise being injected into the BG, the variability in the motor output decreases, leading to the convergence of the vocal production at the position consolidated within the cortical pathway. Progress in learning is accompanied by a non-monotonic increase in the average performance evaluation obtained by the model, as shown in Figure 3d. The daily shift in the RL pathway output induces dips in the performance evaluation, which rapidly improves within the time course of a single day, as the RL pathway finds the best local optimum within the exploration range. The variability in performance evaluation reduces as learning proceeds and as the influence of the BG-injected noise reduces over time.

B. Robustness

Having observed a demonstration of the dual pathway architecture, in this section we proceed to demonstrate the versatility and robustness of the dual pathway framework. We test the system under various types of scenarios and compare the dual pathway model with a set of benchmarks using a single pathway framework.

First, we verify the stability of the model under identical conditions (20% initial RL noise level) on the Gaussianbased performance landscape consisting of "medium" (10-20) number of local optima. We simulate motor learning within this scenario using 100 different seeds for the random number generator resulting in varying performance landscapes and observe that the model is successful in finding the global optima in 76% cases (success defined as per section II-E) (Figure 4a). Moreover, in successful simulations, the model consistently achieves a high terminal performance (above 0.9).

Second, we verify the robustness of the model under different types of performance landscapes. We test whether the number of local optima has an impact on the model performance. In order to do this, we randomly generate 100 different performance landscapes each with "low" (1-5) and "high" (30-50) number of local optima in addition to the global optimum. We simulate sensorimotor learning on these performance landscapes and observe that the model is successful in 92% and 64% cases, respectively (Figure 4a). Further, in successful simulations, the model has a terminal performance higher than 0.9. Thus, we observe that the model success rate decreases with increasing number of local optima (Figure 4a), however the terminal performance of successful simulations is unaffected.

Third, we verify the performance of the model under a biologically-inspired performance landscape, built using a model of the avian syrinx (described in section II-C). Under identical levels of BG-induced noise (20% initial RL noise), we simulate vocal learning using 100 different seeds for the random number generator. We observe that the model is successful in 92% cases and reaps a terminal performance above 0.9 in these cases (Figure 4b). Thus, the model is capable of sensorimotor learning under biologically realistic performance landscapes as well.

Fourth, we test the stability of the model when higher levels of noise are injected into the RL pathway. We simulate the model with different higher initial levels of noise (30%, 40% and 50%) injected into the RL pathway. We observe that the increase in the RL noise level leads to an improvement in the success rate of the model (Figure 4b). On the other hand, there is a slight decrease in the terminal performance of successful simulations with increase in initial RL noise levels.

C. Benchmarks

Now, we compare the performance of the dual pathway architecture with that of the single pathway framework, as described in section II-F, when governed by different learning rules. First, we test the performance of the single pathway framework using the standard reinforcement learning approach (StdRL), as described in section II-F. We observe that, over 100 simulations, each at 20% RL noise level using the syrinx-based performance landscape, the dual pathway framework (92% success rate, n=100, median terminal per-formance=0.96) performs significantly better (Mann-Whitney U=9437, p<0.01) than the single pathway framework with StdRL (55% success rate, n=100, median terminal perfor-mance=0.62), as shown in Figure 4b. Now, increasing the noise level injected in the pathway does facilitate the model to escape local optima and find the global optimum. However, this leads to a drawback where the high variability leads to a low terminal performance (n=100, median terminal perfor-mance=0.4 at 50% noise) being harvested by the model post learning (when RL noise is above 20%, terminal performance is below 0.6 even at global optimum).

Second, in order to address the aforementioned disadvantage of StdRL, we compare the performance of the dual pathway system with a modified standard RL approach, RL with decreasing noise (DevRL), as described in section II-F. Here, we explicitly reduce the noise injected into the single pathway, as learning progresses. We observe in Figure 4b that the dual pathway system achieves a higher success rate than the single pathway system with the DevRL approach at all noise levels (Mann-Whitney U=3816, p<0.01, n=100 at 50% noise). Now, increasing the level of noise injected into the pathway does help resolve this issue, and drastically improves the success rate, however this is at the cost of high vocal variability and reduced terminal performance.

Third, we compare the performance of the dual pathway system with a single pathway system implementing simulated annealing [START_REF] Tsallis | Generalized simulated annealing[END_REF]. When lower levels of noise are injected into the pathway, simulated annealing performs better (71% success rate) on the syrinx-based performance landscapes than the standard RL (55% success rate) approach due to its superior ability to escape local optima. With increasing noise, this approach yields improved success rates. On the other hand, the dual pathway framework obtains a higher success rate (92%) as well as significantly higher terminal performance metrics (Mann Whitney U=9306, p<0.01) than the simulated annealing approach (71%) at 20% RL noise as well as higher noise levels (Figure 4b).

Thus, the dual pathway framework, with its advantages and shortcomings, provides a viable approach for sensorimotor learning, even under low noise conditions.

IV. DISCUSSION AND PERSPECTIVES

Inspired from the dual pathway architecture of the vocal learning circuitry in songbirds and by the large overnight changes undergone by juvenile vocalizations during early phases of learning [START_REF] Derégnaucourt | How sleep affects the developmental learning of bird song[END_REF], we proposed a new algorithmic implementation for efficient sensorimotor learning in the face of uneven performance landscapes with multiple optima.

In songbirds, a BG pathway drives early vocalizations and rectifies vocal output through RL guided by a dopaminergic signal denoting performance evaluation [START_REF] Gadagkar | Dopamine neurons encode performance error in singing birds[END_REF]. In parallel, a cortical motor pathway consolidates BG-driven changes, utilising activity-dependent plasticity at the HVC-RA synapses [START_REF] Mehaffey | Naturalistic stimulation drives opposing heterosynaptic plasticity at two inputs to songbird cortex[END_REF]. Similarly, in the model, a trace of the RL pathway output is gradually consolidated within the motor pathway (Eq 4). First, to reflect on the large day-to-day changes in juveniles vocalizations that do not necessarily align with the longterm improvement of the song [START_REF] Kollmorgen | Nearest neighbours reveal fast and slow components of motor learning[END_REF], the initial state of the RL pathway is reset every day with a partial copy of the previous day's final output (Eq 10). As synapses in the nervous system are known to be very volatile [START_REF] Holtmaat | Transient and Persistent Dendritic Spines in the Neocortex In Vivo[END_REF], [START_REF] Roberts | Rapid spine stabilization and synaptic enhancement at the onset of behavioural learning[END_REF], the daily shift in the BG output could reflect large overnight changes in the HVC-X synapses, inducing a partial loss of the memory formed by RL in the previous days. In the model, this synaptic fluctuation has been simulated using the daily consolidation trace w k . Such a biologically plausible mechanism for partial forgetting in the RL pathway facilitates the exploration of a new region of the performance landscape each day, which helps guide the motor pathway away from the local optimum. Second, the cortical motor pathway exhibits a delayed maturation during sensorimotor learning [START_REF] Mooney | Waiting periods versus early innervation: the development of axonal connections in the zebra finch song system[END_REF]. It has been hypothesised that the development of the HVC-RA synapses plays a role in reducing the RA sensitivity towards LMAN input and thereby, gradually suppresses the BG-induced bias in motor output [START_REF] Garst-Orozco | A neural circuit mechanism for regulating vocal variability during song learning in zebra finches[END_REF]. Correspondingly, the relative influence of the two pathways in the model to the motor output changes over development (Eq 7, 8). As the RL pathway contribution to the global output decreases over development, so does the variability of the vocal output due to the daily shift and the noise in RL pathway output, consistent with the progressive decrease in song variability during learning in songbirds [START_REF] Derégnaucourt | How sleep affects the developmental learning of bird song[END_REF]. Interestingly, a parallel can be drawn between the effect of these two mechanisms and simulated annealing, a technique used to optimise stochastic gradient descent [START_REF] Tsallis | Generalized simulated annealing[END_REF]. The daily shift in BG output facilitates escape from local optima, akin to discrete fluctuations implemented within simulated annealing. The change in relative influence exerted by the two pathways plays a strikingly similar role as temperature in simulated annealing. Indeed, the exploitationexploration trade-off is initially skewed towards exploration with high trial-by-trial variability and daily changes, analogous to the high temperature condition. As learning progresses, it leans towards exploitation akin to low temperature scenarios in annealing, as mentioned in [START_REF] Derégnaucourt | How sleep affects the developmental learning of bird song[END_REF].

It is to be noted that such a heuristic does not convey any guarantees concerning the optimality of the solution. Other optimization techniques used in machine learning such as simulated annealing may succeed in pathological landscapes that pose difficulties to the model proposed here. They are however less biologically realistic as they rely on a long-term maintenance of the memory of all explored options. Alternatively, developmental regulation of variability in song through sexual hormones is known in songbirds and may provide a biologically realistic mechanism to partially overcome the drawbacks of gradient descent (as implemented above using DevRL) [START_REF] Sizemore | Premotor synaptic plasticity limited to the critical period for song learning[END_REF].

The dual pathway architecture we've introduced and illustrated in the songbird is, in fact, a widespread cerebral organization among vertebrates, including birds, rodents and primates [START_REF] Boraud | A natural history of skills[END_REF]. A rough description of this architecture is that the BG provides the necessary motor variability for exploration while the cortex (pallium for some taxa) provides stability and late exploitation. Once acquired, skills are expressed solely by the motor cortex without the need for the BG. This constitutes a generic and powerful mechanism for the acquisition of sensorimotor skills that departs from modern machine learning techniques. Given the architectural, behavioral and physiological constraints we've introduced, the dual pathway model constitutes a plausible approach to sensorimotor learning that is strongly rooted in neuroscience and behavior.

Fig. 1 .

 1 Fig. 1. Song system in zebra finches and a simplified schema of the dual pathway architecture. A The specialised vocal learning circuitry comprises of two pathways: the cortical motor pathway (brown) and the BG-thalamocortical pathway (grey). The cortical pathway governs song production and includes the premotor cortical nucleus HVC and the RA. The RA projects to downstream regions which control respiratory musculature. The parallel BG pathway receives performance evaluation from mid-brain dopaminergic neurons and projects to the RA. B. The dual pathway architecture inspired by the vocal learning circuitry. The BG pathway (grey) is based on reinforcement learning (RL) and provides a tutor signal, which is consolidated gradually within the parallel cortical motor pathway (brown). The syrinx transforms the combined output of these two pathways into a syllable vocalisation.

Fig. 2 .

 2 Fig. 2. Various types of performance landscapes. The concentric circles show equipotential surfaces. A-C. Examples of Gaussian-based performance landscapes with 1 global optimum and A. 'low' (1-5) B. 'medium' (10-20) and C. 'high' (30-50) number of local optima. D-F. Reward contour generated using a model of the avian syrinx [13]. D. The 50ms waves of tension and pressure used as input to the syrinx model to generate the target syllable.E. The spectrogram of a common zebra finch syllable chosen as the target syllable, as generated by the model. F. The performance landscape generated using the similarity between the target syllable and vocalisations generated over the parameter range used in[START_REF] Amador | Elemental gesture dynamics are encoded by song premotor cortical neurons[END_REF]. It has three global optima and several (=11) local optima.

Fig. 3 .

 3 Fig.3. Simulation of the dual pathway model on a Gaussian-based reward contour with medium number of local optima[START_REF] Roberts | Rapid spine stabilization and synaptic enhancement at the onset of behavioural learning[END_REF] and 1 global optimum using 20% initial BG variability. A The cortical motor pathway, in brown, follows the BG-led exploration to several local optima on the performance landscape before coverging at the global optimum. The black dots denote the total motor output. B Initially, the contribution of the RL pathway P rl , in grey, drives a strong bias in the motor output P , in black. As the contribution of the motor pathway Pmtr, in brown, reaches the global optimum, the BG contribution recedes. C The range of BG-led exploration, around the motor pathway, shrinks with development. Each dot represents the bias driven by the RL contribution P rl at a given trial. D Performance evaluation, in purple, fluctuates over the coarse of learning on both daily (inset) and weekly timescales. The daily BG consolidation trace w k , in black, determines the shift on the following day.

Fig. 4 .

 4 Fig. 4. Robustness and benchmarking. The grey bar and the percentage value next to it denote the success rate, i.e. the proportion of simulations with a high terminal performance (above 0.6). The purple dots represent the terminal performance of individual simulations, i.e., the mean performance evaluation obtained in the last five days. The opacity of the dots denotes the number of simulations that received a similar terminal performance. Yellow crossed markers represent the median terminal performance in each scenarioo. A. Performance of the dual pathway architecture on the Gaussian-based performance landscapes with 'low', 'medium' and 'high' number of local optima, at 20% initial RL noise. B. Performance of the dual pathway architecture on the syrinx-based performance landscapes at different noise levels and comparison with benchmarks.

The scripts are available at https://doi.org/10.5281/zenodo.6407128.