A Manchon

N Ryzhanova

A Vedyayev

M Chschiev

B Dieny

Description of current-driven torques in magnetic tunnel junctions

Keywords: Spintronics, Tunnel Magnetoresistance, Magnetic Tunnel Junction, Spin Transfer Torque PACS numbers: 85.75.Mm, 72.25.-b, 72.25.Ba, 85.75.Dd

A free electron description of spin-dependent tranport in magnetic tunnel junctions with non collinear magnetizations is presented. We investigate the origin of transverse spin density in tunnelling transport and the quantum interferences which give rise to oscillatory torques on the local magnetization. Spin transfer torque is also analyzed and an important bias asymmetry is found as well as a damped oscillatory behaviour. Furthermore, we investigate the influence of the s-d exchange coupling on torque in particular in the case of half-metallic MTJ in which the spin transfer torque is due to interfacial spin-dependent reflections.

Introduction

The theoretical demonstration of spin transfer torque in metallic spin valves (SV) ten years ago [1] gave a new breath to giant magnetoresistance related studies [2], promising exciting new applications in non-volatile memories technology [3] and radio-frequency oscillators [4]. A number of fundamental studies in metallic spin valves revealed the different properties of spin torque and led to a deep understanding of current-induced magnetization dynamics [5,6,7,8,9]. Particularly, several theoretical studies described the structure of the torque in metallic magnetic multilayers and showed the important role of averaging due to quantum interference, spin diffusion and spin accumulation [10,11,12].

Since the first experimental evidence of spin-dependent tunnelling [13], magnetic tunnel junctions (MTJs) have attracted much attention because of the possibility to obtain large tunnelling magnetoresistance (TMR) at room temperature [14]. The possibility to use MTJs as sensing elements in magnetoresistive heads, as non-volatile memory elements or in reprogrammable logic gates has also stimulated a lot of technological developments aiming at the optimization of MTJs' transport properties and their implementation in silicon-based circuitry [3,15]. Because of these applications, MTJs have been intensively studied and the role of interfaces [16], barrier [17], disorder [18] and impurities [19] have been addressed in many publications [21]. The recent achievement of current-induced magnetic excitations and reversal in MTJs [20] has renewed the already very important interest of the scientific community in MTJs.

The observation of spin transfer torque in low RA (resistance area product) MTJ using amorphous [20] or cristalline barriers [15,22] opened new questions about the transport mechanism in MTJ with non collinear magnetizations orientations. As a matter of fact, whereas the current-perpendicular-to-plane (CPP) transport in SV is mostly diffusive and governed by spin accumulation and relaxation phenomena [11,12], spin transport in magnetic tunnel junctions is mainly ballistic and governed by the coupling between spin-dependent interfacial densities of states: all the potential drop occurs within the tunnel barrier. J. C. Slonczewski first proposed a free electron model of spin transport in a MTJ with an amorphous barrier [24], deriving TMR, spin transfer torque (STT) and zero bias interlayer exchange coupling (IEC). This first model only considered electrons at Fermi energy, neglecting all non-linear tunnel behaviour (consequently, current-induced IEC was found to be zero). More recently, the author proposed a more general model based on Bardeen's Transfer Matrix (BTM) method [25].

Another group presented at tight-binding model (TB) of a MTJ, giving more realistic band structures than the usual free electron model [26,27]. These studies showed that spin torque should present an important bias asymmetry and the dissipative part of IEC (also called current-induced effective field) should be of the same order of magnitude than STT with a quadratic dependence on the bias voltage [26]. Finally, we note that in the same spirit as Ref. [28], Levy and Fert studied the role of hot electrons-induced magnons on STT in MTJ [29]. In recent experiments, the important relative amplitude of current-induced effective field compared to the spin torque term has been verified [30,31,32] but the role of magnons is still under investigation (in the first experiment the current-induced magnetization reversal occured while the TMR was quenched by magnons emissions [20]). These specific features show that tunnelling transport has a strong influence on spin transfer torque characteristics.

We recently presented [12] a description of spin-dependent transport in a MTJ treated in a free-electron assumption, based on Keldysh non-equilibrium technique [33] applied to a MTJs with an amorphous barrier (such as AlO x ). This method is close to Ref. [24], although more general since we consider the contribution of all electrons lying below the Fermi energy. In the present article, we focus on the anatomy of spin transfer torque in such a MTJ, paying attention to the origin of the specific characteristics of this torque in the particular case of MTJ. The paper is organized as follows. In section 2, we give a reminder of the origin of spin transfer torque, and the way to calculate it.

In section 3, the formulation of spin-dependent currents and torques in non-equilibrium Green function formalism (Keldysh formalism) is developed. Section 4 presents the results of the model and describe the anatomy of spin torque in a magnetic tunnel junction, underlying the role of tunnelling process.

Current-induced torques

All along this paper, we consider the s-d model in which s-electrons are itinerant and d-electrons are localized and give rise to the local magnetization of the ferromagnet. We also assume that the d local moments remain stationnary. This model applies to the electron structures of ferromagnetic electrodes whose compositions lie on the negative slope side of the Slater-Néel-Pauling curve [34] (Ni, Co, NiFe, CoFe). No spin flip is taken into account. In a magnetic tunnel junction composed of two semi-infinite ferromagnets separated by a tunnel barrier (see Fig. 1), majority spins and minority spins refer to the electron spin projection in the left ferromagnet respectively parallel or antiparallel to the local magnetization. In this framework, the motion of s-like electrons in a ferromagnet is represented by the non-relativistic Hamiltonian including s-d coupling:

H = p 2 2m + U(r) + J sd -→ σ . -→ S d (1)
where the first and second terms are the kinetic and potential energies, the third term is the s-d exchange energy, -→ S d being a unit vector collinear to the local magnetization due to the localized d-electrons, J sd the s-d exchange constant and -→ σ is the vector of Pauli matrices in spin space. After some algebra [12,35], it is possible to derive the equation of continuity of the spin density :

d dt -→ s (r, t) = 2 { d dt Ψ * -→ σ Ψ + Ψ * -→ σ d dt Ψ} (2)
where Ψ = Ψ ↑ , Ψ ↓ is an arbitrary 2-dimension Hartree-Fock wavefunction. The two dimensions refer to majority (↑)and minority (↓) spin projections of the Hartree-Fock wavefunction. Here, -→ s (r, t) refers to the local spin density (namely the local out-ofequilibrium magnetization due to the itinerant polarized electrons):

-→ s (r, t) = Ψ * (r, t) 2 -→ σ Ψ (r, t) (3)
Defining

J s = - 2 2m ℑ{Ψ * (r, t) -→ σ ⊗ ∇ r Ψ (r, t)} (4)
where J s refers to the spin-current density, we obtain, in steady states:

∇ r J s (r, t) = 2J sd -→ S d × -→ s (r, t) (5)
Eq. 5 implies that the spatial transfer of spin momentum from the itinerant selectrons to the localized d-electrons (left-hand side of Eq. 5) is equivalent to a torque exerted by the transverse spin density on the local magnetization (right-hand side of Eq. 5). This equivalence has been demonstrated by Kalitsov et al. [27] in magnetic tunnel junctions using Keldysh formalism and TB description.

In the following, we calculate spin transfer torque from the torque exerted by the transverse spin density on the local magnetization. The spirit of our calculation is depicted in the top panel of Fig. 1. The out-of-equilibrium magnetic tunnel junction is modelled by a "conductor" (in the sense that the tunnel barrier is not infinite) linking two magnetic reservoirs (F L and F R ) with non collinear magnetizations and with different chemical potentials µ L and µ R [START_REF] Datta | Electronic Transport in Mesoscopic Systems[END_REF] arrows) and right electrodes (leftward arrows). In low bias limit (µ L ≈ µ R ), the charge transport can be approximately determined by the electrons originated only from the left electrode at the Fermi energy.

(µ L > µ R ). A bias voltage V = (µ L -µ R )/
In our case (middle panel of Fig. 1), the magnetic tunnel junction is composed of two ferromagnetic layers, F L and F R (made of the same material, for simplicity), respectively connected to the left and right reservoirs and separated by an amorphous tunnel barrier. The x-axis is perpendicular to the plane of the layers and the magnetization of F L is oriented following z:

-→ M L = M L -→ z . The magnetization -→ M R
of F R is in the (x,z) plane and tilted from -→ M L by an angle θ. In this configuration, the spin density in a ferromagnetic layer possesses three components : -→ m = (m x , m y , m z ).

In F L (we obtain the same results considering F R ), the transverse components are m x =< σ x > and m y =< σ y >, where σ i are the Pauli spin matrices and ¡¿ denotes averaging over orbital states and spin states, i.e. averaging over electrons energy E, transverse momentum -→ κ and spin states. Thus, the transverse spin density exerts a torque -→ T on the background magnetization -→ M L following two axes:

-→ T = J sd µ B -→ M L × -→ m = J sd µ B m x -→ M L × -→ M R -m y -→ M L × -→ M L × -→ M R (6)
One should introduce the previous formula in the usual Landau-Lifshitz-Gilbert equation to describe the modified dynamics of the magnetization -→ M L :

d -→ M L dt = α -→ M L × d -→ M L dt -γ -→ M L × --→ H ef f + -→ T ( 7
)
where α is the Gilbert damping, γ is the gyromagnetic ratio and µ B is the Bohr magnetron. The two terms in the right hand side of Eq. 6 stand for two types of torques:

-m y -→ M L × -→ M L × -→ M R
is the usual STT term (also called in-plane or parallel torque [26]) whereas m x -→ M L × -→ M R is the current-driven interlayer exchange coupling (also called field-like torque, out-of-plane or perpendicular torque [26]). The former vanishes at zero bias, whereas the latter exists even without current [24,26,27]. An explanation of the physical nature and origin of these two terms will be given in section 4. The transverse spin density in the left layer is then given by < σ + >=< σ x + iσ y > :

m x + im y =< σ + >=< Ψ * ↑ Ψ * ↓ 0 2 0 0 Ψ ↑ Ψ ↓ >= 2 < Ψ * ↑ Ψ ↓ > (8)
In other words, STT is given by the imaginary part of < σ + >, while IEC is given by its real part. One can understand the product < Ψ * ↑ Ψ ↓ > as a correlation function between the two projections of the spin of the impinging electron. In ballistic regime, an electron impinging on a ferromagnet with a spin polarization tilted from the background magnetization will precess around this magnetization [10,27]. Locally, its two projections ↑ and ↓ following the quantization axis (defined by the background magnetization) will be non-zero. Then, the electron will contribute locally to the transverse spin density m x and m y . If the electron spin is fully polarized parallel or antiparallel to this magnetization, no precession will occur and its contribution to the transverse spin density will be zero.

We remind that we defined majority (minority) states as the spin projection parallel (antiparallel) to the magnetization of the left electrode. Then, < Ψ * ↑ Ψ ↓ > will be the fraction of electrons whose spin is following x (real part) and y (imaginary part) in spin space.

Formulation of currents and torques

Keldysh Green functions

As explained previously, in Keldysh out-of-equilibrium formalism [33,[START_REF] Datta | Electronic Transport in Mesoscopic Systems[END_REF], any physical quantity should be calculated considering the contribution of the electrons originated from the left reservoir and from the right reservoir (top panel of Fig. 1). Then, an out-of-equilibrium Green function G(r, t, r ′ , t ′ ) (or Keldysh Green function) is defined as a superposition of these two contributions:

G (r, t, r ′ , t ′ ) = f L Ψ L (r, t) Ψ * L (r ′ , t ′ ) + f R Ψ R (r, t) Ψ * R (r ′ , t ′ ) ( 9
)
where Ψ L(R) (r, t) are the electron wavefunctions originated from the left (right) reservoir at the location r and time t and f L(R) are the Fermi distribution functions in the left and right reservoirs.

Thus, the Schrödinger equation of the magnetic tunnel junction is:

HΨ = p 2 2m + U + J sd -→ σ . -→ S d Ψ ↑ Ψ ↓ = E Ψ ↑ Ψ ↓ (10)
where -→ σ the vector in Pauli matrices space : -→ σ = (σ x , σ y , σ z ) T , E is the electron energy, U is the spin-independent potential along the junction:

J sd -→ σ . -→ S d = J sd σ z and U = E F for x < x 1 J sd -→ σ . -→ S d = 0 and U(x) = U 0 - x -x 1 x 2 -x 1 eV for x 1 < x < x 2 J sd -→ σ . -→ S d = J sd (σ z cos θ + σ x sin θ) and U = E F -eV for x > x 2
We consider that the potential drop occurs essentially within the barrier and we apply a low bias voltage compared to the barrier height (V << U/e). This allows to use WKB approximation to determine the wavefunctions inside the barrier. Furthermore, the free electron approximation implies parabolic dispersion laws which also restricts our study to low bias voltage.

To describe the spin-dependent transport within the MTJ, we define the wavefunctions Ψ σ ′ (σ) i (r, ǫ), where ǫ = E F -E and E is the tunnelling electron energy.

|Ψ σ ′ (σ) i (r, ǫ)| 2
is the probability that an electron originated from electrode i, at the energy ǫ, initially in spin state σ, possesses a spin projection σ ′ at the location r. For example, an electron initially in majority state, originated from F L , is described by six wavefunctions along the structure:

Ψ ↑(↑) L = 1 √ k 1 e ik 1 x + b L e -ik 1 x Ψ ↓(↑) L = d L e -ik 2 x
in the left electrode

F L (x < x 1 ), Ψ ↑(↑) L = a ′ L E(x 1 , x) + b ′ L E(x, x 1 ) q(x) Ψ ↓(↑) L = c ′ L E(x 1 , x) + d ′ L E(x, x 1 ) q(x)
where E(x i , x j ) = exp

x j x i q(x)dx , in the tunnel barrier (x 1 < x < x 2 ), Ψ ↑(↑) L = a ′′ L e ik 3 x + b ′′ L e ik 4 x Ψ ↓(↑) L = c ′′ L e ik 3 x + d ′′ L e ik 4 x
in the right electrode F R (x > x 2 ). k 1(2) and k 3(4) are the wavevectors for majority (minority) spin projection in the left and right electrodes, whereas q(x) is the spinindependent wavevector inside the tunnel barrier. Connecting the wavefunctions and their derivatives at the interfaces, we obtain the 24 wavefunctions (two spin projections and two reservoirs). These wavefunctions are given in Appendix.

In the 2-dimensionnal Hartree-Fock representation, spin-dependent current and spin density are defined using the out-of-equilibrium lesser Keldysh Green function:

G -+ σσ ′ (r, r ′ ) = dǫ f L Ψ σ ′ (↑) * L (r ′ ) Ψ σ(↑) L (r) + Ψ σ ′ (↓) * L (r ′ ) Ψ σ(↓) L (r) +f R Ψ σ ′ (↑) * R (r ′ ) Ψ σ(↑) R (r) + Ψ σ ′ (↓) * R (r ′ ) Ψ σ(↓) R (r) (11)
where f L = f 0 (ǫ) and f R = f 0 (ǫ + eV ), and f 0 (ǫ) is the Fermi distribution at 0 K. For conveniency, we use the mixed-coordinate system (x, -→ κ ), where -→ κ is the momentum parallel to the plane and x is the coordinate perpendicular to the plane.

With r = (x, -→ ρ ), we get:

G -+ σσ ′ (r, r ′ ) = a 0 2 √ π 2 √ π/a 0 0 e i -→ κ -→ ρ - -→ ρ ′ G -+ σσ ′ (x, x ′ )d -→ κ ( 12
)
where a 0 is the lattice parameter of the electrodes [37]. Spin transfer torque (STT, T || ) and interlayer exchange coupling (IEC, T ⊥ ) can now be determined from Eq. 8, whereas spin-dependent electrical current densities are calculated from the usual local definition:

T ⊥ + iT || = J sd µ B < σ + >= 2 J sd µ B a 3 0 (2π) 2 G -+ ↑↓ (x, x, ǫ)κdκdǫ (13)
m z = J sd µ B a 3 0 (2π) 2 G -+ ↑↑ (x, x, ǫ) -G -+ ↓↓ (x, x, ǫ) κdκdǫ (14)
J ↑(↓) = e 4πm e ∂ ∂x - ∂ ∂x ′ G -+ ↑↑(↓↓) (x, x ′ , ǫ)| x=x ′ κdκdǫ (15)
J = J ↑ + J ↓ (16)
G -+ ↑↑ (x, x, ǫ) and G -+ ↓↓ (x, x, ǫ) are the energy-resolved local density-of-states (LDOS) for up-and down-spins respectively, whereas G -+ ↑↑ (x, x, ǫ)dǫ and G -+ ↓↓ (x, x, ǫ)dǫ give the number of up-and down-electrons at the location x along the structure.

Calculation of spin transfer torque

As demonstrated in Eq. 5, it is possible to calculate spin transfer torque from the divergency of spin current density or from the spin density itself. We now demonstrate that this relation holds in our model. Spin current densities and spin density are defined as [10]:

m x = Ψ ↓ Ψ * ↑ + Ψ ↑ Ψ * ↓ ( 17
)
m y = -i Ψ ↓ Ψ * ↑ -Ψ ↑ Ψ * ↓ (18) J s x = - 2 2m ℑ{Ψ * ↑ ∂Ψ ↓ ∂x + Ψ * ↓ ∂Ψ ↑ ∂x } (19)
J s y = - 2 2m ℜ{Ψ * ↓ ∂Ψ ↑ ∂x -Ψ * ↑ ∂Ψ ↓ ∂x } (20)
We evaluate these quantities for electrons originating from the left reservoir in the left electrode (x < x 1 ). The equations are given in Appendix. The spin densities for majority (↑) and minority (↓) electrons are:

m ↑ x = 8q 1 q 2 (k 3 -k 4 ) sin θ e -i(k 1 +k 2 )(x-x 1 ) -r * ↑ 1 e i(k 1 -k 2 )(x-x 1 ) den + c.c. ( 21
) m ↓ x = 8q 1 q 2 (k 3 -k 4 ) sin θ e -i(k 1 +k 2 )(x-x 1 ) -r * ↓ 1 e -i(k 1 -k 2 )(x-x 1 ) den + c.c. ( 22
)
m ↑ y = -8iq 1 q 2 (k 3 -k 4 ) sin θ e -i(k 1 +k 2 )(x-x 1 ) -r * ↑ 1 e i(k 1 -k 2 )(x-x 1 ) den -c.c. ( 23
)
m ↓ y = -8iq 1 q 2 (k 3 -k 4 ) sin θ e -i(k 1 +k 2 )(x-x 1 ) -r * ↓ 1 e -i(k 1 -k 2 )(x-x 1 ) den -c.c. ( 24
)
Finally we obtain:

m x = m ↑ x + m ↓ x = 8q 1 q 2 (k 3 -k 4 ) sin θ (25) × 2 e -i(k 1 +k 2 )(x-x 1 ) den + c.c. - r * ↑ 1 den + r ↓ 1 den * e i(k 1 -k 2 )(x-x 1 ) + c.c. m y = m ↑ y + m ↓ y = -8iq 1 q 2 (k 3 -k 4 ) sin θ r ↑ 1 den * + r * ↓ 1 den e -i(k 1 -k 2 )(x-x 1 ) -c.c. (26)
By the same way, we evaluate the spin current density for majority and minority spins:

J s↑ x = -8q 1 q 2 2 2m (k 3 -k 4 ) sin θ -ik 2 e -i(k 1 +k 2 )(x-x 1 ) den + ik 2 r * ↑ 1 e i(k 1 -k 2 )(x-x 1 ) den +ik 1 e i(k 1 +k 2 )(x-x 1 ) den * + ik 1 r ↑ 1 e -i(k 1 -k 2 )(x-x 1 )
den * (27)

J s↓ x = -8q 1 q 2 2 2m (k 3 -k 4 ) sin θ ik 2 e i(k 1 +k 2 )(x-x 1 ) den * + ik 2 r ↓ 1 e i(k 1 -k 2 )(x-x 1 ) den * -ik 1 e -i(k 1 +k 2 )(x-x 1 ) den + ik 1 r * ↓ 1 e -i(k 1 -k 2 )(x-x 1 ) den ( 28
)
J s↑ y = -8q 1 q 2 2 2m (k 3 -k 4 ) sin θ ik 2 e -i(k 1 +k 2 )(x-x 1 ) den -ik 2 r * ↑ 1 e i(k 1 -k 2 )(x-x 1 ) den +ik 1 e i(k 1 +k 2 )(x-x 1 ) den * + ik 1 r ↑ 1 e -i(k 1 -k 2 )(x-x 1 )
den * (29)

J s↓ y = -8q 1 q 2 2 2m (k 3 -k 4 ) sin θ -ik 2 e i(k 1 +k 2 )(x-x 1 ) den * -ik 2 r ↓ 1 e i(k 1 -k 2 )(x-x 1 ) den * -ik 1 e -i(k 1 +k 2 )(x-x 1 ) den + ik 1 r * ↓ 1 e -i(k 1 -k 2 )(x-x 1 ) den (30)
Taking the imaginary (and real) part of the right-hand-side of the above equations, we obtain, similarly to Eqs. 25 and 26:

J s x = -8q 1 q 2 2 2m (k 3 -k 4 ) (k 1 + k 2 ) 2 sin θ r * ↑ 1 den + r ↓ 1 den * e i(k 1 -k 2 )(x-x 1 ) + c.c. (31)
J s y = -i8q 1 q 2 2 2m (k 3 -k 4 ) sin θ e i(k 1 +k 2 )(x-x 1 ) den * -c.c. (k 1 -k 2 ) - (k 1 + k 2 ) 2 r * ↑ 1 den + r ↓ 1 den * e i(k 1 -k 2 )(x-x 1 ) -c.c. (32)
The divergency then gives:

∂J s x ∂x = -8iq 1 q 2 2 2m (k 3 -k 4 ) sin θ k 2 1 -k 2 2 2 r * ↑ 1 den + r ↓ 1 den * e i(k 1 -k 2 )(x-x 1 ) -c.c. ( 33
)
∂J s y ∂x = 8q 1 q 2 2 2m (k 3 -k 4 ) sin θ k 2 1 -k 2 2 2 2 e -i(k 1 +k 2 )(x-x 1 ) den * + c.c. - r * ↑ 1 den + r ↓ 1 den * e i(k 1 -k 2 )(x-x 1 ) + c.c. ( 34
)
Setting

J sd = 2 2m k 2 1 -k 2 2
2 , Eqs. 25, 26, 33 and 34 give the following relation:

∂J s x ∂x = -J sd m y ∂J s y ∂x = J sd m x ⇒ ∇J s = J sd M × m (35)
Then, the relation 5 can be derived analytically in the free electron approach. This relation does not depend on the particular description adopted (Tight-binding or free electron approximation) but emerges from the definition of the considered Hamiltonian itself.

Results and discussion

To illustrate the above calculation, we use material parameters adapted to the case of Co/Al 2 O 3 /Co structure: the Fermi wavevectors for majority and minority spins are respectively

k ↑ F = 1.1 Å-1 , k ↓ F = 0.6 Å-1 , the barrier height is U -E F = 1.
6 eV, the effective electron mass within the insulator is m ef f =0.4 [38] and the barrier thickness is d=0.6 nm. These parameters have been choosen to fit the experimental I-V characteristics of the magnetic tunnel junctions studied in Ref. [31]. In all this section, the magnetizations form an angle θ=90 • between them. We will justify this choice in the following.

Anatomy of spin transport

Although spin-dependent tunnelling is a well known process, the description we give here is of great importance to understand the specific characteristics of spin transfer torques in tunnelling transport. In this part, we will consider the linear approximation in which the bias voltage V b is low enough so that the current is due to Fermi electrons injected from the left electrode. When the electrodes magnetizations are non collinear, the electrons are no more described as pure spin states, but as a mixing between majority and minority states. For example, let us consider one electron from the left reservoir, initially in majority spin state, impinging on the right electrode (see Fig. 2 -step 1).

The first reflection (step 2) at the F L /I interface do not introduce any mixing since the insulator is non magnetic. However, when (the transmitted part of) this electron is reflected or transmitted by the second interface I/F R (step 3), the resulting state in the right electrode is a mixing between majority and minority states since the quantization axis in the right electrode is different from the quantization axis in the left electrode.

Then, the transmitted spin is reoriented and precesses (step 4) around the magnetization of the right electrode. Furthermore, the reflected electron (step 5) is also in a mixed spin state and precesses around the left electrode magnetization. In other words, after transport through the barrier, the electron spin is reflected/transmitted with an angle. This reorientation gives rise to spin transfer torque.

Note that there is not reason why the electron spin should remain in the plane of the electrodes magnetizations. We will see that after the reorientation, the electron spin possesses three components in spin space (and so two transverse components).

Tunnelling transport

We are first interested in the spin-dependent reflectivity R σ(σ ′ ) and transmittivity T σ(σ ′ ) for electrons at the Fermi energy from the left electrode.

Let us consider an electron initially in majority spin state (↑). Its wavefunction will be described by a plane wave in the left electrode :

e ik 1 (x-x 1 )
√ k 1 The mixing between majority and minority spin states can be expressed through mixing reflectivities R ↑↑ and R ↓↑ and transmitivities T ↑↑ et T ↑↓ , so that:

R ↑↑ + R ↓↑ + T ↑↑ + T ↓↑ = 1
where:

R ↑↑ = |r ↑ 1 | 2 (36)
R ↓↑ = 16| q 1 q 2 (k 3 -k 4 ) m 2 ef f den sin θ| 2 (37)
T ↑↑ = |Ψ ↑(↑) L dΨ * ↑(↑) L dx -Ψ * ↑(↑) L dΨ ↑(↑) L dx | (38)
T ↓↑ = |Ψ ↓(↑) L dΨ * ↓(↑) L dx -Ψ * ↓(↑) L dΨ ↓(↑) L dx | (39)
Ψ

σ(σ ′ ) L
is evaluated in the right electrode and given in Appendix. By the same way, we can define the transmittivity and reflectivity of an electron initially in minority spin state. Fig. 3 displays the κ-dependence of R and T (we omit the superscripts for simplicity), where κ is the wavevector component in the plane of the layers.

More than 97% of the majority and minority spins are reflected conserving their spin projection, whereas less than 3% are transmitted without spin flip. This reflectivity (transmittivity) reaches a minimum (maximum) at perpendicular incidence and increases (decreases) quickly with κ. Note that T ↑↓ and T ↓↑ are equal due to the particular configuration of the electrodes magnetizations (θ=90 • ). Thus, after interaction with the barrier, only a very small part of the spin is flipped (the flipped spins have to tunnel through the barrier twice) : less than 2.7 × 10 -3 % of the reflected wave has flipped its initial spin. 1.6×10 -3 % of the electron spins initially in minority states reverses its spin during reflection.

Thus only a very small part of the injected polarized wave is flipped during the tunneling process. However, this does not mean that spin transfer torque is small in MTJ; as a matter of fact, only coherent mixing states will contribute to transverse spin density, generating spin transfer torque.

Finally, we note that only electrons close to the perpendicular incidence contribute significantly to the current. This has important consequences on the impact of quantum interferences on spin transfer.

Spin density and spin transfer torque

In the linear regime under consideration, the three components of spin density in the left electrode can be described as follows:

m ↑ +L = Ψ ↑(↑) L Ψ * ↓(↑) L = 8q 1 q 2 (k 3 -k 4 ) sin θ m 2 ef f den * e i(k 1 +k 2 )(x-x 1 ) -r ↑ 1 e -i(k 1 -k 2 )(x-x 1 ) (40)
m ↓ +L = Ψ ↑(↓) L Ψ * ↓(↓) L = 8q 1 q 2 (k 3 -k 4 ) sin θ m 2 ef f den e -i(k 1 +k 2 )(x-x 1 ) -r ↓ * 1 e -i(k 1 -k 2 )(x-x 1 ) (41)
m ↑ zL = Ψ ↑(↑) L Ψ * ↑(↑) L -Ψ ↓(↑) L Ψ * ↓(↑) L (42) = (1 + |r ↑ 1 | 2 ) k 1 - 8q 1 q 2 √ k 1 (k 3 -k 4 ) sin θ m 2 ef f den 2 - 1 k 1 r * ↑ 1 e 2ik 1 (x-x 1 ) + r ↑ 1 e -2ik 1 (x-x 1 ) m ↓ zL = Ψ ↑(↓) L Ψ * ↓(↑) L -Ψ ↓(↓) L Ψ * ↓(↓) L (43)
= -

(1 + |r ↓ 1 | 2 ) k 2 + 8q 1 q 2 √ k 2 (k 3 -k 4 ) sin θ m 2 ef f den 2 + 1 k 2 r * ↓ 1 e 2ik 2 (x-x 1 ) + r ↓ 1 e -2ik 2 (x-x 1 )
Observing m

↑(↓)
+L in Eq. 40, one can distinguish two components: the first one is proportional to e ±i(k 1 +k 2 )(x-x 1 ) , and due to the interference between the incident wave with majority (resp. minority) spin and the reflected wave with minority (resp. majority) spin; the second one is proportional to e -i(k 1 -k 2 )(x-x 1 ) and due to the interference between the reflected waves with majority and minority spins. We note that the first components of m ↑ +L and m ↓ +L are complex conjugated so that their sum is real. Then, the interference between the incident wave with majority spin and the reflected wave with minority spin does not contribute to STT but only to IEC. STT is then generated by the coherent interferences between reflected electrons with opposite spin projection (∝ e -i(k 1 -k 2 )(x-x 1 ) ).

Concerning m zL , it is composed of one component proportionnal to e 2ik 1 (x-x 1 ) , one component proportionnal to e 2ik 2 (x-x 1 ) and one constant as a function of x. The two formers are due to the interference between wavefunctions in the same spin projection but with opposite propagation direction while the latter is due to interference between wavefunctions in the same spin projection and the same propagation direction. Note that the conservative part of IEC is only proprotionnal to e -i(k 1 +k 2 )(x-x 1 ) . But at non zero bias, the dissipative part of IEC is proportionnal to both e -i(k 1 +k 2 )(x-x 1 ) and e -i(k 1 -k 2 )(x-x 1 ) . Then, at non zero bias, the electrons will not precess circularly around the background magnetization, but will present a more complex structure.

Following the previous discussion about spin reorientation (see Fig. 2), it is possible to deduce the angles at which the electron spin is reflected by the barrier. We define the azimuthal angle azimuthal η and the polar angle φ as indicated in the insert of Fig. 5:

η = arctan m c y m c x (44) φ = arccos m c z m c2 x + m c2 y + m c2 z ( 45
)
In definition of the vector -→ m c , we only considered the coherent interferences between plane wave propagating in opposite direction (∝ e ±i(k 1 -k 2 ) and constant component of m z ), as discussed above. Fig. 5 shows the κ-dependence of these angles at the interface F L /I (x=-3 Å) for an electron spin initially in majority state and for different barrier thicknesses (top panel) and heights (bottom panel). The azimuthal angle η varies between -64 • to +77 • while the polar angle φ remains very small (less than 0.2 • ), which means that the electron spin stays very close to the quantization axis, as discussed above.

At κ = 0.6 Å-1 (corresponding to k ↓ F ), η = 0 which indicates that the effective spin density lies in the plane of the magnetizations -→ M L , -→ M R . Finally, the polar angle does not vary with the distance, which means that the reflected electron spin precesses around O z with a small angle φ. A "Bulk" spin transfer only occurs under the interferences of all the reflected electrons.

The strong dependence of η as a function of the in-plane wavevector κ, together with the dominant contribution of nearly perpendicularly incident electrons (see Fig. 3), implies that the effective electron spin, resulting from the averaging over all the incident electrons, possesses an important out-of-plane component. In other words, the effect of the spin-dependent tunneling is to strongly enhance the dissipative IEC component of the spin torque, compared to metallic spin valves. As a matter of fact, in SV the whole Fermi surface contributes to the spin transport so that the effective angle η is very small [10]: the dissipative IEC is thus negligible.

Note that increasing the thickness of the barrier only weak influence on η and strongly decreases the amplitude of φ (the mixing reflection decreases since the barrier thickness increases, then reducing the transverse spin density). Furthermore, when increasing the barrier height, both amplitudes of the angles φ and η decreases near the perpendicular incidence. These results are consistent with the reduction of spin mixing when decreasing the barrier transmittivity. during the reflection process) increases with J sd so that -→ m c gets closer to the plane of the magnetizations.

Spin Transfer Torques

We now take into account all the electrons in the calculations (from the left and the right electrodes). Fig. 7 shows STT and IEC as a function of the angle θ between the electrodes magnetizations, at V b = 0 and V b = 0.1 V. It clearly appears that IEC and STT are proportionnal to sin θ (the deviation from sin θ is minor than 10 -4 ).

This dependence is strongly different from what was predicted in metallic spin valves [ 11,12,39] and has been attributed [25] to the single-electron nature of tunneling.

As a matter of fact, because of the important height of the tunnel barrier, all the potential drop occurs inside the insulator and spin accumulation (i.e. the feedback of the current-induced longitudinal spin density on the spin current) is negligible. In this case, the angular dependence of torque is determined by the angular dependence of the transmition matrix, as discussed in Ref. [25] and yields a sine shape. In the following, we will estimate the spin density for θ = π/2.

Note that, at zero bias, interlayer exchange coupling is still non-zero, contrary to spin transfer torque. The conservative part of IEC (IEC at zero bias) comes from the contribution of electrons located under the Fermi level. At zero bias, the currents from left and right electrodes are equal, but the electron propagation still corresponds to the scheme shown in Fig. 2: the mixing between majority and minority states induces a transverse component in the spin density.

m ↑ +R = Ψ ↑(↑) R Ψ * ↓(↑) R (46) = 8 √ q 1 q 2 m 2 ef f den 2 sin θ 2 k 3 Ψ(q 1 , k 2 , q 2 , k 4 )Ψ * (q 1 , k 1 , q 2 , k 4 )e -i(k 1 -k 2 )(x-x 1 ) (47) m ↓ +R = Ψ ↑(↓) R Ψ * ↓(↓) R (48) = 8 √ q 1 q 2 m 2 ef f den 2 sin θ 2 k 4 Ψ(q 1 , k 2 , q 2 , k 3 )Ψ * (q 1 , k 1 , q 2 , k 3 )e -i(k 1 -k 2 )(x-x 1 ) (49)
It is now possible to show that in the general expression of transverse spin density

m + = m ↑ +L + m ↓ +L + m ↑ +R + m ↓ +R the terms proportional to e -i(k 1 -k 2 )(x-x 1 )
vanish at zero bias and m + reduces to terms proportional to e ±i(k 1 +k 2 )(x-x 1 ) . Furthermore, these last terms only give a real component since, as discussed above, the majority and minority components of m y compensate each other. Consequently, at zero bias, only the conservative part of interlayer exchange coupling exists, due to the interference between incident and reflected electrons with opposite spin projection. But when the bias voltage is non zero, the transport becomes asymetric and the terms proportional to e -i(k 1 -k 2 )(x-x 1 ) do not compensate each other anymore and lead to two periods of oscillations as shown in Fig. 8(a).

Spin transfer torque, proportional to m y , only exits at non zero bias and possesses only one period of oscillation T 1 (see Fig. 8(b)). It is worthy to note that the transverse components of spin density is damped by 50% within the first nanometers, and that the amplitude of IEC is of the same order than STT. This decay lenght is very large compared to previous theoretical predictions [10,39] and experimental investigations on SV [40]. As a matter of fact, the ballistic assumption holds for distance smaller than the mean free path (≈ 5 nm in Co). In realistic devices, spin diffusion should increase the decay of STT and IEC. Another source of this difference compared to metallic SV is the fact that we consider perfect interfaces and no in the barrier. First principle studies of realistic Co/Cu interfaces [41] showed that the mismatch of the electronic structure at the interface strongly reduces the transverse component of spin density. In MTJ, the non spherical nature of the spin-dependent Fermi surface [42,43] should also dramatically alter the transverse spin density. This could also explain the fact that the amplitude of spin torque is two orders of magnitude higher than in experiments.

Another characteristic specific to MTJ is that in our calculation we find that dissipative IEC is of the same order of magnitude than STT. This is coherent with the theoretical results of Theodonnis et al. [26] as well as with the experimental studies of Petit et al. [31]. This can be attributed to the high κ-selection due to the tunneling transport. We previously found that the contribution to torque strongly decrease with κ (see Figs. 3 and5) so that only electrons with small κ strongly contribute to spin torque.

In this case, the averaging of torques (and specifically IEC) will be less destructive than in metallic spin valves where all the Fermi surface is involved in the quantum interferences.

Finally, Fig. 9 shows the out-of-equilibrium longitudinal spin density ∆n defined as ∆n

↑(↓) = n ↑(↓) (V b = 0.1) -n ↑(↓) (V b = 0)
. ∆n oscillates and asymptotically reaches a non zero value. This means that when the bias voltage is turned on, a non equilibrium spin accumulation builds up. However, this effective spin accumulation is very small (∆n ↑ -∆n ↓ ≈ 10 -7 electron/atom) and cannot influence spin current building. Then, neglecting the role of longitudinal spin accumulation (spin density) in MTJ is justified.

Bias dependence

The bias dependence of STT and IEC in MTJ also presents strong differences with SV.

We first calculate the total spin torque exerted on the left electrode. Following the definition of Ref. [1] and Ref. [26], the total torque is: parameter J sd . Our results are consistent with Theodonnis et al. [26]. The dissipative IEC is quadratic whereas STT is a combination between linear and quadratic bias dependence. In Ref. [26], the authors proposed a general formula, derived from Slonczewski circuit theory [39], linking total spin transfer torque with interfacial spin current densities [26]:

-→ T total = -∞ x 1 -∇J s dx = J s (x 1 ) (50)
T || = J s AP -J s P 2 (51)
where J s AP (P ) are interfacial spin current densities when the electrodes magnetizations are antiparallel and parallel respectively (see the definition in Ref. [26]).

The authors claimed that this relation should hold for any electronic structure, so any transport description. As a matter of fact, the top inset of Fig. 10(b) shows STT calculated using Eq. 50 (solid line) and Eq. 51 (symbols). It shows very good agreement between the two members of Eq. 51.

Experimental studies by Cornell's group [32,44] demonstrated a linear variation of spin transfer torque as a function of the applied bias voltage. This linear variation is also usually assumed in interpreting excitations studies [30,31]. Moreover, the very recent article of Sankey et al. [32] seems to confirm the fact that the dissipative exchange coupling is quadratic as a function of the bias voltage. Finally, note that a change of sign of spin transfer torque at high positive bias voltage is expected, consistently with

Ref. [26]. The STT change of sign should be observed in MTJ with low enough barrier height and high breakdown voltage (MgO seems a good candidate). Nevertheless, more technological development is needed to fabricate such junctions.

Eq. 50 assumes that all the transverse spin density has been absorbed in the free layer. However, in very thin free layer, one can expected that transverse spin density is not fully absorbed when leaving the free layer. In this case, one should consider that the free layer is finite. Fig. 11 displays the bias dependence of IEC and STT for different integration depths t (namely, different layer thicknesses):

-→ T partial = x 1 -t x 1 -∇J s dx = J s (x 1 ) -J s (x 1 -t) (52)
The dependence can change drastically and IEC can even change its sign (note that STT keeps its general shape). These dependences are strongly affected by the tunnel barrier characteristics and one should to be careful in the analysis of bias dependence.

From weak ferromagnetic to half-metallic tunnel junction

To conclude this article, we study the dependence of the total spin transfer torque and interlayer exchange coupling as a function of the energy of the bottom of the minority electrons conduction band ǫ ↓ , as indicated in Fig. 11. This energy is defined from the Fermi energy as:

ǫ ↓ = E F -E ↓ c = - 2 k ↓2 F 2m (53)
where E ↓ c is the absolute energy of the bottom of the conduction band. In the present study, we vary ǫ ↓ , keeping ǫ ↑ and E F constant. When ǫ ↓ is close to ǫ ↑ , k ↑ F ≈ k ↓ F , the metallic electrodes loose their ferromagnetic nature. For ǫ ↓ ≈ 0, the Fermi wavevector for minority electrons becomes smaller and the current polarization is strongly enhanced.

In this case, we expect an important spin transfer torque. When ǫ ↓ > 0, k ↓ F becomes imaginary and the electrodes behave like a tunnel barrier for minority spins. Increasing ǫ ↓ increases the evanescent decay of minority wavefunctions in the electrodes. Then, the product < Ψ * ↑ Ψ ↓ > still exists so that spin torque is non zero and decrease exponentially from the interface. Fig. 12 shows the amplitude of total STT and current-induced IEC in the three different regimes: weak ferromagnetic electrodes (WFM), strong ferromagnetic electrodes (SFM) and half-metallic electrodes (HM). As expected, in ferromagnetic regime, STT and dissipative IEC increase until ǫ ↓ = 0 (vertical line). When ǫ ↓ becomes positive, the bottom of the conduction band of minority electrons lies above the Fermi level: no minority electrons can propagate because only evanescent states exist near the interfaces for this spin projection. However, STT and dissipative IEC do not vanish but reach a plateau which slowly decreases to zero when increasing J sd (not shown).

To understand this behaviour, we calculated the spatial dependence of the transverse spin density in the free layer. Fig. 13 shows the transverse spin density in a usual ferromagnet, ǫ ↓ = -1.37 eV (which corresponds to J sd = 1.62 eV), as a function of the distance from the interface in the left electrode. The oscillation possesses the same characterisics than discussed above and we observe that the transverse spin density is damped far from the interface. When decreasing ǫ ↓ , the interfacial spin density increases, due to strong spin filtering at the interface (strong spin-dependent selection), as shown on Fig. 14. But when ǫ ↓ changes its sign, only majority electrons can propagate and the transverse spin density is (see Eqs. [21][22][23][24]:

m ↑ x = 16q 1 q 2 sin θ ℜ{(k 3 -k 4 ) e -i(k 1 +k 2 )(x-x 1 ) -r * ↑ 1 e i(k 1 -k 2 )(x-x 1 ) den } (54) m ↑ y = -16q 1 q 2 sin θ ℑ{(k 3 -k 4 ) e -i(k 1 +k 2 )(x-x 1 ) -r * ↑ 1 e i(k 1 -k 2 )(x-x 1 ) den } (55)
Considering Fermi electrons at perpendicular indidence, very small bias voltage (eV ≈ 0) and imaginary minority electron spin wavevector, k 2(4) = ik, we obtain straightforwardly:

m ↑ x = 16q 1 q 2 e k(x-x 1 ) sin θ ℜ{(k 3 -ik) e -ik 1 (x-x 1 ) -r * ↑ 1 e ik 1 (x-x 1 ) den } (56) m ↑ y = -16q 1 q 2 e k(x-x 1 ) sin θ ℑ{(k 3 -ik) e -ik 1 (x-x 1 ) -r * ↑ 1 e ik 1 (x-x 1 ) den } (57)
The transverse spin density is a product between oscillating function of k 1 and exponentially decaying function of k. Fig. 15 shows the spatial evolution of the transverse spin density in the case of a half-metallic tunnel junction. All the oscillations are damped very quickly so that the only important contribution to torque comes from the interface. Contrary to usual MTJ (where both bulk averaging due to spatial interferences and interfacial spin reorientation contribute to spin torque), in a strong half-metallic tunnel junction all the torque comes from spin reorientation due to spindependent reflection. In this last case, the contribution of the spatial averaging between all the impinging electrons (κ-summation) is reduced compared to interfacial spin transfer.

Conclusion

A free-electron s-d model has been proposed to analyze spin transfer effects in magnetic tunnel junctions with amorphous barrier and non collinear electrode magnetizations.

We first studied the anatomy of spin transport in such MTJ, showing that only a small part of the current undergoes spin-flipping due to the non collinearity of the electrode magnetizations. This corresponds to only a small deviation of the reflected spin from the local magnetization. Nevertheless, we showed that this small amount of precessing spin gives rise to an important transverse spin density leading to spin torque.

We also showed that the tunnel barrier acts like an incidence filter which increases the contribution of the electrons impinging with angle close to the perpendicular incidence. This κ-selection is at the origin of an important IEC, contrary to what is observed experimentally in metallic spin valves. The ballistic transport dominating the tunnel transport in MTJ is expected to induce large oscillations of STT and IEC as a function of the distance from the interface. If the oscillation period is large compared to the exchange length and one will observe a twist of the magnetization in the thickness of the layer. Otherwise, if the oscillation period is short compared to the exchange length, one will observe the torque integrated over the layer thickness.

The bias dependence of spin transfer torque shows a strong asymmetry and a change of sign at positive bias voltage. This results is coherent with tight-binding calculations [26]. However, we saw that this model is strongly limited to small bias voltage because of the simplicity of the adopted band structure.

Finally, we analyzed STT and IEC when varying the s-d exchange coupling and we demonstated that the torque still exists in MTJ composed of half-metallic electrodes, due to spin-dependent reflections. However, for infinite half-metallic MTJ (for infinite s-d coupling), it is shown that STT and dissipative IEC vanishes to zero. Furthermore, several numerical studies have shown that, even in amorphous barrier, the interfaces composition and specially the presence of interfacial oxygen have a very deep influence on the spin polarization and thus on TMR and STT [42]. The recent development of MgO-based MTJ in spin transfer studies reduced the interest in amorphous barriers. However, amorphous barriers have the ability to present a simple physical framework which can constitute a basis to understand spin transfer in MTJ. Nevertheless, because of its more complex band structure and spin-filtering effect associated with the symmetry of wavefunctions, microscopic analysis of spin transfer in MgO-based MTJ would present exciting fundamental characteristics even on spin transfer effects [43].

Appendix: Spin-dependent wave-functions in a clean MTJ

In this appendix, we give the analytical formulae for the spin dependent wavefunctions in the MTJ. Some functions which will be used in the description of this wavefunctions are first defined:

q 2 0 = 2m 2 (U -E F ) q(x) = q 2 0 - 2m 2 x -x 1 x 2 -x 1 eV -ǫ + κ 2
q 1 = q(x 1 )

q 2 = q(x 2 ) k 1(2) = k ↑(↓) F 2 - 2m 2 ǫ -κ 2 k 3(4) = k ↑(↓) F 2 - 2m 2 (ǫ -eV ) -κ 2 E(x i , x j ) = exp x j x i q(x)dx E n = E(x 1 , x 2 )
where E F is the Fermi energy, U is the height of the barrier, V is the bias voltage and ǫ = E F -E, E being the energy of tunnelling electron. We define: Ψ(q 1 , k i , q 2 , k j ) = E n (q 1 -ik i )(q 2 -ik j ) -E -1 n (q 1 + ik i )(q 2 + ik j ) φ(q 1 , k i , q 2 , k j ) = E n (q 1 + ik i )(q 2 -ik j ) -E -1 n (q 1 -ik i )(q 2 + ik j ) den = Ψ(q 1 , k 1 , q 2 , k 3 )Ψ(q 1 , k 2 , q 2 , k 4 )(1 + cos θ) + Ψ(q 1 , k 2 , q 2 , k 3 )Ψ(q 1 , k 1 , q 2 , k 4 )(1 -cos θ)

r ↑ 1 = 1 den
[φ(q 1 , k 1 , q 2 , k 3 )Ψ(q 1 , k 2 , q 2 , k 4 )(1 + cos θ) + φ(q 1 , k 1 , q 2 , k 4 )Ψ(q 1 , k 2 , q 2 , k 3 )(1 -cos θ)]

r ↑ 3 = 1 den
[φ(q 2 , k 3 , q 1 , k 1 )Ψ(q 1 , k 2 , q 2 , k 4 )(1 + cos θ) + φ(q 2 , k 3 , q 1 , k 2 )Ψ(q 1 , k 1 , q 2 , k 4 )(1 -cos θ)]

Electrons initially in the left electrode have the following wavefunctions along the structure :

Ψ ↑(↑) L (-∞ < x < x 1 ) = 1 √ k 1 e ik 1 (x-x 1 ) -r ↑ 1 e -ik 1 (x-x 1 ) Ψ ↓(↑) L (-∞ < x < x 1 ) = 8q 1 q 2 √ k 1 (k 3 -k 4 ) sin θ den e -ik 2 (x-x 1 )
Ψ ↑(↑)

L (x 1 < x < x 2 ) = -2i den k 1 q 1 q(x) (E (x 2 , x) [Ψ (q 1 , k 2 , q 2 , k 4 ) (q 2 + ik 3 ) (1 + cos θ) +Ψ (q 1 , k 2 , q 2 , k 3 ) (q 2 + ik 4 ) (1 -cos θ)] +E -1 (x 2 , x) [Ψ (q 1 , k 2 , q 2 , k 4 ) (q 2 -ik 3 ) (1 + cos θ) +Ψ (q 1 , k 2 , q 2 , k 3 ) (q 2 -ik 4 ) (1 -cos θ)])

Ψ ↓(↑) L (x 1 < x < x 2 ) = 4q 2 den k 1 q 1 q(x) (k 3 -k 4 ) sin θ E (x 1 , x) (q 1 -ik 2 ) + E -1 (x 1 , x) (q 1 + ik 2 ) Ψ ↑(↑)
L (x 2 < x < ∞) = -4i den k 1 q 1 q 2 e ik 3 (x-x 2 ) Ψ (q 1 , k 2 , q 2 , k 4 ) (1 + cos θ) +e ik 4 (x-x 2 ) Ψ (q 1 , k 2 , q 2 , k 3 ) (1 -cos θ)

Ψ ↓(↑)
L (x 2 < x < ∞) = -4i den k 1 q 1 q 2 e ik 3 (x-x 2 ) Ψ (q 1 , k 2 , q 2 , k 4 ) -e ik 4 (x-x 2 ) Ψ (q 1 , k 2 , q 2 , k 3 ) sin θ

Electrons initially in the right electrode have the following wavefunctions along the structure :

Ψ ↑(↑)
R (-∞ < x < x 1 ) = -8i den q 1 q 2 k 3 Ψ(q 1 , k 2 , q 2 , k 4 ) cos θ 2 e -ik 1 (x-x 1 ) Ψ ↓(↑) R (-∞ < x < x 1 ) = -8i den q 1 q 2 k 3 Ψ(q 1 , k 1 , q 2 , k 4 ) sin θ 2 e -ik 2 (x-x 1 ) Ψ ↑(↑) R (x 1 < x < x 2 ) = -4i den k 3 q 2 q(x) Ψ(q 1 , k 2 , q 2 , k 4 ) cos θ 2 E(x 1 , x)(q 1 -ik 1 ) + E -1 (x 1 , x)(q 1 + ik 1 )

1 Figure 1 .

 11 Figure 1. Schematics of the magnetic tunnel junction with non collinear magnetizations orientation. Top panel: spin-dependent out-of-equilibrium transport in a conductor linking two reservoirs F L and F R (whose electrochemical potentials are respectively µ L and µ R ) with non collinear magnetizations orientations. The solid arrows represent the majority spins and the dotted arrows represent the minority spins. Middle panel: MTJ with non collinear magnetization orientations. Bottom panel: Corresponding energy profile of the MTJ. In free-electron approximation, the local density of states are parabolic for majority (solid line) and minority (dotted line) electrons with a splitting between the two spin subbands equals to the exchange interaction J sd .

Figure 2 .

 2 Figure 2. Schematics of the principle of spin transport in a magnetic trilayer with non collinear electrodes magnetizations. Step 1: the electron spin is polarized along the magnetization of the left electrode. Step 2: After the first reflection/transmission by F L /I interface the reflected and transmitted parts remain in a pure spin state. Step 3: The reflection/transmission by the second interface I/F R reorientes the electron spin. Step 4 and 5: The transmitted and reflected spins precess around the local magnetization.

Figure 3 .

 3 Figure 3. Reflectivity (top panel) and transmittivity (bottom panel) as a function of κ. The solid and dotted lines represent the spin conserving reflectivity and transmittivity for initially majority and minority spins respectively (left axes); the dashed and dotted-dashed lines represent the mixing reflectivity and transmittivity for initially majority and minority spins respectively (right axes). The applied bias voltage is V b = 0.1 V and θ=90 • .

Fig. 4

 4 Fig. 4 displays the details of the spin density components m x , m y et m z (described in Eq. 40) in the left electrode as a function of x, when V b = 0.1 V. m x possesses a quite complex behaviour with two periods of oscillation (the dashed lines show the enveloppe of the curve), whereas m y is reduced to a single oscillation (The oscillation period k 1 +k 2 vanishes when suming the contribution of majority and minority spins); m z oscillates around mean values represented by horizontal dashed lines.

Figure 4 .

 4 Figure 4. Projections of spin density due to Fermi electrons in perpendicular incidence from the left electrode, as a function of the distance from the interface. Top panel: m x component of spin density (solid line); the dashed lines are the enveloppes of the curve. Middle panel: m y component of spin density. Bottom panel: m z component of spin density due to initially majority (solid line) and minority (dotted line) spin projection; the dashed lines are the mean values of the oscillations. The applied bias voltage is V b = 0.1 V. The vertical line on the right is the interface between the left electrode and the tunnel barrier.

Fig. 6 Figure 5 .

 65 Fig.6shows the dependence of the angles as a function of the s-d exchange constant J sd for perpendicular incidence κ = 0. Quite intuitively, the precession angle φ increases with J sd whereas the initial azimuthal angle η decreases in absolute value with J sd . The spin-filtering effect (the selection between majority and minority spin

Figure 6 .

 6 Figure 6. Reflection angles as a function of the s-d exchange constant, for a Fermi electron initially in majority spin state. The parameters are the same as in Fig. 5.

Fig. 8 Figure 7 .

 87 Fig.8displays the two components of transverse spin density as a function of the location in the left electrode. The interference process between polarized electrons yields a damped oscillation of IEC as presented in Fig.8(a). We can distinguish two periods of oscillationT 1 = 2π/ k ↑ F -k ↓ F and T 2 = 2π/ k ↑ F + k ↓F whereas at zero bias, only T 2 appears (see inset of Fig.8(a)). This can be easily understood by considering electrons from left and right electrodes. Transverse spin density in the left electrode due to electrons from the right electrode is:

Figure 8 .

 8 Figure 8. Total spin density as a function of the location in the left electrode: a) Current-induced interlayer exchange coupling -inset: Interlayer exchange coupling at zero bias voltage; b) Spin transfer torque. These quantities are calculated at V b = 0.1 V.

Figure 9 .

 9 Figure 9. Out-of-equilibrium longitudinal spin density along the magnetic tunnel junction for majority (solid line) and minority (dotted line) electron spin projections. The bias voltage is V b = 0.1 V.

Figure 10 .

 10 Figure 10. Bias dependence of interlayer exchange coupling (a) and spin transfer torque (b) for different values of s-d coupling: J sd = 0.38 eV (open circles), J sd = 0.76 eV (filled circles), J sd = 1.62 eV (open squares), J sd = 2.29 eV (open triangles),J sd = 2.97 eV (filled squares). Top inset: Bias dependence of STT for J sd = 1.62 eV; the solid line was calculated following the usual way and the symbols were calculated using Eq. 51.

Fig. 10 displays

 10 Fig. 10 displays the total interlayer exchange coupling (a) and spin transfer torque (b) as a function of the applied bias voltage, for different values of the s-d exchange

Figure 11 .

 11 Figure 11. Bias dependence of interlayer exchange coupling (a) and spin transfer torque (b) for J sd = 1.62 eV and different values integration depth: t = 0 Å(open squares), t = 4 Å(filled triangles), t = 10 Å(filled circles), t = ∞ Å(open circles).

Figure 12 .

 12 Figure 12. Spin transfer torque (solid line) and dissipative interlayer exchange coupling (dotted line) as a function of s-d exchange coupling. The vertical line shows the limit between ferromagnetic (weak ferromagnetic -WFM-and strong ferromagnetic -SFM-) regime and half-metallic regime.

Figure 13 .

 13 Figure 13. Transfer spin density (black line) as a function of the distance in the left ferromagnetic electrode in a usual ferromagnetic regime. We set ǫ ↓ = -1.37 eV and V b = 0.1 V.

Figure 14 .

 14 Figure 14. Transfer spin density (black line) as a function of the distance in the left ferromagnetic electrode in a strong ferromagnetic regime. We set ǫ ↓ = -0.38 eV and V b = 0.1 V.

Figure 15 .

 15 Figure 15. Transfer spin density (black line) as a function of the distance in the left ferromagnetic electrode in half-metallic regime. We set ǫ ↓ = 19 eV and V b = 0.1 V.

Acknowledgments

The work and results reported in this publication were obtained with research funding from RTN SPINSWITCH and from Russian Fundings for Basic Research 07-02-00918a. The views expressed are solely those of the authors, and the other Contractors and/or the European Community cannot be held for any use that may be made of the information contained herein.


Bhargav Teja Nallapu
email: bhargav.teja-nallapu@inria.fr

Frédéric Alexandre

A survival task for the design and the assessment of an autonomous agent

Keywords: computational neuroscience, autonomy, simulation platform

Learning to survive in a complex environment is a more relevant task for the design of intelligent autonomous machines than complex problem solving. To establish this statement, we emphasize that autonomy requires emotional-and motivational-like characteristics that are much more straightforward to define in the context of survival tasks than in problem solving which is classically considered with intelligent agents. We also propose that using a simulation platform is a good preliminary step before the design of real machines because it allows to consider another fundamental challenge, related to the association of these emotional and motivational characteristics to higher cognitive functions. These considerations are illustrated by current simulations that we are carrying out with a bio-inspired neuronal model of the cerebral architecture of primates.

Introduction

This paper presents a neuromimetic approach to emulate intelligent behavior in autonomous machines. Lately, striking progresses have been reported in the field of Artificial Intelligence, on hierarchical neuronal models such as Deep Networks applied to specific tasks like playing Go or answering general knowledge questions, setting the focus on dedicated complex problem solving tasks. However, in reference to some characteristics of intelligence in animals, autonomy raises other challenges at least as important as abstract intelligence does. Firstly, in reference to the emotional domain, an autonomous agent should be able to set its own goals by anticipating outcomes and to acquire its preferences by learning. Similarly, motivation should also be considered, to energize potential behaviors according to a level of need felt or to the anticipated cost of an action [START_REF] Dickinson | The Role of Learning in the Operation of Motivational Systems[END_REF]. As these characteristics have been often associated to bodily aspects [START_REF] Cardinal | Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal cortex[END_REF], these challenges will require further studies about the body and its physiology. Secondly, it has been shown that these embodied aspects play an important role in the most complex aspects of cognition, including decision-making, planning or social skills [START_REF] Damasio | The Somatic Marker Hypothesis and the Possible Functions of the Prefrontal Cortex[END_REF], but the underlying neuronal mechanisms are to be understood in more details.

The above mentioned challenges correspond to two very ambitious research programs of different nature, that are difficult to carry out in tandem, although based on common ground. One issue is to modify the material characteristics of machines to allow them to reflect certain sensory and motor characteristics associated to emotions and motivations, as we observe in the case of animals. The other issue is to understand how these characteristics influence the most elaborate forms of reasoning, particularly allowing to associate subjective and incomparable quantities such as the level of need for a resource and its cost or its social impact.

We present here some elements of a modeling work that bridges the challenges between the above mentioned directions. This work considers the possibility to define an autonomous agent by adapting a well established neuronal model proposed in [START_REF] Hazy | Banishing the homunculus: making working memory work[END_REF], mimicking the function of major loops in the brain of primates and primarily involving two cerebral structures, the cortex and the basal ganglia. The so-called cortico-basal loops have been associated to the generic function of decision making and action selection and several such loops have been experimentally observed [START_REF] Alexander | Parallel organization of functionally segregated circuits linking basal ganglia and cortex[END_REF]. Based on these observations, we have implemented four loops. Two loops are called sensorimotor and are fed with external spatial and visual information and select actions of orientation and object reaching. The other two loops are called limbic and are fed with internal information carrying emotional and motivational characteristics and make decisions about the need to satisfy and the object to select.

In addition to implementing each of the four loops and their interactions, a major question in our work was to assess the ability of the whole model to emulate an autonomous behavior, by defining what could stand for body and an environment to interact with this model of cerebral architecture and what could be the task to address, to quantify the behavior of the brain+body system, possibly under different dimensions.

In the next section, we propose that learning to survive in a complex environment is a more relevant task than complex problem solving. We define a survival task where the autonomous agent must monitor basic needs such as hunger or thirst to select the resource to be reached in its environment. In the subsequent section, we introduce a simulation platform that simulates the agent, the body as well as its environment, and allows us to control the complexity level of the environment and modify its properties. This also leads to the definition of the emotional and motivational characteristics of the agent and their encoding in the model. We propose that, even if this software platform brings a radical simplification compared to a hardware machine, it is an excellent first step towards building autonomous machines and addresses fundamental questions about embodiment. We then report for illustration the results of the first experiments that we have carried out and finally mention further work that could be considered.

Let us first state that, if there is a precise task that characterizes an autonomous agent, it is a task of survival. An autonomous agent has to be able to survive in an unknown environment, being aware of its vital variables and being able to keep those variables in acceptable bounds at any given moment and for the forthcoming moments. Consequently, it has to minimize uncertainty and avoid surprise to ensure that it accomplishes the goal of keeping variables in bound. This has been formalized in, e.g., [START_REF] Friston | Active inference and learning[END_REF] and is revisited differently here. A step further, biological systems often fail, e.g. have limited generalization capability or learning performances, and the utopian vision that they are "very general autonomous systems" is questionable: They may more "adapt" than "learn", i.e. change some parameters of some versatile behaviors, instead of "inventing" new behaviors.

To demonstrate this principle, we consider an agent in an environment that has 'objects' that are primarily given an incentive value -their relevance to the amount of hunger they satisfy or the amount of thirst they quench if these needs are considered. The objects also carry a preference value with respect to the agent, which could be retrieved from the agent's pre-existing knowledge (or being learned, this very important aspect will not be considered in the present study but see [START_REF] Carrere | A pavlovian model of the amygdala and its influence within the medial temporal lobe[END_REF] for previous work we made on pavlovian learning, known to contribute to preference learning). The two limbic loops mentioned above that we have implemented are respectively in charge of selecting the need to be considered and the preferred object. We will mention below how they are associated, through the simulation platform, to the internal perception of needs (for motivational aspects) and to the representation of preference as an anticipation of a certain level of pleasure (for emotional aspects).

At any given instant, it is possible that there are no objects perceived by the agent in the vicinity. In this case, the other two loops, the sensorimotor loops, are responsible for the orientation of the agent in space until objects are perceived. If any reveal pertinent to the task, the loops are also responsible for navigation, reaching and consumption of this object. We mention below how the spatial and visual aspects to be processed in the sensorimotor loops are extracted from the simulation platform.

For the moment in our description, the loops are presented independent to each other and each of them respectively carries out a specific function. It is of course necessary to consider some coordination between the loops to emulate a really intelligent and autonomous behavior and we will evoke below several strategies of coordination and how they can be implemented, particularly to give rise to a possibility of different starting points of the survival task. Before all these developments, we describe, in the following section, the generic algorithm implemented for each loop in the model and the Malmo platform we used to create an environment for the task.

In the brief description above, each loop has been assigned a very different function, but it is also noticeable to mention that each loop is dynamically computed with the same algorithm, since they are all part of the same circuitry linking the basal ganglia and the sensory and frontal cortical areas.

Firstly, sensory cues, corresponding to actual or desired sensations activate candidate actions in the frontal area. A primitive strategy is to trigger the action most often associated to these sensations. This corresponds to habits. Else, a selection process takes place to make a decision based on a deeper contextual analysis. This is attributed as one of the major roles of the basal ganglia.

The basal ganglia is often presented as a competitive system (Go-No Go in [START_REF] Hazy | Banishing the homunculus: making working memory work[END_REF]), based on several excitatory and inhibitory pathways respectively adding more weight to the selection and the inhibition of each candidate action until one reaches a threshold and is triggered. This dual evaluation is based on the estimated values of the expected sensory consequences of the action and other modulatory contextual information.

There have been computational accounts of such action selection in the basal ganglia using a two-arm bandit task [START_REF] Pasquereau | Shaping of motor responses by incentive values through the basal ganglia[END_REF], that go on to demonstrate how the outcomes of such actions are learned, thereby being able to choose the best rewarding one later [Redgrave et al., 1999,Topalidou and[START_REF] Topalidou | [END_REF]. The conflict between two almost equivalent options is resolved by an external noise driving one of them towards selection. Even without a complicated task in context, the same selection between two equivalent stimuli can be demonstrated in a single channel of neural populations. This kind of selection is achieved by implementing the above mentioned complementary pathways in the basal ganglia as neural ensembles with excitatory and inhibitory connections [START_REF] Leblois | Competition between feedback loops underlies normal and pathological dynamics in the basal ganglia[END_REF].

Each loop described in our model has a respective substrate of the basal ganglia involved for the local selection. Given the space constraints in the paper and the main emphasis being on the high level behaviour, we restrict the description of the dynamics within the basal ganglia in the equation 1 as in [START_REF] Leblois | Competition between feedback loops underlies normal and pathological dynamics in the basal ganglia[END_REF]. Assuming each unit represents an ensemble tuned towards a particular stimulus : I ext is the external input representing the salience of the stimulus, I s is the input to the unit from its connections (synaptic input) and τ is the decay time constant of the synaptic input and m is the output of the unit.

τ dm dt = -m + I s + I ext (1)
The synaptic input to a unit j, I j s , which is the input as a result of the connections from units of other structures (say i), depends on the connections weights (w ij ) between units i and j , as shown in the equation 2. We would like to emphasize that the dynamics given by the equations 1 and 2 are phenomenological and are not constrained to a specific neuronal architecture.

I j s = Σ i w ij * m i (2)
When an action is triggered, its expected sensory consequences are also activated to a specific desired level, representing the goal of the action. The action will be maintained until the expected sensory consequences (or other conditions for interruption, not developed here) are met.

In some cases, triggering the action is not sufficient to reach the goal (e.g. deciding to eat is not sufficient to get some food) and the desired activity can itself trigger new actions in other loops (e.g. finding some food). This process can recursively trigger other secondary goals, until some goal is immediately achieved, stopping the corresponding action. This generic algorithm is applied in parallel in the four loops. Among the limbic loops, one is responsible for selecting the need, based on the interoception of internal levels. We refer to this loop as Why loop as it corresponds to the motivation behind selecting the need. The other is responsible for selecting perceived items known to satisfy some needs, based on their levels of preference. This loop is referred to as What loop as it signifies the qualitative nature of each stimulus owing to its preference level. Among the sensorimotor loops, one is responsible for orienting the agent towards the selected target, which we call Where loop. The other, referred to as How loop, is responsible for moving to reach the target. The Why loop also receives an input from the sensori-motor loops, representing the cost of action involved in reaching a certain stimulus, to be able to compare with the appetitive gain the stimulus might offer. This can be simply implemented using negative connection weights w ij in equation 2.

In addition, this selection made locally in a specific loop is also modulated by the selection made in a different loop. For example, the activity strength of selection in What loop modulates the activities competing for selection in the Why loop. Similarly, the Why loop modulates the Where loop which in turn modulates the How loop. To keep the modulation simple and tractable, we implement a simple biasing factor b ij from unit i to j, as a function of the source activity a i as shown in equation 3.

b ij = a tan σ IJ π 4 i (3)
In equation 3, σ IJ is a bias strength parameter that is specific between two populations I and J. For instance, the bias that the preference-based choice might have on the need-based choice could be less than the bias that the overall limbic choice has on that of any of the sensorimotor loops. This kind of interactions between different cortico-basal ganglial loops in animals (including primates) is a question of wide interest in the field of neuroscience. Here in this work, we stick to a rather simple implementation for the ease of the demonstration of the principle. Therefore, it has to be noted that all the equations given above represent a generalized principle (lateral interactions, modulation etc.) between the neuronal structures and their intra-loop and inter loop dynamics.

Minecraft is a sandbox video game with a virtual 3D world that can be generated procedurally. With single or multiple agents in it, activities like exploration and resource gathering can be simulated. Malmo is an Artificial Intelligence experimentation platform on the top of Minecraft [START_REF] Johnson | The malmo platform for artificial intelligence experimentation[END_REF]. Malmo allows to incorporate various models of reinforcement learning, planning and collaborative and competitive strategies into the Minecraft game environment among the agents. Malmo has been used for various kinds of specific experimentations like learning to navigate in the Minecraft world [START_REF] Matiisen | Teacher-student curriculum learning[END_REF] and computational models of animals living in block world [Strannegård et al., 2017a]. In most of such cases, very specific feature of Malmo is used (either the 3D space in the environment or the block nature of the world or the agent to perform a task. We exploit, simultaneously, various features of Malmo like the agent's internal body attributes, external constraints like vision, and uncertainty that can be induced into the scenario owing to the vast environment and its contents. Furthermore, Malmo provides great convenience to connect our model of generic loops to the Minecraft world, since the model inevitably requires an elaborative environment to study and visualize the behavior in a survival task.

Malmo provides an agent, on which we have a full control on its actions, either through the tool or through our model. The agent by default has an attribute life that is affected by the external world (e.g, when in contact with fire or attacked by other agents). Malmo allows to construct a 3D world of the size of choice, in which the agent is present. We can procedurally place certain items in the world, in the vicinity of the agent or elsewhere. Items can be attributed respective reward values that the agent is able to gain when it collects them. The agent is allowed to execute finely controlled actions like turn, move, jump, etc. At any given time step, the framework provides the state of both the agent and the environment with respect to the agent. The state includes attributes like agent's life, its position and orientation in absolute coordinates with respect to the world, the items that are present in the chosen accessibility range around the agent and their positions and attributes.

For the survival task we have chosen, attributing bodily and other emotional and motivational characteristics to the agent forms a key aspect of our model. Besides the relevant features of Malmo we use, we augment these characteristics to the agent as well as to the items in the task. Agent has two vital variableshunger and thirst -which increase with time as well as with the level of efforts (meaning an action move or turn). Instead of a one dimensional reward, each item carries a motivational index that is relevant to the hunger or the thirst level it would satisfy, and a level of preference with respect to the agent, expressing its emotional value. Also, from a functional point of view, we adapted few aspects like visibility of the agent and the information about the positions (of items as well as the agent itself). For instance, as the model is desired to be bio-inspired, we consider few bodily, biological constraints to the agent and restrict the agent's field of vision to a plausible value (in our case, 120 • ). The information about the objects in the environment is also restricted depending on the distance from the agent within the field of vision. These adaptations were important to add certain biologically plausible restrictions to the task given that we would not model them in detail as they don't fall in the scope of our goals.

Illustrations

Figure 1 shows several moments from an episode in the task implemented in Malmo, primarily concerned with different questions each loop in the model addresses. In a basic scenario, the agent starts exploring the environment (figure 1.a) with a desired activation for a particular item that (known from previous experience) would satisfy the current major need (figure 1.a, inset). This is a result of the internal state processing in the Why loop. When the agent perceives multiple stimuli (figure 1.b), along with the appetitive relevance of each of the stimuli, the action costs, depending on their positions, are also provided (implemented as a negative signal from the sensory-motor loops). Furthermore, the choice on the pre-existing preferences towards the stimuli corresponding to the selected need is made in the What loop and it modulates the selection in the Why loop.

Once the decision has been made in the Why loop and the goal has been set, the execution of goal involves two steps in the two sensorimotor loops. Once a stimulus is chosen, the goal is to orient towards it. The Where loop is responsible for the agent to start turning towards it (see apple in figure ??.c) until the selected stimulus is in the sight of the agent. And finally, owing to the processing in the How loop, the agent moves to reach the stimulus that it has oriented towards and consumes it (an imaginary action which we equate to the agent reaching the item and updates the corresponding hunger and thirst values).

Sustaining the selection of goal until it is achieved is at the core of the processing in each loop. The Where loop, after choosing the orientation to turn, sustains the activity until the object is in sight. And the How loop sustains its activity from the point of orienting to the point where it has reached closer to the stimulus, to be able to consume it. This would now cascade back to the limbic loops which have been sustaining their selected responses. The Why loop, which has been active since selecting the current need, is sustained until the need levels are modified by the consumption of the stimulus. Similarly, the What loop, which has been sustaining activity since selecting a preferred stimulus, continues until verifying the consumed stimulus has the expected value.

Discussion

In the wake of the field of Artificial General Intelligence that advocates intelligent agents should be able to solve general problems, it is rather easy to classify survival of an autonomous agent as a general problem. We argue that survival of an autonomous agent, in fact, addresses a very specific question, that requires the agent to keep its vital variables within acceptable bounds in the current as well as a future, anticipated state. We emphasize the body, which completes a closed-loop of interaction for the agent with the environment, as the key to the process of survival. Consequently, we derive our basis from neuroscience of animal cognition and behavior to build a bio-inspired model that helps an artificial agent survive. The components of our model, representing the cortical and basal ganglia structures in the brain, are inspired to be close to the biological plausibility in their working dynamics. As mentioned earlier, understanding the interactions between these cortico basal-ganglial loops is an active ongoing work in the field of neuroscience and we, through the description and further development of our model, would like to be in a position of useful contribution to that work.

In the field of robotics, there have been accounts of motivation driven robotic systems and a few design frameworks [Strannegård et al., 2017b,Konidaris andBarto, 2006]. Most of them address the task of making a choice based on motivation. We derive inspiration from such formalisms and begin to build much more comprehensive scenarios involving the behaviour of agent, in a bio-inspired manner, in which action selection forms a key component. We remain aware that building such scenarios in a bio-inspired manner requires further more understanding of higher level neural mechanisms involved in animal foraging behavior [Kolling et al., 2012,Constantino andDaw, 2015]. However, we would like to emphasize that the fundamental basis of demonstrating comprehensive animal behaviours in artifical agent lies in the description of the underlying generic characteristics of the survival task. Therefore, in this work, we contain ourselves to the description of the basic task and required attributes, keeping the developments and performance studies to a much finer study elsewhere. In fact, using the neural structures already involved in certain loops, we account for context-specific preferences of the agent (in the What loop) and the action costs involved in choosing a stimulus (in the Why loop). In the context of the task that we target to demonstrate, given its complexity, it is considerably difficult to choose the right kind and number of attributes to be encoded in the model. Rightly so, we chose an experimentation platform that inspires our concrete choice of emotional and motivational characteristics of the model.

Different starting points characterize certain scenarios in which agent exhibits a distinct behavior, quite often, that belongs to a set of behaviors extensively studied in neuroscience related to primate cognition. The example scenario illustrated in the section Illustrations describes a very prominent animal behavior in neuroscience, called goal-directed behavior [START_REF] Balleine | The integrative function of the basal ganglia in instrumental conditioning[END_REF]. As a part of our ongoing work, we plan to study other key animal behaviors like stimulusdriven behavior where, even in the absence of internal need, the external stimuli in the environment drive the motivation of the animal. We also are interested in extending the model to explain much more interesting, opportunistic behaviors where the agent's chosen current behavior can be interrupted to switch to another, anticipating a higher satisfaction for a future need as opposed to a lower satisfaction of a current need. It is the emergence of such versatile behaviors that helps the animal survive in a dynamic external world. The platform we chose, Malmo, invariantly supports demonstrating various behaviors with no or minimal changes to itself but only from the changes in the state of the agent or the objects in the environment. Unlike other works that used Malmo to interface with Minecraft gaming environment, we utilize its features to have a finer control on the environment and to demonstrate the modulation between the cognitive behaviour and action execution in the environment.

Interestingly enough, understanding the dynamics between such different behaviors is still an open problem in computational neuroscience. That encourages us to strengthen the basis of our model more from biology, exploit the convenient visualization platform like Malmo to demonstrate the model to neuroscientists and gain their insights. After all, we, the intelligent primates, are a perfect example of an autonomous agent trying to survive, only in a real world.

Fig. 1 .

 1 Fig. 1. Snapshots at different stages in the task. (a) Internal needs monitored in the Why loop (inset). (b) Processing information about the stimuli in the What loop. (c) Orienting towards the selected stimulus using the Where loop. (d) Reaching the selected stimulus using the How loop, once oriented. Note: The different third person and first person views in (a) and (b) are only chosen to show the change in proximity of stimuli to the agent, but these views have no effect on the task. They can be switched while watching the task. Similarly, for (c) and (d)

Acknowledgements

We would like to acknowledge Thierry Viéville, INRIA Sophia Antipolis, France, for his collaboration.