Ramezani Amir
email: amir.ramezani.my@gmail.com

Dooraki

What Are Some of the Best Ways for a Machine to Learn?

Keywords: Artificial Intelligence, Supervised Learning, Reinforcement Learning, Unsupervised Learning, Machine Imagination, Machine Learning, Cognitive Development

Artificial intelligence recently had a great advancements caused by the emergence of new processing power and machine learning methods. Having said that, the learning capability of artificial intelligence is still at its infancy comparing to the learning capability of human and many animals. Many of the current artificial intelligence applications can only operate in a very orchestrated, specific environments with an extensive training set that exactly describes the conditions that will occur during execution time. Having that in mind, and considering the several existing machine learning methods this question rises that 'What are some of the best ways for a machine to learn?' Regarding the learning methods of human, Confucius' point of view is that they are by experience, imitation and reflection. This paper tries to explore and discuss regarding these three ways of learning and their implementations in machines by having a look at how they happen in minds.

Introduction

How minds work, or in another word how a human brain thinks, with the goal of implementing it in machines, is a long-term question in artificial intelligence. In the recent years, Artificial Intelligence algorithms have demonstrated outstanding progress and nowadays can be found in a variety of applications such as autonomous vehicles, computer games and health care automations. Having said that, the learning capability of artificial intelligence is still in its infancy comparing to the learning capability of humans. They can only operate in a very orchestrated, specific environments with an extensive training set that exactly describes the conditions that will occur during execution time. Systems of this kind are limited to the expertise and educated guesses of their human programmer; they lack the ability to learn in, or tune themselves to real-world environments autonomously or to employ learning in novel situations.

In order to create smarter machines, with an abstract similarity to human minds, design and development of complicated computational algorithms inspired by natural intelligence has already started. Among bioinspired algorithms those which simulated cognitive architectures are crucial where they are "the computational implementation of a cognitive model, and as such, constitute the substrate for all the cognitive functionalities in robots, like perception, attention, action selection, learning, reasoning, etc" [1]. As a result of these efforts, there are several successful methods of machine learning such as reinforcement learning, supervised learning and unsupervised learning where each group has its own branches: temporal difference and actor-critic methods for reinforcement learning, artificial neural network and its deep architectures for supervised learning and self-organizing and clustering methods for unsupervised learning. Saying that, this question rises that "What are some of the best ways for a machine to learn?" In this paper, I try to first address this question from a philosophical point of view and then I try to find the machine learning counterpart algorithms or in another word the way that a particular learning method is implemented in machines. Further, I try to discuss about the most important of learning method, considering this paper point of view, and the techniques and methods useful for its implementation in machines.

Learning Methodology

"By three methods we may learn wisdom: First, by reflection, which is the noblest; Second, by imitation, which is easiest; and third by experience, which is the bitterest"; Confucius. It is possible to find similar terminologies to each of the third ways of learning wisdom according to Confucius in machine learning field. "Third by experience, which is the bitterest": this is similar to 'the trial and error part of the reinforcement learning framework'. "Second, by imitation, which is easiest", this is similar to 'learning from demonstration or imitation learning in computer science'. Finally, "First, by reflection, which is the noblest", this important way of learning wisdom ends up creating new ideas and shaping new theories as a result of serious thought or consideration considering a person's ideological framework and according to what that he is learned through the Second and Thirds methods: imitation and experience.

By looking at the background of machine learning algorithms, one can see that supervised learning, unsupervised learning and reinforcement learning are studied and used by researchers for long time and in several cases, and they are still some of the important categories of learning in machines. Nonetheless, it is also possible to categorize them as following:

-Learning by Experience -Reinforcement Learning -Unsupervised Learning -Learning by Imitation -Learning From Demonstration • Pure Classification • Inverse Reinforcement Learning -Supervised Learning -Learning by Reflection -?

In the rest of this paper, I first explain briefly about the 'learning by experience' and 'learning by imitation' in machine learning field and later focus on the 'learning by reflection' and try to finds some references for it among machine learning algorithms.

Learning by Experience

Learning by trial and error (that is experience) is one of the foremost methods of learning specially for the sensorimotor development stage of intelligent creatures where they get master in moving their motors gradually by trying and fixing their errors. "By observing the development stages of an infant in terms of learning to articulate words, one can see that he first discovers how to control phonation, then focuses on vocal variations of unarticulated sounds, and finally automatically discovers and focuses on babbling with articulated protosyllables" [2]. In the following, it is explained how this method is implemented in machine learning community under different titles.

Reinforcement Learning

"RL algorithms address the problem of how a behaving agent can learn to approximate an optimal behavioral strategy, usually called a policy, while interacting directly with its environment" [START_REF] Barto | Intrinsic Motivation and Reinforcement Learning[END_REF]. Reinforcement learning family of algorithm are important in machine learning community for their high capability in decision making by learning from their own experiences. For an agent to be able to learn in an unknown environment, it should be able to autonomously interact with the environment. The framework of reinforcement learning is able to give the agent such an ability by letting it to take actions and measure the value of each action based on the reward it receives. More actions the agent takes, it gets more information about what is the best move in each state of the environment. Reinforcement learning [START_REF] Sutton | Introduction to Reinforcement Learning[END_REF] algorithms can be divided into several groups such as model-based and model-free, temporal difference methods, policy gradient methods and actor critic methods. Furthermore, they can be based on one or multi-step prediction [START_REF] Watkins | [END_REF]. An interesting trend in these algorithms is the idea of intrinsic moti-vation that makes the embodiment of qualities such as curiosity in machines possible.

The idea of 'learning by experience' in reinforcement learning frame work can be considered as a Markov Decision Process case. Where an agent interacts with its environment at discrete time steps and at each time step agent is located in a unique state according to its sensory input s t ∈ S. In each s t agent can take an action, a t ∈ A based on its current policy π(s t ). In the next time step, the state of agent changes to s (t+1) ∈ S based on its previous action and a reward R that it receives from the environment. (Figure 1).

Temporal Difference Approaches; Model-based and Modelless

Temporal difference algorithms are the reinforcement learning algorithms that try to calculate the value of a state or the values of each possible action in that state with attention to the reward received in each state. Furthermore, this information will be used by the agent to use in its policy in which it helps the agent to choose the best action in each state, and as a result the agent finally would be able to maximize its accumulated reward.

Q-Learning; Model-less Method

In many problems, there is no model of the environment. As a result, a model-free off-policy method such as q-learning [START_REF] Watkins | [END_REF] can be best to suite the scenario where it can be combined with experience replay method [START_REF] Lin | Reinforcement learning for robots using neural networks[END_REF] for designing more capable learning algorithms. A temporal difference reinforcement learning method can work for example based on an epsilon-greedy policy for action selection. The goal of this kind of policies is to achieve the maximum discounted reward over time and by using an iterative method of selecting the action with maximum value in each state-and updating this value using itself, the maximum action value in the next state and the reward it receives from the environment [START_REF] Sutton | Introduction to Reinforcement Learning[END_REF].

Dynamic Programing, Monte-Carlo and SARSA; A Modelbased Method

As it can be noticed by looking at the title of a modelbased reinforcement learning algorithm, its difference with model-less RL is in having a model of the environment. No matter, how the environment is, either 1D, 2D or 3D environment, or a mathematical formula of the dynamics of the system, a model based algorithm is able to calculate a future state of the environment with attention to the current state and the desired action. A very good example of a model-based algorithm is the work in [START_REF] Silver | [END_REF] where a machine is able to play Go game in a human level and win over many human players.

Function Approximator

Many reinforcement learning algorithms use the mentioned formula in order to find the optimal action value. However, there are two main problems, the lack of generalization and the problem of number of the states. In order to tackle these problems a function approximator an be used in order to find the optimal values of each action or state. As a result of using function approximators, Combination of supervised and unsupervised learning with reinforcement learning can be seen in several works. For example, combination of unsupervised learning (self-organizing map) and reinforcement learning can be seen in several works such as [8] where the input signals of the robot sensors are clustered using a dynamic variation of Kohonen self-organizing map algorithm [START_REF]Self-Organizing Maps[END_REF], and each cluster is defined to be an state for the reinforcement learning agent. As a result, the RL agent is able to control the robot movement based on pre-defined goals and rewards. Further, combination of reinforcement learning algorithm and supervised learning algorithms such as neural network can ends up creating a robust class of algorithm in terms of learning to achieve an optimal behavior similar to a human [START_REF] Mnih | [END_REF] in playing Atari game by defining a Deep Q-Network which is as a matter of fact combination q-learning and deep learning [11].

Actor Critic Approaches

Based on this view, reinforcement learning methods can be grouped under two categories. Actor-only methods and critic-only methods.

-"Actor-only methods work with a parameterized family of policies. The gradient of the performance, with respect to the actor policies, is directly estimated by simulation, and the parameters are updated in a direction of improvement." [12] -"Critic-only methods rely exclusively on value function approximation and aim at learning an approximate solution to the bellman equation, which will the hopefully prescribe a near-optimal policy." [12] Actor-critic methods [13], try to mix the important properties of actor-only and critic-only methods. The actor's policy parameters will be updated with the goal of improving the performance using the value function learned by critic through approximation architecture.

"The word actor and critic are synonyms for the policy and action-value function, respectively" [14]. One of the fresh important works in this area is the deep deterministic policy gradient [15] where it is an actorcritic network that is consist of two multilayer or deep artificial neural network based actor and critic. This algorithm could successfully solve the problem of continuous action control in some games and classic control problems.

Policy Gradient Methods

As mentioned in the previous section, actor-only methods are those in which the policy is parameterized and can be optimized directly. "The benefit of a parameterized policy is that a spectrum of continuous actions can be generated, but the optimization methods used (typically called policy gradient methods) suffer from high variance in the estimate of the gradient, leading to slow learning [12][16] [17][18] [19]." [14]

Hierarchical Reinforcement Learning Framework

Considering human behavior it "has long been recognized to display hierarchical structure: actions fit together into subtasks, which cohere into extended goaldirected activities" [20]. "Hierarchical reinforcement learning (HRL) decomposes a reinforcement learning problem into a hierarchy of sub problems or subtasks such that higher-level parent-task invoke lower-level child task as if they were primitive actions. A decomposition may have multiple levels of hierarchy" [START_REF] Hengst | Hierarchical Reinforcement Learning[END_REF]. This "decomposition into subproblems has many advantages. First, policies learned in subproblems can be shared (reused) for multiple parent tasks. Second, the value functions learned in subproblems can be shared, so when the subproblem is reused in a new task, learning of the overall value function for the new task is accelerated. Third, if state abstractions can be applied, then the overall value function can be represented compactly as the sum of separate terms that each depends on only a subset of the state variables. This more compact representation of the value function will require less data to learn, and hence, learning will be faster." [START_REF] Dietterich | [END_REF] 3.2 Unsupervised Learning Unsupervised learning method cannot be categorized under learning that happens by imitation since there is no output for the input to be imitated by machine, and it is hard to categorize it under learning that happens by experience perhaps because it does not have an attached reward to classify that particular experience as good or bad. However, this paper argues that receiving Fig. 2 Learning from Demonstration framework, Adapted from [START_REF] Argall | [END_REF] reward happens through many time steps where each time step can be considered as a new experience for our machine, that is each new sample has a subtle reward (loss) that tries to move a particular input toward a particular class. Thus it can be concluded that it is under the category of 'learning by experience'.

Learning by Imitation

The learning capability of human kind develops throughout his life by getting master in 'learning by imitation' (e.g. learning how to write a word from the first grade teacher), a wide spectrum of what we learn after our early ages comes from this type of learning.

Learning From Demonstration (Apprenticeship Learning)

Learning from Demonstration is a natural and intuitive machine learning approach where an agent that called apprentice tries to learn a behavior from demonstration of another agent called the expert. "We simply show the robot how to achieve a task. This has the immediate advantage of requiring no (or very little) specialized skill or training, and makes use of a human demonstrator's existing procedural knowledge useful to identify which control program to acquire" [24]. A very good example for learning from demonstration, is how to fly an airplane, considering all the different possibilities exist for using the buttons and handles in the cabin of an airplane, the best way to learn how to fly is by observing an expert. Perhaps it is just impossible to try to learn to fly an airplane simply by random pushing of buttons. Looking at the literatures there are at least two methods to deal with this problem of Apprenticeship Learning (AL); Pure Classification and Inverse Reinforcement Learning (IRL)

Pure Classification

Pure classification, for instance, the large margin method [START_REF] Ratliff | Proceedings of the 2007 7th IEEE-RAS International Conference on Humanoid Robots[END_REF] has several advantages and also some disadvantages that is explained in the following; It is "easy to implement, fast, no need to resolve MDPs, no need to do a choice of features thanks to boosting techniques. However, they do not take into account the structure of the MDP because the temporal structure of the expert trajectories is not used in a pure classification method. To tackle this drawback, the authors of [START_REF] Melo | Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 6322 LNAI[END_REF] use a kernel-based approach to encapsulate the structure of the MDP into the classifier, which needs the calculation of the MDP metrics and thus the knowledge of the whole dynamics." [START_REF] Piot | Proceedings of the 2014 International Conference on Autonomous Agents and Multi-agent Systems (International Foundation for Autonomous Agents and Multiagent Systems[END_REF]

Inverse Reinforcement Learning

Inverse Reinforcement Learning (IRL), introduced in [START_REF] Russell | Proceedings of the 11th Annual Conference on Computational Learning Theory (COLT)[END_REF] and formalized in [START_REF] Ng | Proceedings of the Seventeenth International Conference on Machine Learning[END_REF], is related to Apprenticeship Learning (AL) family of algorithm. However, IRL searches to find a reward where this reward should be able to explain the expert behavior and not the expert policy. "The key idea behind IRL is that the reward may be the most succinct hypothesis explaining the expert behavior. Some algorithms [START_REF] Abbeel | Proceedings of the 21st International Conference on Machine Learning (ICML)[END_REF][31] [32] use IRL as an intermediary step to find a policy but other algorithms [33] [34] are "pure" IRL algorithms and output a reward. This reward must then be optimized via a direct reinforcement learning algorithm." [START_REF] Piot | Proceedings of the 2014 International Conference on Autonomous Agents and Multi-agent Systems (International Foundation for Autonomous Agents and Multiagent Systems[END_REF] 4.4 Supervised Learning This famous method of machine learning helps the machine to learn to imitate that if a particular input received what should be the particular output. As a result, in this paper it is considered to be under the category of 'learning by imitation'.

Learning by Reflection

If we describe 'learning by reflection' as the result of serious thought and consideration which can cause for example the creation of a scientific discovery or theory, then it can be explained as a logical outcome of the following cognitive activities:

Reasoning (Deduction and Induction)

Many of the scientific (e.g computation, biological, social) theories are as a result of a chain of reasonings. Where one can interpret reasoning as an unsupervised or supervised pattern recognition, where for instance by seeing a view a person can recognize it or categorize it as a special event or group, and as a result it can be categorized under the 'learning by experience' or 'learning by imitation'. Reasonings help us to understand and being able to explain where an incident is coming from or where it will ends up or what is the meaning of a particular perception (e.g. a view, a sound, an smell or a combination of several senses (biological sensors)). Furthermore, following the trajectory of incidents (started by incident 'a') using chain of reasonings one can end up to a particular incident 'b' and form a social or scientific theory, where it begins by incident 'a' and ends up to incident 'b'. For example, a person look is faint, we can reason that its because he is hungry or sick, let say the person is hungry thus he does not have money or is very lazy to buy food, or let say the person is sick so he does not have money to buy food or he got a virus, perhaps his immune system is weak or the virus is new, and this chain of reasoning can continue to reach a particular theory based on the look of a person.

Imagination

When a person imagines, he creates or shapes a perception in his mind, which has been obtained according to what he has perceived previously (either through experience or imitation) with attention to his current thoughts and ideas; this imagination can be altered or enhanced with-in the framework of his believes (e.g. a scientific belief or a religious belief). Here, I emphasize that even though machine imagination concept that is discussed in [35] and similar works are interesting but they are not the topic that this paper intends to discuss about. In those works imagination for machine is discussed and explained as a process that convert an input, for example a text, to an image where the process and type of learning can be categorized more under the category of 'learning by imitation' (learning to produce a particular image when input is a particular text or sentence). Nonetheless, what this paper means by "imagination" in a machine is as a matter of fact a process that uses machine memory and knowledge in order to create new perceptions that did not shown to machine before and in order to learn from this new perception.

This paper categorizes imagination into the following categories:

-The first category is a synthesization of world model, based on the past experiences, for example, imagine a person that seen a place before, he is able to imagine this place, or world model, again in his mind and take new actions and calculates new results accordingly. In computer science there are instances of this kind of imagination, for instance SLAM [36], where a machine learn a model of the world and as a result imagines this model and particular outcomes with-in this imaginary model of the world according to particular actions. Model-based reinforcement learning algorithms are among those in which a model of the world can be built, and then refined and completed using experiences and observations. Model-based RL aims to endow an agent "with a model of the world, synthesized from past experience. By using an internal model to reason about the future, here also referred to as imagining, the agent can seek positive outcomes while avoiding the adverse consequences of trial-and-error in the real environment including making irreversible, poor decisions" [37]. One important consideration in this type of imagination is that it is limited by the model of the world, made by observation, that is, this kind of imagination cannot go beyond the observation of person or machine. In another world, this imagination is limited to experience and observation and it is kind of learning by experience. Thus, perhaps it is possible to think that model-based thinking (method) is a reasoning process which of course can reach to very important discoveries and outcomes. -In order to explain the second category, first, let's imagine Einstein thinking about time being relative, he is considering that his bus can move faster than a beam of light coming from the clock tower, this idea perhaps is not based on experience or observation (it is more than a model-based method). He was imagining something out of the world-model he experienced by creating at the first point an exception beyond or contradictory to the experiences and observations he had because nobody can move faster than light so it can not be experienced nor it can not be imitated, however, this way of thinking (exceptional or contradictory) paved the way to create a great theory. Furthermore, following the chain of reasonings it is possible to reach a point that the person does not have any experience or imitation information about it, for example considering biological advancements and discovery of a method that makes human im-mortal, the fact of immortal human perhaps is not experienced before, it can be imitated from the movies, but for the first time there was no movie about it, this example is also an imagination that is based on pure imagination contradictory or beyond experiences and imitations.

Is it possible to implement the 'learning by reflection' in machines? As explained in this section, a reflection can be explained as an imagination that is not limited to experience or imitation. Chain of reasonings and limited imagination both can be explained by 'learning by experience' or 'learning by imitation', however, a reflection, needs an elaboration, enhancement, or exception beyond the experienced world-model. This enhancement would happen according to the believe frame-work of the machine or person where this belief frame work can be scientific, religious or spiritual. Further, this enhancement can happen using what learned by imitation or by mixing different experiences into each other and judge the possibility of the result according to the believe framework of the person or machine.

Conclusion

In this paper, several ways of learning for a machine -learning by experience, learning by imitation and learning by reflection-inspired by human ways of learning wisdom explained by Confucius, introduced and discussed. Further, it explained that two of these methods, 'learning by experience' and 'learning by imitation' are widely researched and implemented in machine learning community under other titles. For example, reinforcement learning and unsupervised learning are based on 'learning by experience' and learning from demonstration and supervised learning are based on 'learning by imitation'. Nevertheless, the third way of learning wisdom, 'learning by reflection', is focused less and less implemented as well in machine learning community and algorithms. As a result, this paper discussed about this method of learning and the ways it can be explained through them in minds and machines.

Fig. 1

 1 Fig. 1 Reinforcement Learning Framework


Elsa Lhotel

Olivier Coupiac

François Lefloch
email: francois.lefloch@cea.fr

Hervé Courtois

M Sanquer

Divergence at low bias and down-mixing of the current noise in a diffusive superconductor-normal metal-superconductor junction

Keywords: numbers: 74.45.+c, 72.70.+m, 74.40.+k

Divergence at low bias and down-mixing of the current

noise in a diffusive superconductor-normal metal-superconductor junction

The charge transport in a diffusive superconductornormal metal -superconductor (S-N-S) junction occurs mainly through multiple Andreev reflections (MAR) [1]: normal metal quasiparticles with an energy ǫ < ∆ go through successive Andreev reflections at the two N-S interfaces until they reach the superconducting gap energy ∆. In one Andreev reflection, an incident electron (hole) is retro-reflected as a hole (electron). With an applied bias V , the number of successive MAR is equal to 2∆/eV + 1. A case of particular interest deals with long junctions defined by a junction length L larger than the superconducting coherence length ξ S but still smaller than the single particle coherence length L φ , in contrast with the short junction case L < ξ S [2] met for instance in superconducting atomic point-contacts [3,4].

In a long S-N-S junction, the Thouless energy E T h = hD/L 2 (D being the electron diffusion constant) is smaller than the gap ∆ and sets the energy scale for the coherent transport. Thus, two regimes can be met depending on the voltage bias. In the (high-bias) incoherent regime eV > E T h , the electron and the hole have, away from the interface, independent trajectories. The electronic transport then occurs by incoherent MAR, which induce excess noise as compared to a similar N-N-N system. The experimentally measured shot noise [8,9] can be described in the framework of the semiclassical theory [5,6,7].

In the (low-bias) coherent regime eV < E T h of interest here, coherent multiple Andreev reflections occur within an energy window of width E T h . Since the quasiparticles need to overcome the gap before leaving the normal metal, other processes like incoherent MAR and inelastic interactions are relevant. This makes the description of the coherent electronic transport in a S-N-S junction rather rich and complex. While the currentvoltage (I-V) characteristics can be calculated at zero temperature [10], the noise has not yet been derived. Recent measurements [8,9] show that the noise in such junctions is strongly enhanced at low voltage, which is partially described in the framework of coherent MAR. However, this analysis leads to surprisingly large effective charges compared to the expected cut-off of coherent MAR due to inelastic processes.

Thus the coherent transport in a long S-N-S junction is poorly understood yet. In particular, such a junction is a strongly non-linear conductor, where the differential resistance dV /dI is bias-dependent and generally differs greatly from the normal-state resistance R N or the ratio V /I. The non-linearity appears on a voltage scale eV much smaller than the thermal energy k B T . In the framework of the fluctuation -dissipation theorem, this raises the question of what dissipation-related quantity should be taken into account.

In this letter, we focus on the shot noise in the coherent regime of long diffusive S-N-S junctions (eV < E T h < ∆). Our low temperature noise measurements show that, when decreasing bias, the current noise exhibits a strong divergence together with a broad peak. We identify this peak as due to the non-linear mixing of the ac-Josephson current at the Josephson frequency ω J /2π = 2eV /h with the noise of the junction itself. We can then extract, from the measured noise, the intrinsic noise of the junction. The central result of this paper is that the diverging low-bias noise corresponds exactly to the thermal current noise of a non linear resistor with R = V /I, which throws new light on the problem of the fluctuation -dissipation theorem in non-linear conductors.

We measured the transport properties and the current noise of S-N-S junctions using a superconducting quantum interference device (SQUID) experimental setup [11] inserted in a dilution fridge (T min = 30 mK). The SQUID is connected to a resistance bridge (Fig. 1) composed of the sample on one side and of a reference resistor R ref on the other side. The current fluctuations propagating in this loop come from the reference resistor and from the sample. The value of the reference resis- tor R ref = 29 mΩ is chosen so that most of the measured current noise comes from the sample. The intrinsic noise level is about 8 µΦ 0 / √ Hz, which corresponds to 1.6 pA/ √ Hz in the SQUID input coil. Hereafter, the current noise measurements were performed in the frequency range between 4 kHz and 6.4 kHz.

The sample is a Cu bridge of 700 nm length and 425 nm width connected to two thick superconducting Al electrodes (see Fig. 2 inset). It has been fabricated by twoangles evaporation through a PMMA-PMMA/MAA bilayer mask in a ultra-high vacuum chamber. The Cu and Al thicknesses are 50 and 500 nm respectively. From the Cu diffusion constant estimated to D = 100 cm 2 .s -1 by resistance measurements, we get the Thouless energy of the sample E T h equal to 13 µeV.

The zero bias resistance is represented as a function of temperature in Fig. 2. It shows a small drop at 1.18 [12]. Such a value for a shows that the possible barrier resistance is negligible. Below 450 mK, the IV characteristic is hysteretic, with a retrapping current I R significantly below the critical current. In the following, we will focus on the non-hysteretic regime, T ≥ 500 mK.

IV characteristics measured in this regime are shown in Fig. 3 together with the resistively shunted junctions (RSJ) calculation at finite temperature [13] for T = 500 and 800 mK. As already mentionned in other studies on S-N-S junctions [8,14], the curves cannot be described within the RSJ model. Because all the experimental curves are lying between the 500 mK and 800 mK theoretical curves, the discrepancy cannot be simply due to heating effect. From the overall shape of IV characteristics, we can state that some current in the junction is supported by a normal "channel" (see right of Fig. 1), since in pure Josephson junctions no current can flow at a voltage less than the gap.

We first measured the shot noise over a large voltage scale (See Fig. 4 inset). The voltage dependence at V > 50 µV can be nicely described within the semiclassical theory of incoherent MAR [5,6] (with ∆ = 163 µeV) as already reported in a previous work on the same kind of junctions [9]. We also confirmed the existence of a noise minimum at the Thouless energy, which appears as a signature of the transition towards the coherent regime at low voltage, on which we will focus now. Fig. 4 shows the low voltage (V < 8 µV) part of the noise spectral density as a function of bias, at five temperatures from 500 to 800 mK. The first striking result is the overall divergence of the noise at low bias for every temperature. The second important feature is the broad peak with a maximum between 2 and 4 µV. When the temperature increases, it is less and less pronounced, moves to lower voltages and finally disappears around 800 mK. We analyze this broad peak in terms of a mixing between the ac-Josephson current I J and the fluctuating current δI N generated by the junction through the non-linear response of the junction. In such a situation, the voltage fluctuations S(ω) at low frequency are determined not only by the current fluctuations at the frequency ω, but also by those close to the Josephson frequency ω J and its harmonics, inducing the so-called mixed down noise. Following Likharev and Semenov [15], the resulting noise spectral density is:

S V (ω) = k |Z k | 2 S I (kω J -ω) (1)
where S I is the current noise spectral density of the fluctuations source and Z k are the Fourier coefficients of the junction impedance (V (ω) = k Z k I(ω -kω J )). Note that, due to the large SQUID impedance at the Josephson frequency ω J , there is no coupling between the S-N-S junction and the reference resistor at ω J . Thus, the reference resistor can not contribute to the mixed-down noise. At low frequency (ω ≪ ω J ), only the first harmonics (k = -1, 0, 1) of the ac-Josephson current are relevant [16]. Therefore, Eq. (1) becomes at zero frequency

S V (0) = R 2 d S I (0) + 2|Z 1 | 2 S I (ω J )
where R d is the differential resistance ∂V /∂I. For an arbitrary current-phase relation, Kogan and Nagaev [17] have shown that the co-efficient |Z 1 | 2 is equal to -V /4 × ∂R d /∂I. This leads to :

S V (0) = R 2 d [S I (0) -ηS I (ω J )] with η = V 2R 2 d ∂R d ∂I (2)
In our experimental set-up, the SQUID measures the resulting low frequency current noise that propagates through the SQUID input coil. It is given by S Imeas (0) = S V (0)/R 2 d = S I (0) -ηS I (ω J ). Because the thermal energy k B T is larger than the Josephson frequency ω J /2π = 2eV /h, we can assume that the spectral noise S I (ω J ) of the junction does not differ from its zero-frequency value S I (0). Then,

S Imeas (0) = S I (0)(1 -η) (3)
We obtained the coupling factor η from the measured quantities V and R d by numerically differentiating R d with respect to the calculated I. The quantity 1 -η is plotted in the inset of Fig. 5. We can then extract, from the measured noise, the intrinsic junction noise S I (0) = S Imeas (0)/(1 -η), see Fig. 5. For all the curves the broad peak has disappeared or, at least, has been strongly reduced [START_REF]At 500 mK, the differential resistance R d is so large[END_REF]. This clearly justifies the mixed down noise analysis using the generalized coupling factor η. To our knowledge, this prediction of Kogan and Nagaev [17] had never been verified experimentally. From Fig. 5, we see that S I (0) exhibits a strong divergence at low voltage. Therefore, it cannot be described by the thermal noise of the normal resistance 4k B T /R N as it was considered in Ref. 19 and 23. We cannot either understand the data by introducing an effective temperature. Indeed, we would get an effective temperature as high as 100 K which appears unrealistic since the aluminum critical temperature is 1.2 K.

We propose, in our temperature and voltage regime, that the noise of the junction can be written as:

S I (0) = 4k B T V /I (4)
The comparison with experimental data of S I (0) is shown in Fig. 5 with no adjustable parameter. Between 800 and 550 mK, the agreement is remarkable. The use of Eq. ( 4) can be understood in two ways. First, it is the low voltage limit (eV ≪ k B T ) of the quasiparticle noise of a junction with low transmitting channels : S I (0) = 2eI coth (eV /2k B T ) [20,21]. If effective charges e * > e are responsible for the transport, e should be replaced by e * = N e in the above expressions. In that case, the condition e * V = N eV ≪ k B T for Eq. 4 to remain valid, is verified as long as N < 30 at 1 µV and 550 mK. In this description, only channels with low transmission coefficients contribute to the current and to the noise. Eq. ( 4) might also be related to the fluctuationsdissipation theorem in a non-linear system, in which the resistance R N would be replaced by V /I(V ). Here, dissipation arises from electron-electron interactions that necessarily occur at low voltage due to the large number of Andreev reflections needed for the quasiparticles to escape into the superconducting electrodes. In both cases, we conclude that, at very low voltage and "high" temperature, the current noise in long S-N-S junctions is governed by the transport of incoherent quasiparticles between the two superconducting reservoirs, which induces a noise divergence when the voltage goes to zero. When decreasing further the temperature, the experimental data present a large excess noise compared to the noise obtained from Eq. (4). Therefore, additional noise sources must be involved. It is worth noting that this deviation from a pure quasiparticles noise occurs below 525 mK, which corresponds to k B T ≃ 3.5E T h , very close to the mini-gap width (3.1E T h ) [22]. This suggests that coherent processes, that become relevant in that energy scale, should play a role here.

In conclusion, we have presented current noise measurements in the non-hysteretic temperature range of a long diffusive S-N-S junction at very low bias (eV < E T h < ∆). The observed noise spectral density is enhanced by several orders of magnitude compared to the thermal noise of a normal junction. It exhibits a broad peak arising from the non-linear coupling between the junction current noise and the ac-Josephson current. This mechanism is of particular interest for studying the high frequency (quantum) noise of a dissipative conductor by means of low frequency measurements [23]. Here the dissipative channel is perfectly coupled to the nonlinear oscillator since they are both part of the same junction. The intrinsic noise of the junction that we could extract corresponds extremely well to the thermal-like noise 4k B T /R of a non-linear resistor R = V /I(V ). This shows up as a divergence at low voltage and appears to be analogous to the quasiparticles noise of a normal junction with channels of low transmission coefficients.

We acknowledge M. Houzet and F. Hekking for fruitful and continuous discussions. Samples have been fabricated at Nanofab-CNRS Grenoble. This work was funded by EU STREP project 'SFINx' and the IPMC/Grenoble.

FIG. 1 :FIG. 2 :

 12 FIG. 1: Left: Scheme of the measurement circuit. The mean current in the sample and the current fluctuations are measured through the input coil of the SQUID. The reference resistor R ref is at the same temperature than the sample of differential resistance R d . Right: Scheme of the sample, considered as a mesoscopic conductor with a fluctuations source δIN (ω) in parallel with a Josephson junction oscillating at ωJ .

FIG. 3 :

 3 FIG.3: IV curves at several temperatures between 500 and 800 mK. The lines are obtained from the RSJ model at 500 and 800 mK[13], with the parameters: RN = 0.85 Ω and the critical currents set to 12.7 and 3.4 µA respectively.

FIG. 4 :

 4 FIG.4: Current noise SI meas vs. bias voltage V at several temperatures between 500 and 800 mK. Inset: SI at 500 mK up to 120 µV. The solid line is the noise prediction due to incoherent MAR[5,6,9].

FIG. 5 :

 5 FIG.5: Current noise SI (0) = SI meas /(1 -η) vs. bias voltage V at several temperatures between 500 and 800 mK (Data are sucessively shifted by 100 pA 2 /Hz for clarity). Lines are the calculated noise 4kBT /(V /I). Inset: 1 -η vs. V at the same temperatures, where η is the coupling factor (See Eq. (2)).


Pierrick Guégan

Arnaud Poitou

Pierre Brémond

High speed experimental analysis of orthogonal planing using high speed infrared camera

Keywords: High speed infrared camera, high-speed machining, orthogonal cutting, machinability

An experimental analysis of the orthogonal cutting process is presented. Machining experiments have been performed using advanced techniques. A high-speed camera is coupled with an infrared thermal measurement aid on a dynamic test bench. This continuously reveals the deformations occurring on the flank of a chip, and the infrared camera gives information on the variation of thermal radiation (see the pictures under the abstract). The challenge consists of understanding the effect of machinability improvement treatments on metallic materials.

INTRODUCTION

Recent studies in the field of machining simulation make possible the creation of numerical tools, to simulate continuous or segmented chip formation [START_REF] Bäker | Finite-element simulation of segmented chip formation[END_REF]. These tools allow computation of the different criteria observed during the chip's formation (temperature, strain distribution, shear bands, etc). The validation of these numerical tools uses the experimental results of stopped tests or punctual observations [2, 3, 5, and 7]. A new method of experimental investigation is being developed on the crossbow system of Ecole Centrale de Nantes. The aim of this study is to reveal continuously the thermal phenomena and the associated strain distribution during an orthogonal cutting chip generation. A direct method is to use a thermal imaging focal-plane-array for spatially resolved non contact temperature measurements especially under conditions where abrupt spatial gradients in temperature can exist.

Non contact thermal measurements were made with an infrared camera with a close up lens made of a standard 50mm lens and additional extension ring. This device allows a spatial resolution of 35µm to be achieved.

EXPERIMENTAL METHOD

Experimental bench

The experimental bench used is a crossbow system. The machining energy is produced by the impact of a 210 kg mass on a machining mechanism (Fig. 1), which is mounted on a Hopkinson bar in order to measure the axial component of the machining force. A calibrated focal plane array monitors the test at a safe distance to avoid contaminants.

Machining mechanism

The machining mechanism includes a body and a crosshead (Fig. 2) which comes into the body when the mass impacts the machining mechanism. The test specimen is attached to the crosshead, and the body supports a tool. This allows the tool to cut the test specimen and to generate a chip by planing.

Video systems

For this kind of dynamic situation, the video systems must enable a continuous recording of the cutting operation and the heat phenomena (Fig. 3). Firstly, a high speed camera records the chip flank evolution. The observed surface of the test specimen is previously prepared by chemical attack, in order to make a post-treatment by correlated picture analysis. The pictures are analysed with specific software (ICASOFT from INSA Lyon, VIC 2D from Techlab, CORELLI from LMT Cachan) to determine displacements and strains in the material (Fig. 4).

Secondly, a high speed infrared camera JADE MWIR monitors this zone to study the thermal field variation due to machining. The infrared camera is a 320 x 240 pixel focal plane array using MCT technology. These detectors are known to have high sensitivity and high speed at high temperature. The spectral response within the middle wave band is 3.7 µm to 4.9µm. In this spectral range both CaF² and Sapphire windows can be used to protect lenses against contaminants. The detector size is 28µm and the pitch 30µm. By selecting an adapted camera configuration without filter alignments were done before tests. A narrow band filter centred around 3.99µm installed on filter wheel is then installed using the Altair software and remote control. This allows temperature to be increased to 1000°C. Windowing capability of the FPA limits the number of pixels in order to increase the frame rate. Fixed and random windowing are available to reach different frame rates and fields of view. Nevertheless accurate measurements are limited by emissivity evaluation. A precise evaluation of the temperature variation is not possible because the orientation between the observed surfaces and the camera axis varies, thus affecting the emissivity ε (λ,θ) . Some previous works show that the emissivity was around 0.14 and depends on the chip temperature. The temperature evaluation must then be performed carefully. However, the comparison of the thermal fields between two or more materials is now possible with good precision, and for identical geometric criteria.

Experimental conditions

The first tests are made with two austenitic stainless steels: UGINE 4301 and UGIMA 4301. The latter is an improved machinability version of the first steel. The tool material is a 4025 carbide from Sandvik-Coromant. The tool's geometry is modified so as to perform an orthogonal cutting operation (rake angle γ = 0°and clearance angle α = 3°). The selected impact speed is 3 m/s, a sufficient value for a high speed machining condition for these materials, and adequate for the performances of the camera. Altair software controls remotely the infrared camera. The recording of infrared films is controlled and triggered by a TTL signal delivered by the test machine. The workstation uses a standard notebook PC connected with a Dock in station. Recording in full frame format is possible in real time. The acquisition frequency selected was 3200 frames / s for infrared thermal camera (thermal image size 80 x 60 pixels). The picture definitions are chosen to enable the post treatment, and the maximum acquisition frequency is selected. Finally, the depth of cut « s » is fixed at 2 mm, and the width « a » at 4 mm.

INFRARED CAMERA & THERMAL RESULTS

Observation during chip formation

This method allows comparison of the machining of a couple of materials for identical experimental conditions. The observation with the infrared camera gives thermal field during machining on the flank of the test specimen (Figure 5).

Temperature estimation

The principal limitation of our method concerns the control of the emissivity ε (λ,θ) . In fact, this parameter is a function of the material, and influenced by the orientation surface / camera axis and temperature. If the orientation between the cutting face of the tool and the axis of the camera is constant, it is not the same for the orientation with the shear band and the plane. However, the Lambert law [START_REF] Gaussorgues | La thermographie Infrarouge[END_REF] explains for incidences between 0 and 60°, the maximal diminution of the emissivity is about 20 %. When the incidence angle is more than 60°, the emissivity decreases drastically and reaches 0 for 90°. Remaining in the range of 0 to 60°we can thus evaluate a global uncertainty of 20% for our measurements.

Figure 6 shows the thermal measurement for the two materials, on the cutting face of the tool (rep. Tool) and the shear zone on the flank, for each chip formation. These measurements are obtained with an emissivity of 0.8 which is an upper bound of the actual one, leading to temperature values which are thus very probably at least underestimated. With these considerations, we can estimate that 800 °C is the minimal temperature on the cutting face, and 500 °C that of the primary shear zone.

Comparison between two materials

The comparison between the stainless steels UGINE and UGIMA 4301 shows a higher thermal elevation in the shear band for the first (+ 4,5 %), on the mean temperature elevation value obtained for 20 segmented chips (Fig. 7). This observation is repetitive for identical experimental conditions.

Figure 1 :

 1 Figure 1: Machining mechanism on crossbow system.Figure 2: Machining mechanism.

Figure 2 :

 2 Figure 1: Machining mechanism on crossbow system.Figure 2: Machining mechanism.

Figure 3 :

 3 Figure 3: High speed and infrared thermal cameras.Figure 4: Deformation in shear zone

Figure 4 :

 4 Figure 3: High speed and infrared thermal cameras.Figure 4: Deformation in shear zone

Figure 5 :

 5 Figure 5: Two steps of thermal evolution in shear zone (left: beginning of segmentation, right: chip segmented), stainless steel UGINE 4301.

Figure 6 :

 6 Figure 6: Thermal evolution on tool and shear zone (stainless steel, left: UGINE 4301, right: UGIMA 4301). Emissivity ε (λ) = 0.8

Figure 7 :

 7 Figure 7: Thermal comparison in shear band, stainless steel UGINE and UGIMA 4301.

  The test camera specifications are presented in table1.Calibration of the camera is made by the supplier on black bodies. The calibration was made between 5°C and 1000°C with 6 calibration curves. Note that calibrations are independent of the fame rate. A drift control device within the camera allows compensation of the thermal drift during long tests or to compare tests made on different days.

	Detector size	Filter	Integration	Frame rate	Temperature	Spatial resolution
	(windowing)		time		range	(IFOV)
	80 x 60	3.99µm	50µs	3200Hz	Up to 1000°C	35µm

CONCLUSION

The infrared camera can be used in high speed machining to estimate the temperature field generated in the segmented chip. With some care and with consideration of an uncertainty factor, the absolute temperature determination is possible, but the comparative investigation between materials machined in the crosshead receives same conditions is more exact. However, this method can be optimized with the estimation of the incidence between observed surface and camera axis. Improvement could be possible in using two video cameras for a stereo correlation which could allow us to measure precisely the incidence angle of the infrared camera leading to a better evaluation of the surface emissivity.