M Shahsavari
email: m.shahsavari@imperial.ac.uk

J Beaumont

D Thomas

POETS: A parallel cluster architecture for Spiking Neural Network Mahyar Shahsavari, Jonathan Beaumont, David Thomas, and Andrew D. Brown Spiking Neural Networks (SNNs) are known as a branch of neuromorphic computing, and are currently used in neuroscience applications to understand and model the biological brain. SNNs could also potentially be used in many other application domains such as classification, pattern recognition, and autonomous control. This work presents a highly-scalable hardware platform called POETS, and uses it to implement SNN on a very large number of parallel and reconfigurable FPGAbased processors. The current system consists of 48 FPGAs, providing 3072 processing cores and 49152 threads. We use this hardware to implement up to four million neurons with one thousand synapses. Comparison to other similar platforms shows that the current POETS system is twenty times faster than the Brian simulator, and at least two times faster than SpiNNaker.

Index Terms-spiking neural networks, Parallel distributed system, reconfigurable architecture.

I. INTRODUCTION

Artificial Neural Networks (ANNs) are a flexible and robust computing means for solving complex problems. However due to frequent accesses to memory, it suffers from a memory bottleneck when running on the conventional hardware platforms [START_REF] Indiveri | Memory and information processing in neuromorphic systems[END_REF]. Spiking Neural Networks (SNNs) use biologically plausible neuronal models, and are a promising approach for hardware implementation of neural networks with capability of overcoming inefficient memory accessing by having the processor unit next to the memory [START_REF] Furber | The SpiNNaker Project[END_REF], [START_REF] Shahsavari | Spiking Neural Computing inMemristive Neuromorphic Platforms[END_REF], [START_REF] Merolla | A million spiking-neuron integrated circuit with a scalable communication network and interface[END_REF]. SNNs are potentially capable of modelling complex information processing in the brain, in addition to other potential applications in accelerators, robotic brains, low-power mobile processors, deep learning [START_REF] Tavanaei | Deep learning in spiking neural networks[END_REF], [START_REF] Fatahi | Towards an spiking deep belief network for face recognition application[END_REF], or Medtech [START_REF] Feng | Optimal deep brain stimulation of the subthalamic nucleus -a computational study[END_REF], [START_REF] Kasabov | Probabilistic computational neurogenetic modeling: From cognitive systems to alzheimer's disease[END_REF].

Several pure software SNN simulators have been developed, such as NEURON [START_REF] Carnevale | The NEURON Book[END_REF], NEST [START_REF] Gewaltig | Nest (neural simulation tool)[END_REF], or BRIAN [START_REF] Dan | Brian: a simulator for spiking neural networks in Python[END_REF], and these are widely used as research tools in the community of computational neuroscience. Although these tools have been used to train, model and simulate biologically plausible neuronal networks, they are faced with hardware performance constraints such as power, speed, flexibility, memory accessing latency so simulation of large-scale networks requires explicitly parallel processing [START_REF] Shahsavari | N2s3, a simulator for the architecture exploration of neuromorphic accelerators[END_REF]. In recent years advancements in high performance computing have led to the development of several large-scale hardware platforms dedicated to SNN applications, known as neuromorphic architectures. The most widely known large-scale neuromorphic systems are SpiNNaker [START_REF] Furber | The SpiNNaker Project[END_REF], IBM TrueNorth [START_REF] Merolla | A million spiking-neuron integrated circuit with a scalable communication network and interface[END_REF], NeuroGrid [START_REF] Benjamin | Neurogrid: A Mixed-Analog-Digital Multichip System for Large-Scale Neural Simulations[END_REF], and BrainScales [START_REF] Schemmel | A wafer-scale neuromorphic hardware system for large-scale neural modeling[END_REF] projects. Because of the parallel nature of neural networks, these large-scale concurrent systems are more efficient for data communication and spike transport compared to conventional platforms. However, the drawback is the programming complexity for these parallel systems, plus the need for analog or mixed analog/digital also increases complexity. In this work, we present the POETS (Partial Ordered Event Triggered Systems) [START_REF]Poets project website[END_REF] machine as a route to SNN simulation; POETS is a computation platform using an event-driven parallel programming model, backed by a custom FPGA many-core platform. POETS uses concepts from graph theory to provide a programming abstraction that makes programming this concurrent system manageable. This abstraction splits problems or applications into graphs, with events captured as messages moving between nodes in the graph, with events implementing both control-and data-flow. This allows for a high degree of concurrency, and allows us to get very large numbers of CPUs to work closely together on a single application. The current POETS system consists of 48 FPGAs, providing 3072 processing cores and 49152 threads. Our contributions in this work are:

• investigating and explaining the POETS architecture, and how it is used to implement SNNs; • hardware modeling of two neuron models, LIF (Leaky-Integrate-and-Fire) and Izhikevich, and a comparison of the two models in a large-scale network; • a demonstrator showing 4 million neurons, with each neuron connected to 1000 synapse, for a total of 4 billion synapses; • a comparison between POETS and state-of-the-art simulators and large-scale platforms.

II. POETS HARDWARE ARCHITECTURE

POETS is a project focusing on hardware support for an event-driven parallel programming model [START_REF]Poets project website[END_REF]. Applications running on POETs must first be transformed into graphs, in which vertices construct computation units, while edges represent communication links which sending and receiving messages. Somewhat similar programming models are Google's Pregel model [START_REF] Malewicz | Pregel: A system for large-scale graph processing[END_REF] and GraphStep model [START_REF] Delorimier | Graphstep: A system architecture for sparse-graph algorithms[END_REF], which provide a computing abstraction using both synchronous and asynchronous way of passing the messages; while the synchronous computing approach is deadlock-free and generally easier to program, the asynchronous approach can enable huge parallelism and scalability.

Efficient communication is a major strength of FPGA platforms, mainly due to an ability to process network traffic with minimal latency overheads. However, a major impediment to the wider adoption of FPGA platforms is the level of knowledge that is needed to develop in HDL (Hardware Description Language) effectively. Therefore, a promising solution is to provide a compiler/interpreter for FPGA developers to be able to use a higher level of abstraction for programming without taking the FPGA programming into consideration.

The POETS architecture currently consists of 8 boxes, where each box has an x86 server and 7 interconnected DE5-Net FPGA boards, as shown in Figure 1.a. Each DE5-Net supports 2 × 4GB DDR3 DRAMs, 4 × 8M B QDRII+ SRAMs, and 4 × 10G SFP+ ports. One FPGA is used as a PCIe to SFP+ bridge board, providing a fast connection between the x86 and remaining six worker FPGAs (Figure 1.a). The system supports both asynchronous and synchronous message exchanges between the cores. Each FPGA DE5-Net has 16 tiles, with each tile having 4 cores, and each core having 16 threads, so there are 1024 threads in each FPGA.

The computational heart of the POETS hardware is called Tinsel, which is a multithreaded RISC-V architecture optiimised for FPGA implementation. When a thread executes an instruction with high latency, it is suspended and will resume after instruction completion. The system is highly-pipelined and parallel, using a design-time specific number of threads per core. The default number of threads per core is 16 and it is extensible to 32. Tinsel uses a 2D tiled network-on-chip (NoC), with each tile containing a single mailbox and some number of cores (4 by default), one FPU (floating-point unit), and caches. A data cache is used to access off-chip memory on each DE5, providing access to the two DDR3 DRAMS and four QDRII+ SRAMS (see Figure 1.b andc). If we assume a memory access occurs every four instructions, a single DDR3 DRAM can satisfy a maximum of 64 cores (32-bit) running at 250MHz. As is shown in Figure 1.c, a mailbox mechanism is used to send and receive the messages between the soft-processors. Mailboxes are connected together to establish a distributed network which a thread can send a message to any other threads. FPU operations are executed using Altera IP blocks and have 14 cycles latencies at 250MHz [START_REF]Altera floating-point megafunctions user guide[END_REF]. More detailed information including messaging, codding, interconnections, resource utilization for the Tinsel can be found in [START_REF] Naylor | Tinsel: A manythread overlay for fpga clusters[END_REF].

III. MODEL OF NEURONS

We will now discuss the two SNN model which we have mapped into the POETS system.

A. Leaky Integrate-and-Fire Model LIF models are fast to simulate, and particularly attractive for large-scale network simulations [START_REF] Brette | Simulation of networks of spiking neurons: a review of tools and strategies[END_REF]. Neurons integrate the spike inputs from other connected neurons, with each arriving input spike changing the internal potential of the neuron, known as neuron's membrane potential or state variable. When the integrated inputs cause the membrane potential to passes a threshold voltage, the action potential occurs -in other words, the neuron fires.

The model is described by the neuron membrane potential:

τ n dv dt = -v(t) + RI syn (t) (1)
I syn (t) = j g ij n α(t -t (n) j ) (2)
where: v(t) represents the membrane potential at time t; τ n = RC is the membrane time constant; and R is the membrane resistance. The total input current, I syn (t), is generated by the activity of pre-synaptic neurons. The total input current injected into a neuron is the sum over all current pulses, which is calculated in Equation 2. Time t

(n) j
represents the time of the n th spike of post-synaptic neuron j, and g ij is the conductance of synaptic efficacy between neuron i and neuron j. Function α(t) = qδ(t), where q is the injected charge to the artificial synapse and δ(t) is the Dirac pulse function. If I syn (t) is big enough then the action potential can pass the threshold voltage, so the neuron fires. When there are no or only a few spikes in a time window, the neuron is in the leaky phase and the state variable decreases exponentially. The duration of this time window depends on τ n = RC.

B. Izhikevich Model

Another well investigated simple model of neuron that has been simulated in our work is Izhikevich [START_REF] Izhikevich | Simple Model of Spiking Neurons[END_REF] model. Izhikevich model reproduces the physiological plausibility of Hodgkin-Huxley-type neuron, yet are almost as computationally effective as the LIF neuron. The model is:

dv(t) dt = 0.04v 2 + 5v + 140 -u + I(t) (3)
du(t) dt = a(bv -u) (4)
v is the membrane potential and I is the sum of the synaptic currents from different nodes connected to the neuron in Equation 3, whereas Equation 4 represents the u that is the membrane recovery variable. When v has reached its threshold then the nueron fires, and then reset happens according to Equation 4.

v(v > v th ) = c and u(v > v th ) = u + d (5)
Fig. 2. An overview of a neuron section to be represented as a graph and read by compiler as an xml format file.

The Izhikevich spiking model has the potential to generate several different firing patterns, which can be selected using four dimensionless parameters a, b, c, and d:

• a represents the time scale of the recovery variable u, where a smaller value means slower recovery; • b represents the sensitivity of the recovery variable u to possible sub-threshold of the membrane potential v. Larger values indicated that v and u are strongly dependent on each other; • c presents the reset value of v after spiking; • d is the reset value of u after spiking.

IV. MAPPING SYNCHRONOUS ON AN

ASYNCHRONOUS PLATFORM of the system is to implement a highly scalable many-core system out of lots of tiny cores. Therefore, applications defining large numbers of simple devices with low computing complexity and a high degree of parallelism are desirable. Thus this fact is considered in designing a neuron node consisting of four fan-in, fan-out, clock and computing devices. In this work, a finite state machine (FSM) is controlling the states of neuron synchronously, however the message propagation is performed asynchronously. An overview of neuron unit including different devices and states is depicted in Figure 2. From a software developers point of view, the users and developer will not need to develop low-level FPGA code. A high level python program is used to generate XML that represents the networks, the network elements, the number of neuron and connections from one to other neurons, according to parameters that are set by users. The POETS compiler will take care of the intermediate compilation and loading the network into an FPGA.

The aim of designing a neuromorphic architecture on POETS is to simulate large-scale SNNs in a reconfigurable, flexible and scalable platform. The sequential development procedure of the POETS compilation flow is shown in Figure 3. Vertices in the network represent the neurons, and edges between vertices represent synaptic weight connections. A graph based application called Graph Schema is used to create a network of neurons, with spike messages used to transfer data between neurons with a high degree of parallelism. The general approach is to decompose the graph into clusters of reconfigurable devices, where the amount of intra-cluster edges is large. Similar to the previous works for designing SNN on FPGA [START_REF] Cheung | A large-scale spiking neural network accelerator for fpga systems[END_REF], [START_REF] Wang | An fpga implementation of a polychronous spiking neural network with delay adaptation[END_REF], while also taking advantages of Tinsel for high speed access from FPGA cores to the memory (10 Gbps Ethernet MAC), the system is capable of large-scale networks simulation at high speed. After creating the network graph and the connections, an XML format file will be generated that is a representation of the network graph. This graph is then transferred to network instances, and simulated on local or remote conventional GPP machines. The POETS compiler can also translate this file into FPGA-specific files which can be executed on Tinsel. Due to efficient data cacheing, in addition to the effective communication between threads via their mailbox, neuron devices in the bottom part of Figure 3 receive and send messages synchronously via a high speed network, and transfer the neuron parameters and weight connection modifications to and from memory.

V. RESULTS

We have used this approach to model networks of neurons ranging from 50 to 500000 for one box, and up to 4 million neurons in 8 boxes. In all networks 20 percent of neurons are inhibitory neurons. Both the Izkikevich and LIF models of neuron have been used in the network, but the implementation costs (including running and compilation times) show little difference between Izhikevich or LIF model, as shown in Figure 4. Another interesting result that can be extracted from the same data is the amount of parallelism. The hardware latency performance implementing anywhere from 50 to 1000 neurons is almost the same, as the nodes can be assigned to different threads, which compute simultaneously. In this work, for sending, receiving and neuron computation a duration of 4µS is considered as a time step known as delta cycle. More specific hardware characteristics are presented in Table I. Speed is a significant parameter has been evaluated in this platform,

Network implementation time for different connections

Number of Neurons

Time (Sec) q q q q q q q q q q q q POETS 1000 Synapse/Neuron-Izhikevich 1000 Synapse/Neuron-LIF 100 Synapse/Neuron-Izhikevich 100 Synapse/Neuron-LIF Fig. 4. A running time comparison of two LIF and Izhikevich models of neuron in POETS using small number of neurons with different numbers of synapses for each neuron. particularly when the number of neurons is increased. The maximum number of neurons that could be implemented on one box so far is 500 thousand. Therefore using 8 whole boxes, we are able to simulate 4 million neurons. For instance, the implementation time for 100000 neurons is only 4.05 second -however, this does not take into account compilation time, which could be longer where we use a conventional host computer. Running outputs for scalable devices are depicted in Figure 5. We compare our outputs with the Brian simulator [START_REF] Dan | Brian: a simulator for spiking neural networks in Python[END_REF] and the SpiNNaker [START_REF] Furber | The SpiNNaker Project[END_REF] hardware network in Figure 6.

The results demonstrate that poets is 20 times faster than Brian simulator version II. Compared to SpiNNaker, POETS is slightly slower for small networks, and more than two times faster for large neural networks.

VI. CONCLUSION

Spiking Neural Network (SNN) is a promising approach for future computing platforms, with the ability of learning which could be used in three different scenarios:

• An accelerator in GPP platforms to overcome the Von Neumann memory bottleneck, for example in robotic brains, or low-power mobile processors, [START_REF] Shahsavari | Parameter exploration to improve performance of memristor-based neuromorphic architectures[END_REF], [START_REF] Walravens | Spiking neural network implementation on fpga for robotic behaviour[END_REF]. • Direct implementation of neural network on hardware, taking advantage of low-cost computing for the same purposes as ANN (e.g., Deep Learning) applications such as prediction, detection and recognition [START_REF] Tavanaei | Deep learning in spiking neural networks[END_REF], [START_REF] Stromatias | Scalable energy-efficient, low-latency implementations of trained spiking deep belief networks on spinnaker[END_REF].

q q q q q q q q 100000 200000 300000 400000 500000 q q q q q q q q q q q q q q q q q q q q q q q q POETS 1000 Synapse/Neuron-Izhikevich 1000 Synapse/Neuron-LIF 100 Synapse/Neuron-Izhikevich 100 Synapse/Neuron-LIF Fig. 5. A running time comparison of two LIF and Izhikevich models of neuron in POETS using scalable number of neurons with different number of synapses for each neuron. q q q q q q q q q q 50 100 500 5000 50000 500000 0 20 40 60 80

Time of implementation for POETS v.s Brian and SpiNNaker

Number of Neurons Time (Sec) q q q q q q q q q q q q q q q q q q q q q q q Brian II Simulator POETS SpiNNaker Machine Fig. 6. Speed comparison of implementation among Brian simulator, SpiNNaker machine and POETS.

• In the long term, understanding properties of biological neural networks could be used as a hippocampal prosthesis to be connected to the biological network or to replace a damaged biological memory, for example in the Alzheimer effected memory [START_REF] Feng | Optimal deep brain stimulation of the subthalamic nucleus -a computational study[END_REF], [START_REF] Kasabov | Probabilistic computational neurogenetic modeling: From cognitive systems to alzheimer's disease[END_REF]. Several large scale spiking brain-like computing or neuromorphic hardware have been developed during recent years such as SpiNNaker, IBM TrueNorth, NeuroGrid and the BrainScales projects. In this work, we introduced POETS as a new largescale neuromorphic system which is flexible using FPGA clusters, reliable with guaranty of receiving messages, and fast regarding to the parallel processing of data yet relatively lowpower. The characteristics of large-scale systems are shown in the Table II to compare with POETS. In this work, we focused on scalability and architecture of system, while for future work we will investigate the accuracy of learning and neural network

Fig. 1 .

 1 Fig. 1. a) One POETS box, each box has 7 FPGAs, 6 workers connected to an X86 GPP via one intermediate FPGA. b) A default configuration of the Tinsel Network on Chip (NoC) on a single FPGA. c) Default structure of a Tinsel tile; the cores are highly reconfigurable soft-processors using the Tinsel micro-architecture.

Fig. 3 .

 3 Fig. 3. An overview of the design flow, from high-level neural model down to implementation in FPGA.

  Dr. David B. Thomas is a Senior Lecturer in the Dept. of Electrical and Electronic Engineering at Imperial College London, where he performs research and teaching in the area of computer engineering, particularly at the boundaries between software, processors, and custom digital hardware. Much of his research concerns the efficient use of reconfigurable hardware, include the design of highlevel languages and libraries for FPGAs, optimised numerical algorithms and IP cores, and the design of hardware optimised random number generators. Prof. Andrew Brown is a professor of electronics with Southampton University, United Kingdom. He has held visiting posts at IBM Hursley Park (UK), Siemens NeuPerlach (Germany), Multiple Access Communications (UK), LME Design Automation (UK), Trondheim University (Norway), Cambridge University (UK), and EPFL (Switzerland). He is a fellow of the IET and BCS, a chartered engineer, and a European engineer. He is a senior member of the IEEE.

TABLE II A

 II COMPARISON OF THE LARGE-SCALE NEUROMORPHIC SYSTEMS.Jonathan Beaumont is a research associate at Imperial College London. He achieved a PhD in Computer Engineering from Newcastle University in 2018, where he researched asynchronous systems. This included applications for asynchronous circuits, asynchronous algorithms, graph-based design methodologies, developing an automated method for designing asynchronous circuitry. Currently he works on the POETS project, developing software and applications to test a new experimental event-driven super-computing platform.

	Model/Properties TrueNorth	Neurogrid	BrainScales SpiNNaker	POETS
	Technology	Digital	Analog	Analog	Digital	Digital (FPGA)
	Feature size	28 nm	180 nm	180 nm	130 nm	28 nm
	Chips	16	16	325	48	48
	Power	3.2 W	3 W	500 W	80 W	42.8 w
	Interconnect	2D mesh-unicast	Tree-multicast Hierarchical 2D mesh-unicast 2D-mesh-unicast
	Neuron model	Configurable LIF Adaptive IF	Adaptive IF	Programmable	LIF/Izhikevich
	Neurons	16 M	1 M	200 k	768 k	4 M
	Synapses	4 G	4 G	40 M	768 M	4 G
	capabilities of POETS.

This work is supported by the UK Engineering and Physical Sciences Research Council under EPSRC grant EP/N031768/1.


F Pedraza
email: fpedraza@univ-lr.fr

R Podor

INFLUENCE OF ANNEALING CONDITIONS ON THE FORMATION OF HOLLOW Al2O3 MICROSPHERES STUDIED BY IN SITU ESEM

Keywords: aluminium microparticles, annealing, oxidation, coating, electron microscopy 1.-Introduction

The transformation of Al microparticles into hollow and broken Al2O3 microspheres was investigated by in situ environmental scanning electron microscopy (ESEM) up to 1150°C under different heating rates and 120 Pa of gas atmospheres. Slow heating rates (2°C min -1 ) resulted in a better coverage of the particles shell than with fast heating rates (20°C min -1 ) that delayed the threshold temperature at which the particles opened. Above this threshold, the amount of opened spheres increased with heating rate, with the coarser particles opening more than the small ones. It appeared that inert atmospheres (He-4%H2) increased the temperature at which the particles transformed, while air and pure oxygen tended to lower it.

In contrast, the temperature interval was larger and was size-dependent when using water vapour. Irrespective of the gas atmosphere, opening of the spheres allowed molten Al to flow out from the core and aluminise the substrate while leaving behind a top coat of hollow alumina spheres.

The synthesis methods and mechanisms involved in the transformation of bulk into hollow and capsule (broken) oxide nanoparticles have received considerable attention over the past few years because of the great variety of applications [START_REF] Lou | Hollow Micro-/Nanostructures: Synthesis and Applications[END_REF]. Whereas some review works focused on the fabrication methods of such hollow nanoparticles [START_REF] Lou | Hollow Micro-/Nanostructures: Synthesis and Applications[END_REF][START_REF] Bertling | Hollow microspheres[END_REF] other papers highlighted that the major mechanisms of formation are related to Kirkendall interdiffusion for instance in Ni/NiO [START_REF] Railsback | Size-dependent nanoscale Kirkendall effect during the oxidation of nickel nanoparticles[END_REF] or in Cu/Cu2O [START_REF] Nakamura | Formation of hollow ZnO through low-temperature oxidation of Zn nanoparticles[END_REF][START_REF] Nakamura | Hollow oxide formation by oxidation of Al and Cu nanoparticles[END_REF] to Cabrera-Mott mechanism, in which the electrostatic fields outweigh Kirkendall interdiffusion in the initial oxidation stages of Al/Al2O3 systems [START_REF] Nakamura | Formation of hollow ZnO through low-temperature oxidation of Zn nanoparticles[END_REF][START_REF] Nakamura | Hollow oxide formation by oxidation of Al and Cu nanoparticles[END_REF] and to Ostwaldripening in template-free Sn/SnO2 [START_REF] Lou | Template-free synthesis of SnO2 hollow nanostructures with high lithium storage capacity[END_REF] or Ti/TiO2 [START_REF] Li | Mesoporous titania spheres with tunable chamber structure and enhanced photocatalytic activity[END_REF] to describe the preferential dissolution of the particle interior because of thermodynamic or energetic considerations.

The synthesis of nano Al/ porous Al2O3 by template or wet chemical methods requires a final calcination step at high temperature in oxygen-rich atmospheres [START_REF] Liu | Synthesis and characterization of Al2O3 hollow spheres[END_REF][START_REF] Guo | Fabrication of SiO2, Al2O3, and TiO2 microcapsules with hollow core and mesoporous shell structure[END_REF]. This results in the formation of hollow particles of alumina by enhanced diffusion of oxygen through the oxide shell [START_REF] Rai | Understanding the mechanism of aluminum nanoparticle oxidation[END_REF]. The strong energy release upon the exothermic combustion reactions is also the basis of explosives [START_REF] Eisenreich | On the mechanism of low temperature oxidation for aluminum particles down to the nano-scale[END_REF] and therefore, various works have been devoted to ascertain the oxidation mechanisms of aluminium powders where the particle size, the heating rate and the annealing atmosphere have been considered [START_REF] Rai | Understanding the mechanism of aluminum nanoparticle oxidation[END_REF][START_REF] Trunov | Effect of polymorphic phase transformations in Al2O3 film on oxidation kinetics of aluminum powders[END_REF][START_REF] Rufino | Influence of particles size on thermal properties of aluminium powder[END_REF][START_REF] Schoenitz | Oxidation of aluminium particles in the presence of water[END_REF][START_REF] Rufino | Structural changes and thermal properties of aluminium micro-and nano-powders[END_REF][START_REF] Zhu | Oxidation of aluminum particles in mixed CO2/H2O atmospheres[END_REF]. Different major phenomena are agreed to occur below, at and over the Al melting temperature (660°C) whereby the amorphous alumina shell surrounding the Al metallic core grows (300-550°C) and converts into a -Al2O3 oxide layer (from 550°C and above) that eventually transforms into  or -Al2O3 at much higher temperatures (above 900°C) [START_REF] Trunov | Effect of polymorphic phase transformations in Al2O3 film on oxidation kinetics of aluminum powders[END_REF]. In air, Trunov et al. demonstrated that the small particles ignite faster and at lower temperature than the coarse ones while the oxidation steps are shifted towards higher temperatures with increasing the heating rates [START_REF] Trunov | Effect of polymorphic phase transformations in Al2O3 film on oxidation kinetics of aluminum powders[END_REF]. In inert atmospheres, Rufino et al. reported that crystallization of the alumina layer surrounding the Al metallic core occurs before the melting of Al regardless of the particle size (157 nm, and 1.2 or 17 µm -mean diameter-) [START_REF] Rufino | Influence of particles size on thermal properties of aluminium powder[END_REF]. However, under oxidizing conditions, ignition was also dependant on particle size even though the thickness of the alumina shell was similar [START_REF] Trunov | Effect of polymorphic phase transformations in Al2O3 film on oxidation kinetics of aluminum powders[END_REF]. In addition, the particles coalesced under oxidative conditions but not in inert atmospheres thereby demonstrating the influence of the annealing atmosphere [START_REF] Rufino | Influence of particles size on thermal properties of aluminium powder[END_REF]. Broken-up (crater-like) and cracked particles were explained to occur due to the receding molten Al interface as the metal flowed out of the particles through cracks and pores [START_REF] Rufino | Influence of particles size on thermal properties of aluminium powder[END_REF]. However, the conditions at which such morphologies appear are much related to the thickness of the crystallized alumina shell as the oxide may become mechanically stable and thus, cannot follow the thermal expansion of the aluminium core [START_REF] Rufino | Structural changes and thermal properties of aluminium micro-and nano-powders[END_REF]. Such increase of the oxide thickness has been shown to be strongly dependant on the annealing atmosphere by tracking the oxidation kinetics as a function of heating rate and particle size in Ar, O2, H2O and CO2 (and combinations thereof) [START_REF] Schoenitz | Oxidation of aluminium particles in the presence of water[END_REF][START_REF] Zhu | Oxidation of aluminum particles in mixed CO2/H2O atmospheres[END_REF]. It clearly appears that water greatly influences the oxidation kinetics compared to oxygen and carbon dioxide but the stepwise oxidation rate changes are better defined for coarse (10-14 µm) than for small (4.5-7 and 3-4.5 µm) particles and for low heating rates (2 and 5°C min -1 compared to 10 and 20°C min -1 ) [START_REF] Schoenitz | Oxidation of aluminium particles in the presence of water[END_REF][START_REF] Zhu | Oxidation of aluminum particles in mixed CO2/H2O atmospheres[END_REF]. The influence of particle size is also clearly demonstrated when comparing the works of Hasani et al. for 100-200 µm particles [START_REF] Hasani | The oxidation mechanism of pure aluminium powder particles[END_REF][START_REF] Hasani | A tension analysis during oxidation of pure aluminium powder particles: non isothermal condition[END_REF][START_REF] Hasani | Non-isothermal kinetic analysis of oxidation of pure aluminium powder particles[END_REF] and of Velasco et al. for 3.5 µm particles [START_REF] Velasco | Oxidation of micro-sized aluminium particles: hollow alumina spheres[END_REF] oxidized in air under both isothermal and non-isothermal conditions. The small particles formed hollow particles after their complete oxidation whereas the coarse ones were not completely oxidized even after exposures to 1400°C.

However, the mechanisms of formation of the hollow/broken alumina particles have been discussed for self-standing particles either in the nano or sub-micron range for combustion applications (explosives) or in the micro-scale from post-mortem observations after cooling and not for thermal barrier coating applications where the hollow and broken alumina microparticles were shown to thermally shield the nickel-based substrate [START_REF] Montero | Multipurpose TBC system based in alumina foam top coat and Al-rich diffusion layer produced by micro-scaled Al slurries[END_REF]. Indeed, preliminary investigations where a slurry containing a different range of Al microparticles deposited onto pure metals or alloys followed by different heat treatments allowed to form a top coat of hollow alumina spheres, and an aluminium-diffused layer underneath both linked through an Al2O3 based thermally grown oxide [START_REF] Kolarik | Multifunction high temperature coating system based on aluminium particle technology[END_REF][START_REF] Montero | A single step process to form in-situ an alumina foam/aluminide TBC system for alloys in extreme environments at high temperatures[END_REF][START_REF] Pedraza | Potential thermal barrier coating systems from Al microparticles. Mechanisms of coating formation on pure nickel[END_REF].

In contrast to previous works, this paper therefore investigates the mechanisms of formation of the hollow alumina top coat from microsized particles (5 µm of mean diameter) at different heating rates and annealing atmospheres by in situ environmental scanning electron microscopy (ESEM) at high temperature. Correlations with post-mortem SEM after cooling of the specimens are also established to elucidate the formation of the top coat of hollow and broken alumina spheres.

2.-Materials and Methods

A slurry containing 50 wt% of 99.7% pure Al-microparticles with a narrow distribution around 5 µm (D10 = 3.57 µm, D50 = 5.42 µm, D90= 8.24 µm) produced by the wire explosion method [START_REF] Ivanov | Productions of ultra-fine powders and their use in high energetic compositions[END_REF] by Sibthermochim (Russia) and 50 wt% of 1/10 polyvynil alcohol (PVA)/milliQ water binder was prepared at room temperature [START_REF] Rannou | Aging and thermal behaviour of a PVA/Al microspheres slurry for aluminizing purposes[END_REF] and air-sprayed onto a nickel based superalloy substrate (Ni-13.8Cr-9Co-0.6Mo-1.2W-1.6Ta-7.8Al-5Ti, at%) kindly supplied by Siemens. The air/slurry ratio was greater (2) than the one typically employed to produce even slurry coatings (1.4) [START_REF] Pedraza | Potential thermal barrier coating systems from Al microparticles. Mechanisms of coating formation on pure nickel[END_REF] in such a manner that open pores can be obtained in the coating to observe the coating/substrate interactions upon heating. The in situ observations were made in a FEI Quanta 200 environmental scanning electron microscope (ESEM) FEG microscope. A dedicated furnace equipped with a platinum heating element allows heating 6 mm diameter pieces directly in the ESEM chamber up to 1400°C, under maximum 750 Pa atmosphere. The sample temperature is controlled by a homemade Pt-Pt/Rh10 thermocouple placed in the sample holder below the sample to be studied [START_REF] Podor | Development of an integrated thermocouple for the accurate sample temperature measurement during high temperature Environmental Scanning Electron Microscope (HT-ESEM) experiments[END_REF] with a 5°C accuracy at the gold melting point (1064°C). In the present case the samples were square-like (3x3 mm 2 ) and 1.20.1 mm thick. Heating rates of 2, 5, 10, and 20 C min -1 were employed in 120 Pa of different gas atmospheres (He-4%H2, O2, air and H2O) up to 1150°C. high purity He-4%H2 (g) and O2

(g) from Alphagaz were injected from a gas bottle into the SEM chamber. Air came from the SEM room and contained less than 1 Pa of water vapour in 120 Pa of air. H2O was generated by heating a milliQ water reservoir and the vapour was injected into the SEM chamber through a Baratron pressure gauge to control a stable pressure of 120 Pa. The main advantage of the use of the HT-ESEM is to record image series of the sample surface with a nanometre resolution upon annealing [START_REF] Podor | In situ HT-ESEM observation of CeO2 grain growth during sintering[END_REF][START_REF] Boucetta | Mechanism of RuO2 crystallization in borosilicate glass: an original in situ ESEM approach[END_REF]. Image analyses to evaluate the number and the average radius of particles evolving with temperature under different testing conditions were performed in a 100 X 125 µm 2 area using ImageJ software [START_REF] Schneider | NIH Image to ImageJ: 25 years of image analysis[END_REF]. Post-mortem SEM/EDS analyses were also carried out in a FEI Quanta 200F with an EDAX detector in environmental mode to avoid charging effects of insulating materials.

3.-Results

3.1.-Influence of heating rate:

Heating in He-4%H2 (inert) gas at different rates (from 2 to 20°C min -1 ) did not result in any transformation of the particles until 690-700°C (Figure 1). In this temperature interval, the spheres began to break. Using image analysis, the amount of opened spheres was determined to increase with the heating rate (Table 1). Furthermore, the mean diameter of the opened spheres was greater (7.000.50 µm) than that (3.750.25 µm) of the non-opened spheres (at 2 and 10°C min -1 ). These effects of heating rate and particle size appeared to be strongly dependent on the transformations of the Al2O3 oxides morphology. Indeed, the slow growth of a compact Al2O3 shell was observed at low heating rates compared to the more porous shells grown at faster rates through which molten Al flowed out from the core. Similarly, the surface of the small particles appeared better covered with oxide than the coarse ones and opened more easily.

3.2.-Influence of gas atmosphere (inert, water vapour, oxygen and air)

The comparison of He-4%H2, H2O, O2 or air in the chamber with a fast heating rate (20°C min -1 ) up to about 750°C highlighted interesting differences. The pictures at the left of S1 to S4. In inert atmosphere, most the particles bursted suddenly at 695°C. In contrast, water vapour allowed to break the spheres progressively at a higher and larger temperature interval [668, 720°C] than in oxygen [665, 670°C]. In air, the temperature interval for the spheres to break was similar than in water vapour.

3.3.-Post-mortem observations

Figure 3 shows the overall view of the surface exposed to 120 Pa of inert, oxygen and water vapour atmosphere after cooling from 1150°C. The particles looked all well sintered but were more deformed and broken (open) in the inert gas than in oxygen or water vapour and had some bright contrasted areas that were found to be rich in nickel. Within the pores, molten Al was seen to come out from the spheres to aluminize the substrate (Figure 4a). In tilt mode (30°) (Figure 4b), a layered structure with a "cliff" appearance developed underneath the hollow particles with two clearly distinctive areas. The top one contained numerous layers whereas the bottom one was more even in morphology and composition according to the backscattered electron contrast. The spot EDS analyses on such different areas (Figure 4c) suggested that the surface of the substrate was oxidised even in the inert atmosphere with major contributions from Al, Cr and Ti. Moving upwards from the substrate to the top coat of spheres, the oxygen and Al contents increased whereas those of the alloying elements of the substrate decreased (Cr, Ti and Co). The enrichment of Ni in the bottom area of the "cliff" is a clear indication that nickel dissolved into molten Al at relatively low temperatures [START_REF] Rannou | On the influence of a heat treatment for an aluminizing process based on Al microparticles slurry for Ni and Ni20Cr. Experimental and theoretical approaches[END_REF][START_REF] Bonnet | Initial aluminizing steps of pure nickel from Al micro-particles[END_REF][START_REF] Galetz | The role of combustion synthesis in the formation of slurry aluminization[END_REF] and therefore the surface of the substrate was poorer in nickel than in the bulk alloy. The interdiffusion of Al and Ni resulted in the aluminisation of the substrate (Figure 4d).

4.-Discussion

4.1.-Mechanism of formation of individual Al2O3 hollow spheres

The transformation of Al into hollow and broken Al2O3 particles considers two concomitant mechanisms including the oxidation of the Al shell and the melting and release of Al at temperatures at or above the melting temperature of Al (660°C). The latter is known to be influenced by the presence of impurities that form eutectic (molten) phases at lower temperatures [START_REF]ASM-Binary alloy phase diagrams[END_REF]. Here, we also demonstrate that the release of Al from the spheres and the simultaneous oxidation are shown to be strongly dependent on heating rate and gas species in the chamber, in agreement with other reports [START_REF] Trunov | Effect of polymorphic phase transformations in Al2O3 film on oxidation kinetics of aluminum powders[END_REF][START_REF] Rufino | Influence of particles size on thermal properties of aluminium powder[END_REF][START_REF] Schoenitz | Oxidation of aluminium particles in the presence of water[END_REF][START_REF] Zhu | Oxidation of aluminum particles in mixed CO2/H2O atmospheres[END_REF]. Indeed, the agreed mechanisms of the very initial formation of hollow and broken particles are based on the Cabrera-Mott mechanism whereby the electric fields make Al and O to diffuse at different rates in the very initial oxidation steps [START_REF] Nakamura | Formation of hollow ZnO through low-temperature oxidation of Zn nanoparticles[END_REF] after which the Kirkendall difference of metal and oxygen flow take the lead at least below the melting temperature of Al (660°C) [START_REF] Park | Size resolved kinetics measurements of aluminum nanoparticle oxidation by single particle mass spectrometry[END_REF]. Aluminium being hydroxylated in the slurry, the shell of the oxo-hydroxide grows with temperature until the binder is burned or evaporated [START_REF] Rannou | Aging and thermal behaviour of a PVA/Al microspheres slurry for aluminizing purposes[END_REF] and is subsequently transformed into the -Al2O3 at about 550°C in air [START_REF] Trunov | Effect of polymorphic phase transformations in Al2O3 film on oxidation kinetics of aluminum powders[END_REF] then in -and -Al2O3 at higher temperatures [START_REF] Eisenreich | On the mechanism of low temperature oxidation for aluminum particles down to the nano-scale[END_REF][START_REF] Trunov | Effect of polymorphic phase transformations in Al2O3 film on oxidation kinetics of aluminum powders[END_REF] but direct transformation  into -Al2O3 can also occur [START_REF] Levin | Metastable alumina polymorphs: crystal structures and transition sequences[END_REF]. These phase transformations bring about shrinkage of the shell [START_REF] Rufino | Influence of particles size on thermal properties of aluminium powder[END_REF] and therefore open access to molten Al to flow outside of the spheres [START_REF] Pedraza | Potential thermal barrier coating systems from Al microparticles. Mechanisms of coating formation on pure nickel[END_REF] and/or to be injected into the shell [START_REF] Rai | Importance of phase change of aluminum in the oxidation of aluminum nanoparticles[END_REF] and expand [START_REF] Firmansyah | Microstructural behaviors of alumina shell and aluminum core before and after melting of aluminum nanoparticles[END_REF]. The disruption of the oxide film depends mostly in the differences in the thermal expansion coefficients and densities of the metal and its oxide, as well as in the volume change of the metal during phase or polymorphic transformations [START_REF] Rosenband | Thermo-mechanical aspects of the heterogeneous ignition of metals[END_REF].

However, when the thickness of the oxide shell is sufficiently thick and thus, mechanically stable, compressive strains in the Al core and tensile stresses in the oxide shell appear irrespective of the particle size [START_REF] Rufino | Influence of particles size on thermal properties of aluminium powder[END_REF]. Mechanically damaging of the oxide shell was quoted to reduce the melting temperature due to a decrease in generated pressure within the Al core in nanoparticles [START_REF] Levitas | Effect of the alumina shell on the melting temperature depression for aluminum nanoparticles[END_REF]. However, no breakage of the shell was required until complete melting of the core for very fast heating rates. Trunov et al. found that the increase of the heating rate shifted the oxidation steps towards higher temperatures in air and related this phenomenon to additional heat release due to oxidation over heat capacity of the microparticles [START_REF] Trunov | Effect of polymorphic phase transformations in Al2O3 film on oxidation kinetics of aluminum powders[END_REF]. However, the oxidation effect cannot solely explain this phenomenon as our experiments in both inert (He-4%H2) and O2 atmospheres resulted both in the full opening of the spheres below 700°C even though our tests were carried out at low pressure (< 750 MPa) instead of atmospheric pressure like in [START_REF] Trunov | Effect of polymorphic phase transformations in Al2O3 film on oxidation kinetics of aluminum powders[END_REF]. It rather appears that the fast heating rates do not allow the Al core to sufficiently heat up. Therefore, the Al core cannot expand and create a pressure build-up within the particles to make them burst over the theoretical melting temperature of Al. This hypothesis would be in agreement with the findings of Hasani et al. on their tension analysis during oxidation of pure Al particles [START_REF] Hasani | A tension analysis during oxidation of pure aluminium powder particles: non isothermal condition[END_REF], who reported that the higher stresses imposed on the crust by the melt resulted in more intense oxidation after rupture, hence in the bursting of the particles that we observed in this work.

In contrast, it appears that the critical point corresponding to the particle opening is clearly dependent on time and temperature for a given annealing atmosphere and heating rate as also demonstrated by Velasco et al. on 3.7 microsized Al particles [START_REF] Velasco | Oxidation of micro-sized aluminium particles: hollow alumina spheres[END_REF]. An example of image series is reported on Figure 5 for a 120 Pa He+4%H2 atmosphere where the evolution from full to hollow particles is shown (see supplementary file S5 for complete video). Before the opening, the particles deform progressively. Above a deformation threshold, the liquid Al trapped in the particle flows outside and the particle is emptied. This final modification occurs very rapidly (the delay between two images is 4 seconds) and leaves behind the alumina shell.

The opening of the alumina shell always appears in the areas that were largely deformed.

However, liquid Al is not directly observed on the surface since it was shown to flow towards the underlying substrate and between the Al particles [START_REF] Bonnet | Initial aluminizing steps of pure nickel from Al micro-particles[END_REF]. The thickness of the oxide shell appears to be very thin under He+4H2 and O2. This is in good agreement with data reported by Rufino et al. who determined 15 to 30 nm-thick alumina layers when annealing aluminium particles in air up to 660°C [START_REF] Rufino | Influence of particles size on thermal properties of aluminium powder[END_REF].

4.2.-General behaviour of a population of Al2O3 hollow spheres.

Statistical results on the particle population were obtained by changing the gas composition in the chamber. Figure 6 displays the statistical distribution of the number of particles that opened in He-4%H2, O2, air and H2O. Air was introduced in this comparative study since flowing N2 (g) could potentially nitride Al although the pressure required to produce nitridation is about five times greater than for oxidation [START_REF] Okada | Direct nitridation of aluminum compacts at low temperature[END_REF]. It appears that the greatest number of particles breakage is centred at around 675°C for both O2 and air and thereby nitrogen does not affect the behaviour in this temperature range. A larger distribution centred between 680-690°C occurs with H2O. In contrast, the inert atmosphere has two distribution areas, one centred at 690 and a second one at 705°C. These results seem to confirm that the opening of the spheres can be governed by various mechanisms implying (a) the burning of the organic binder covering the shells and (b) the transformation and the stabilisation of the -Al2O3 covering the shells. Indeed, in the inert atmosphere, the deshydroxylation of Al occurs at about 625°C because of the transformation of the boehmite type structure into the amorphous alumina [START_REF] Yang | Effects PVA content and calcination temperature on the properties of PVA/boehmite composite film[END_REF]. However, in the presence of oxidising species the organic binder was observed to burn. Therefore, direct transformation of the hydroxide into the -Al2O3 oxide occurs [START_REF] Kou | Hollow Al2O3 microspheres derived from Al/AlOOH•nH2O core-shell particles[END_REF]. This transformation can be also accompanied by shrinkage of the shell allowing Al outward diffusion that likely incorporates in the shell [START_REF] Rai | Importance of phase change of aluminum in the oxidation of aluminum nanoparticles[END_REF][START_REF] Firmansyah | Microstructural behaviors of alumina shell and aluminum core before and after melting of aluminum nanoparticles[END_REF] and, with temperature, is transported towards other particles and/or the substrate [START_REF] Montero | Multipurpose TBC system based in alumina foam top coat and Al-rich diffusion layer produced by micro-scaled Al slurries[END_REF]. However, the extent at which this phenomenon occurs is size-dependent as shown in Figure 7, where the mean particle radius opening with respect temperature is plotted for different gas atmospheres. Indeed, the coarse particles break up at lower temperatures than the small ones regardless of the gas atmosphere. This can be attributed to the larger (re)active surface of the small particles thereby oxidising quicker but is in contrast with the results from Trunov et al., who claimed that the smaller Al microparticles ignited faster and at lower temperatures than the coarser ones in spite of the widely varying ignition temperatures [START_REF] Trunov | Effect of polymorphic phase transformations in Al2O3 film on oxidation kinetics of aluminum powders[END_REF]. Rufino et al. also reported that the nanopowders displayed a greater reactivity than the microparticles but the thickness of the alumina layer was independent of the particle size because the diffusion of oxygen was slowed down by the crystallized alumina [START_REF] Rufino | Influence of particles size on thermal properties of aluminium powder[END_REF]. Two major reasons can help in explaining these apparently, contradictory observations. First, the above studies referred to very different particle sizes (3-4.5 and 10-14 µm [START_REF] Trunov | Effect of polymorphic phase transformations in Al2O3 film on oxidation kinetics of aluminum powders[END_REF] and 200 nm, 1.60 and 8.35 µm [START_REF] Rufino | Influence of particles size on thermal properties of aluminium powder[END_REF]) whereas our particle distribution is much narrower (D10 = 3.57 µm, D50 = 5.42 µm, D90= 8.24 µm) and secondly, our particles were oxy-hydroxylated by the PVA/H2O binder in the slurry prior to heating in the different atmospheres. The smallest particles (5 µm) indeed underwent greater mass losses than the coarse ones (20 µm) when annealed in TGA tests [START_REF] Rannou | Aging and thermal behaviour of a PVA/Al microspheres slurry for aluminizing purposes[END_REF], which is indicative of enhanced adsorption of PVA to the small particles. In addition, whereas transformation of boehmite (AlOOH) into -Al2O3 has been quoted to occur at 400°C in the absence of binder, the addition of PVA resulted in nano pore and crack-free -Al2O3 membranes [START_REF] Lambert | Effect of binder addition on the properties of unsupported -Al2O3 membranes[END_REF] that makes alumina being stable till about 800°C [START_REF] Lin | Thermal stability and its improvement of the alumina membrane top-layers prepared by sol-gel methods[END_REF]. Additional heating at higher temperatures indeed showed that grain growth, appearance of greater number of voids and of whisker formation only occurred from 975 and 1060°C, respectively, in H2O and O2.

Such evolution of the morphology can be associated with -Al2O3 [START_REF] Levin | Metastable alumina polymorphs: crystal structures and transition sequences[END_REF] also observed on the studies of thermal barrier coating formation on pure nickel from a slurry containing Al microspheres [START_REF] Pedraza | Potential thermal barrier coating systems from Al microparticles. Mechanisms of coating formation on pure nickel[END_REF].

Another interesting parameter to be determined is the ratio between the number of reacting particles versus the total number of particles. The images reported on Figure 8 illustrate the sample surface morphology at 850°C, depending on the reacting atmosphere. The surface coverage rates are 33.5%, 73.6% and 79.2% in, respectively, He+4%H2, O2 and H2O, respectively (heating rate of 5°C min -1 ). The reacting atmosphere has therefore a great influence on the ability of the Al particles to form hollow alumina spheres, oxygen and water vapour being the most favourable, which is in agreement with the studies of Schoenitz at al. [START_REF] Schoenitz | Oxidation of aluminium particles in the presence of water[END_REF] and Zhu et al. [START_REF] Zhu | Oxidation of aluminum particles in mixed CO2/H2O atmospheres[END_REF]. This can be related to enhanced growth of the Al2O3 layer in these oxidising conditions.

4.3.-Aluminisation of the substrate

The supply of Al from the spheres to the Ni-based substrate was also directly observed at a 5°C min -1 of heating rate and was concomitant with the formation of the hollow spheres (Figure 9, see supplementary file S6 for complete video). As molten Al flows, dissolution of Ni occurs that provokes great exothermal reactions between Al and Ni, with a subsequent local temperature increase that entertains a self-propagating combustion synthesis mechanism [START_REF] Galetz | The role of combustion synthesis in the formation of slurry aluminization[END_REF].

Therefore, as far as the Al reservoir is not exhausted, Al and Ni interdiffusion occurs along the heat treatment that results in the aluminisation of the substrate underneath the top coat of hollow spheres (Figure 4).

5.-Summary and conclusions

SEM in situ investigations at high temperature and different gas atmospheres and heating rates allowed to identify the mechanisms of formation of hollow alumina particles from Al microparticles. Whereas the fast heating rates resulted in the bursting of particles, more oxidising atmospheres allowed to thicken the oxide shell. The opening of the spheres was therefore shifted to higher temperatures. Opening or bursting of the particles allowed molten Al to flow towards the substrate and to simultaneously aluminise it. Such multi-layered system with hollow/broken spheres of alumina on top and a NixAly aluminised layer underneath open then new ways of investigation for potential applications as thermal barrier coating systems due to the insulating character of entrapped air in the spheres. Figure 9 -Aluminisation of the substrate (at 5°C min -1 ). Note the coverage of the substrate (centre of the pore) with increasing temperature.

Figure 2

 2 Figure 2 are taken at 665°C whereas the ones shown at the right indicate the temperature at

  Part of this work was performed under the programme PARTICOAT FP7-NMP-2007-LARGE-1-CP-IP-211329-2 (2008-2012) funded by the European Union.

Figure 1 .

 1 Figure 1. Influence of heating rate on the evolution of the Al microparticles in He-4%H2 annealing atmosphere.

Figure 2 .

 2 Figure 2. Influence of annealing gas composition on the evolution of the Al microparticles.

Figure 3 .

 3 Figure 3.-BSE images after exposure 1150°C in (a) He-4%H2, (b) O2 and (c) H2O. Note the deformation and open spheres in the absence of oxidising species. The inset in (c) allows to appreciate the thickness of the oxide shell covering the particles.

Figure 4 .

 4 Figure 4.-(a) top view of a pore of the coating after annealing at 1150°C in (here for) He-4%H2 and (b) tilted 30° in another area showing the substrate, the layered structure and the top coat of hollow spheres and (c) the EDS spot analyses of the different areas. Note that the oxygen content was approximately the same (10 at%) regardless of the annealing atmosphere as the analyses were performed in environmental mode. (d) is a cross-section view showing the aluminised substrate (NixAly intermetallic phase) underneath the top coat of Al2O3 hollow spheres.

Figure. 5 -

 5 Figure. 5 -Mechanism of formation of individual Al2O3 hollow spheres.

Figure 6 .

 6 Figure 6.-Statistical distribution of the number of particles that opened in 120 Pa of He-4%H2, O2, air and H2O with respect temperature.

Figure 7 .

 7 Figure 7.-Mean particle radius opening with respect temperature at 120 Pa of He-4%H2, O2, air and H2O.

Figure 8 .

 8 Figure 8. Surface of the samples after heat treatment at 850°C showing the formation of hollow alumina spheres and the surface coverage (33.5% in He+4H2 ; 73.6% in O2 ; more than 79.2% in H2O) with a heating rate 5°C min -1 .

Table 1 .

 1 Evolution of the quantity of reacted Al microparticles as a function of the heating rate in 120Pa He-4%H2 annealing atmosphere (calculation made from images recorded at 700°C).

	Opened	particle	Sample surface covered by
	number (%)	opened particles (%)
	2°C/min	27	33
	10°C/min	53	77
	20°C/min	100	100


Shijia Du

Zhiguo Zeng
email: zengzhiguo@buaa.edu.cn

Lirong Cui

Rui Kang

Reliability analysis of Markov history-dependent repairable systems with neglected failures

Keywords: Markov repairable system, Markov history-dependent repairable systems, neglected failure, availability, aggregated stochastic process

Markov history-dependent repairable systems refer to the Markov repairable systems in which some states are changeable and dependent on recent evolutional history of the system. In practice, many Markov history-dependent repairable systems are subjected to neglected failures, i.e., some failures do not affect system performances if they can be repaired promptly. In this paper, we develop a model based on the theory of aggregated stochastic processes to describe the history-dependent behavior and the effect of neglected failures on the Markov history-dependent repairable systems. Based on the developed model, instantaneous and steady-state availabilities are derived to characterize the reliability of the system. Four reliability-related time distributions, i.e., distribution for the k th working period, distribution for the k th failure period, distribution for the real working time in an effective working period, distribution for the neglected failure time in an effective working period, are also derived to provide a more comprehensive description of the system's reliability. Thanks to the power of the theory of aggregated stochastic processes, closed-form expressions are obtained for all the reliability indexes and time distributions. Finally, the developed indexes and analysis methods are demonstrated by a numerical example.

Introduction

Reliability is an important feature of repairable systems [START_REF] Liu | A cold standby repairable system with working vacations and vacation interruption following Markovian arrival process[END_REF]- [START_REF] Montoro-Cazorla | A reliability system under different types of shock governed by a Markovian arrival process and maintenance policy K[END_REF]. In most existing literatures, reliability analysis is studied by modeling the repairable system as a Markov process (e.g., see [START_REF] Csenki | Joint interval reliability for Markov systems with an application in transmission line reliability[END_REF]- [START_REF] Liu | Interval reliability for aggregated Markov repairable system with repair time omission[END_REF]). Most of these Markov-process-based models assume that there are clear definitions of normal state and failure state and the definitions are independent of the evolutional history of the system. In practice, however, some states in a repairable system might be changeable and depend on the recent evolutional history of the system (for example, see [START_REF] Cui | Markov repairable systems with history-dependent up and down states[END_REF] [START_REF] Wang | Aggregated semi-Markov repairable systems with history-dependent up and down states[END_REF]). This kind of special system was named by Cui et al. [START_REF] Cui | Markov repairable systems with history-dependent up and down states[END_REF] as Markov Repairable System with History-Dependent up and down States (MRS-HDS) and reliability analysis methods were developed by them to calculate its availability. Based on the work of [START_REF] Cui | Markov repairable systems with history-dependent up and down states[END_REF], Wang et al. [START_REF] Wang | Multi-Point and Multi-Interval Availabilities for Markov Repairable Systems with History-Dependent Up and Down States[END_REF] derived multi-point and multi-interval availabilities for the MRS-HDS. Zheng et al. [START_REF] Zheng | Availability of semi-Markov repairable systems with history-dependent up and down states[END_REF] generalized Cui's work in [START_REF] Cui | Markov repairable systems with history-dependent up and down states[END_REF] to semi-Markov repairable systems with history-dependent up and down states.

In many practical systems, if the repair time is shorter than a predefined threshold, the failures will have little influence on systems' performances, so that the failures can be neglected when evaluating the system's availability. For example, consider a failure in a water supply system that cut off the water supply to the end-users. If the failure is recovered in a relatively short period, e.g., several minutes, the end-users are possible to tolerate or even unaware of it. This kind of failure, which can be promptly repaired within the tolerance of the end-users, can therefore, be neglected when calculating the availability of the water supply system. Such failures are regarded as neglected failures and are commonly encountered in repairable systems. For instance, Zheng et al. [START_REF] Zheng | A study on a single-unit Markov repairable system with repair time omission[END_REF] studied a single-unit Markov repairable system with repair time omission. Bao and Cui [START_REF] Bao | An analysis of availability for series Markov repairable system with neglected or delayed failures[END_REF] considered series Markov repairable systems with neglected or delayed failures.

Neglected failures also exist in MRS-HDS, e.g. the water distributed system, the power distributed networks, computer networks, etc. The introduction of changeable states, as a result of the neglected failures, makes standard Markov-process-based models incapable to capture the system's failure and repair behaviors. However, to the best of our knowledge, the effect of neglected failures on the reliability of MRS-HDS has not been considered before. To fill this gap, we develop a new model and associated reliability analysis method based on the theory of aggregated stochastic process and then derive some important reliability indexes for the system. Although a large number of literatures focused on obtaining reliability indexes for repairable systems (e.g. [START_REF] Du | A study on joint availability for k out of n and consecutive k out of n points and intervals[END_REF]- [START_REF] Du | Reliabilities of a single-unit system with multi-phased missions[END_REF]), it is generally difficult to obtain close-formed expressions for these indexes, especially when the system's behavior is complex, as the case of MRS-HDS with neglected failures. In this paper, however, with the help of the aggregated stochastic processes theory, close-formed expressions can be obtained for all the derived indexes.

Aggregated stochastic processes were invented by Burke and Rosenblatt [START_REF] Burke | A Markovian function of a Markovchain[END_REF] in 1957. One of the milestones in the history of the aggregated stochastic process is Colquhoun and Hawkes's successful work in developing probabilistic models to describe the behavior of ion channels (e.g., see [START_REF] Colquhoun | Relaxation and fluctuations of membrane currents that flow through drug-operated ion channels[END_REF] [START_REF] Colquhoun | On the stochastic properties of bursts of single Ion Channel openings and of clusters of bursts[END_REF]). Since then, aggregated stochastic process has received more and more attention from various areas. Among them, Rubino and Sericola [START_REF] Rubino | Sojourn times in finite Markov processes[END_REF] first applied the theory to model a repairable system. Various models were, then, developed based on the aggregated stochastic processes to describe the behavior of different types of repairable systems, e.g., Hawkes et al. (e.g., [START_REF] Hawkes | Modeling the evolution of system reliability performance under alternative environments[END_REF]), Cui et al. (e.g. [START_REF] Cui | A study on a single-unit repairable system with state aggregations[END_REF] and [START_REF] Cui | Reliability measures for two-part partition of states for aggregated Markov repairable systems[END_REF]), Wang et al. ([24] and [START_REF] Wang | Reliability Evaluation for Multi-State Markov Repairable Systems with Redundant Dependencies[END_REF]), Liu et al. (e.g. [START_REF] Liu | A performance measure for Markov system with stochastic supply patterns and stochastic demand patterns[END_REF]), etc. By employing aggregated stochastic processes, we focus on aggregating and decomposing the states of the new model. To be specific, because changeable and down states can present both failure and operational states, these two states are duplicated and redefined, in order to describe the state-dependent and time-dependent behavior of the system. A new stochastic process, which is an aggregated stochastic process, can be constructed and the theory of aggregated processes can be used to achieve analytical expressions of some important reliability indexes and reliability-related time distributions.

The rest of the paper is organized as follows. In Section 2, the motivation of this paper is fully explained with an industrial example and assumptions of the developed model are formally presented. In Section 3, we use aggregated stochastic processes to model the behavior of MRS-HDS with neglected failures. Some reliability indexes for MRS-HDS with neglected failures, i.e., the instantaneous availability, steady-state availability and some sojourn time distributions are derived in Section 4. In Section 5, the developed indexes and analysis methods are demonstrated by a numerical example. Finally, conclusions are given in Section 6.

Motivation and assumptions

Motivation example

In this section, we present an engineering example of MRS-HDS with neglected failures which motivates the research in this paper. Take an electrical power supply system as an example, which is designed to provide electricity to a factory. The electricity comes from three substations, each of which can contribute a maximum amount of 1 MW (Megawatt). To maintain its normal functions, the factory needs a total amount of 2.5 MW of electricity. Due to the advance technology of distributed power generations [START_REF] Ackermann | Distributed generation: a definition[END_REF] [START_REF] Blaabjerg | Overview of control and grid synchronization for distributed power generation systems[END_REF], the factory itself can produce 0.8 MW of electricity since it has installed a photovoltaic generator. However, the photovoltaic generator only works when the factory is in normal state, since it requires power supplies and control commands from a control center, which is located in the factory. If the factory does not receive enough power from the electricity-distributed system, the control center cannot work, which in turn, ceases the normal functioning of the photovoltaic generator.

The state space of the system can be defined by the numbers of the failed substations:

  0,1, 2,3 . S 
It is easy to verify that when the system is in state 0 , the factory is in working state; when the system is in state 2 or state 3 , the factory is in failure state, regardless of the state of the photovoltaic generator. However, state 1 , which represents that one of the three substations fails and the factory receives 2MW of electricity from the electricity supply system, is a changeable state affected by the system's historical dynamics:

(1) Suppose the system transfers to state 1 from a working state, which means that before entering state 1 , the factory was functioning normally. Hence, the photovoltaic generator is working and generates additional 0.8 MW of electricity. The total amount of electricity the factory could receive is, then, calculated as

    M 2 W 0 2.5 MW .8 2.8 .  
Therefore, the factory can perform its function normally. It means state 1 of the system is a working state.

(2) If, on the other hand, the system transfers to state 1 from a failure state (state 2 or state 3 ), which means that before entering state 1 , the factory was already in failure state, and therefore, the photovoltaic generator cannot work. The factory can, then, only receive total amount of 2 MW electricity, which makes it in a failure state.

Consequently, state 1 in the above electrical power supply system is a history-dependent state. Besides, due to the widely application of Uninterrupted Power Supplies (UPSs) in the factory, the factory can withstand a temporary loss-of-power for no more than 5 minutes. Thus, failures that can be recovered within 5 minutes are treated as neglected failures. In this paper, we develop methods to analyze the reliability and availability of such MRS-HDS with neglected failures.

System descriptions and assumptions

In this section, we present a formal description of MRS-HDS with neglected failures. The system is based on the following assumptions:

(1) The repairable system is described by a continuous-time Markov process { ( ), 0} X t t  with finite state space S .

(2) The observation for the system has two statuses: Up and Down periods. Essentially, the state space S can be divided into three sets, U , C , and D . The state in U stands for functional or Up state, which represents that the system is working. When the system stays in U , we say that the system is in Up period. The state in D stands for failure or Down state, which represents that the system fails. When the system stays in D , we say that the system is in Down period. is neglected: If the previous state belongs to Up period, the system is regarded as in Up period for the whole period of R t ; on the other hand, if the previous state belongs to Down period, then the system is regarded as in Down period for the whole period of R t . Therefore, the rule to judge which period the system stays in is to know what period the previous state belongs to. According to this rule, the illustration of MRS-HDS with neglected failures is shown in Figure 2.

Here, we want to mention that the definition of neglected failure is adopted from the definition in [START_REF] Zheng | A study on a single-unit Markov repairable system with repair time omission[END_REF]. According to [START_REF] Zheng | A study on a single-unit Markov repairable system with repair time omission[END_REF], when the system comes from "up" to "down" with a duration longer than  , the whole duration of repair (or failure) is regarded as in "down". Only when the duration of repair time for the system is no longer than  , the "down" duration will be considered to be "operating" (available). That is to say, when the system comes from "up state" to a long D state, the first  time units will be considered as part of downtime. The reason we made explanation is to distinguish two different understanding for the availability of the first  time units in a long D state. The definition used in our model is based on model II in [START_REF] Cui | Reliability measures for two-part partition of states for aggregated Markov repairable systems[END_REF], which is called the neglected failure model. However, if the first  time units in a long D state is regarded as available, the model will become another one which is same as model III in [START_REF] Cui | Reliability measures for two-part partition of states for aggregated Markov repairable systems[END_REF], called failure delayed failure model.

Modelling MRS-HDS with neglected failures using aggregated stochastic processes

In this section, we use aggregated stochastic processes, a mathematical tool mainly developed in the theory of ion channel, to model the behaviour of MRS-HDS with neglected failures. First, the Markov process in which the finite number of states is partitioned into three sets, U , C , and D has transition rate matrix

UU UC UD CU CC CD DU DC DD       Q Q Q Q Q Q Q Q Q Q . ( 1
)
Then, a semi-Markov process can be defined, which is imbedded at the instants when the system switches between the three groups. The stochastic properties of this semi-Markov process are governed by a transition matrix () t G , which can be partitioned in a similar manner to

Q as ( ) ( ) ( ) ( ) ( ) ( ) ( ) UC UD CU CD DU DC tt t t t tt       0 G G G G 0 G G G 0 , (2)
where

( )= exp( ) tt    G Q Q , provided   , for , U  , C , or D .

Modelling the history-dependent behaviour

Next, from another viewpoint, once the system enters one U state, then the system is sojourn in Up period until one long D duration (duration in D longer than the given value) occurs. More specifically, no matter what transitions happening after that U state, the system is always in Up period as long as no arriving in long D duration.

Similarly, the system is sojourn in Down period once it comes to one D duration longer than the given value. And the Down period will terminate when one U state occurs. The following diagram, Figure 3 can illustrate this viewpoint.

U C D Up Down time t time t … … … … … … … … … … … …  Figure 3
. Alternative Up and Down periods path for the system If we use stochastic process   ( ), 0 X t t  to describe the running path for the system, then another stochastic process   ( ), 0 Y t t  , can be seen as the results of our observation. That is whether the system is in Up or Down period. The stochastic process  

( ), 0 Y t t  is an aggregated stochastic process.
If we denote the Up period as W , Down period as F , then the properties of this system may be described by considering a semi-Markov process. That is to say, the aggregated stochastic process  

( ), 0 Y t t  is a semi-Markov process.
The semi-Markov process has a density kernel, which will be denoted by

() () () e e WF e FW t t t      0G G G0 . ( 3
)
For some purposes, it is useful to have their Laplace transforms:

* * * () () () e e WF e FW s s s      0G G G0 . ( 4
)
If we ignore the durations of the sojourns and merely consider the sequence of states that are occupied at the start of each sojourn, we have an embedded Markov chain with transition probability matrix,

         0G G G G G0 . ( 5
)
Here we use the convention of dropping the star and the argument to denote a Laplace transform evaluated at 0 s  , which is the integral of the original function over the interval (0, )

 .

Modelling the neglected failures

Taking neglected failure into consideration, we need to distinguish short (less than  ) and long (greater than  ) sojourns in D . So in the manner of Hawkes et al. [29], we break up the Laplace transform of

         Q Q I I Q I Q Q S G , 1 * * exp( ) exp( ( ) )( ) ( ) ( ) st DD DC DD DD DC DD DC e t dt s s s s           Q Q I Q I Q Q L G
, where * ( ) exp( ( ) )

DD DD ss     L I Q , ( 6
) * ( ) exp( ( ) ) DD DD ss      S I I Q . ( 7
)
In order to get more indexes and distributions for the system, we need to make some change ( ), 0 X t t  is still a Markov process. However, it no longer obeys Markov properties when the neglected failure is taken into consideration. We just want to illustrate the possible transitions among the states if we need to distinguish whether the duration in D is longer than  by Figure 5, which makes the transitions rules more clearly. The transition rules are as follows: a) Rules for U states: The bidirectional transitions can happen between U and other states except D C . Because if the system is in U states following by a changeable state, then it must be a changeable-up state. Therefore, there is no transition from U to . Thus, the system can be considered available during states set A . And we regard the system are "good" in states set A , some changeable states and short-failure states.

The properties of this system may be described by considering a semi-Markov process embedded at the time points which form the start of an available state, the start of a changeable-down sojourn, the start of a sojourn in D states. Denote the semi-Markov process as  

( ), 0

Z t t  , so that   ( ), 0 Z t t  is an aggregated stochastic process.
The semi-Markov process   ( ), 0 Z t t  has a density kernel, which will be denoted by ()

( ) ( ) ( ) ( ) ( ) D DD AD DA DC C A C D t t t t tt        0 G 0 G G 0 G G G 0 . ( 8
)
For some purposes it is useful to have their Laplace transforms:

10 * * * * ** () ( ) ( ) ( ) ( ) ( ) D DD AD DA DC C A C D s s s s ss        0 G 0 G G 0 G G G 0 . ( 9
)
If we ignore the durations of the sojourns and merely consider the sequence of states that are occupied at the start of each sojourn, we have an embedded Markov chain with transition probability matrix, * 0 ( ) (0)

D DD AD DA DC C A C D t dt            0 G 0 G G 0 G G G G G 0 . ( 10
)
Here we use the convention of dropping the star and the argument to denote a Laplace transform evaluated at 0 s  , which is the integral of the original function over the interval (0, )

 .

Reliability indexes of MRS-HDS with neglected failures

In this section, we use the model developed in Section 3 to derive some reliability-related indexes for MRS-HDS with neglected failures, i.e., the instantaneous availability, steady-state availability and some sojourn time distributions.

Instantaneous availability for MRS-HDS with neglected failures

According to the definition of availability, we define the availability for a system in traditional sense, which is based on the original model. If the system is in U and up-changeable states, it is regarded available for the original model. However, taking the neglected failure into consideration, the definition of the availability will be extended.

According to our new model, the system is regarded as available in Up period under several situations. Of course, the system is still available when it is in U and up-changeable states. Besides, when the system is in D whose duration is less than the critical value and the state the D moving from is an A state, then the system is still available. In addition, if the system is in C state whose previous state is one short failure duration with an A previous state, then it is in Up period. Therefore, we list all the scenarios that the system is available as follows.

(1) Scenario 1: The system is in U states Apparently, the new system is available when it is in U states, which is shown in Figure 6(1).

… …

U t

Figure 6(1). The system in U states in Scenario 1

We can get the availability in this situation easily, 1 0 ()

t U A t e   Q 1 , (11) where 1
(1, ,1, 0, , 0)

T U S U   1 .
(2) Scenario 2: The system is in U C states According to the description for the system, the changeable states are history dependent, so that the system is available when it is in U C states, which is depicted in Figure 6(2). We denote the avaliability in

U C states as 2 () At, then 2 ( ) 0 0 () CC t h th U UC C A t e e dh     Q Q E Q 1 , ( 12
)
where

UU U SU        I E 0 , 1 (1, ,1) T C C   1
. Note that h in Figure 6(2) means the duration of the system staying in C before time t . The explanation of equation ( 12) is because the system is working at time th  , then making a transition from the set U to set C and staying in C , which is called U C , during the time length h .

C 1 is a column vector of C ones to sum up all the desired probabilities.

(3) Scenario 3: The system is in

S D states

Since if the repair time in D is sufficiently short, then when the failure can be neglected the system is regarded as available. In order to present the availability function, denoted as

 QQ Q E Q 1 , (13) where 1
(1, ,1)

T D D   1
. Note that s in Figure 6.(3a) means the duration of the system staying in S D before time t . For equation [START_REF] Bao | An analysis of availability for series Markov repairable system with neglected or delayed failures[END_REF]

Q QQ Q E Q Q 1 (14)
Note that in Figure 6.(3b)

u stands for the duration of the system staying in S D before time t and v means the duration of the system staying in U C .

The explanation of equation ( 14) is: The durations that the system stays in

U C is v , in S D until
Q Q Q QQ Q Q QQ EQ QQ Q Q 1 ( 15
)
 Subcase 2 of case 3 for Scenario 3: the previous state to the first

S D state is U C , denoted as 3 (2) () c
At , which is depicted as figure 6(3c)-2.

() 3 (2) 0 2 2 1 1 1 1 ( ) ( ) [ ( )] [ ( )]
.

kk ii i i CC k DD k DD CC k DD k DD CC DD DD t u v vu c U UC CD k k vu DC CD v u DC CD D k k k k A t
Q QQ Q QQ Q Q QQ E Q Q QQ Q Q 1 (16)
And,

3 3 (1) 3 (2) ( ) ( ) ( ) c c c A t A t A t . (17)
Thus,

3 3 3 3 ( ) ( ) ( ) ( ) a b c A t A t A t A t    . ( 18
Q Q Q E Q Q 1 . ( 19
)
Note that in Figure 6.(4a)

u stands for the duration of the system staying in D C before time t and v means the duration of the system staying in S D . For equation [START_REF] Colquhoun | On the stochastic properties of bursts of single Ion Channel openings and of clusters of bursts[END_REF], it is because that the system has a transition from U to D at time t u v  and then spends v stays at D ( v is shorter than  ). Then, the system transfers from

CC CC DD t t u t u v wu v b t u v w U UC CD DC C
A t e e e e dw dv du

                QQ Q Q E Q Q Q 1 . (20)
The explanation of equation ( 20) is similar to that of equation ( 16), adding the consideration of the system has a transition to

() 4 (1) 0 2 2 1 1 1 1 () [] [ ] . kk ii i i DD k CC k DD k CC k CC DD t u v uv c U UD DC k k uv CD DC v u CD DC C k k k k A t e e e
ee e e du dv du dv du dv

            Q QQ QQ Q Q E Q Q QQ Q Q 1 (21)
() 4 (2) 0 2 21 1 1 1 1 1 () [] [ ]
.

kk ii i i CC k DD k CC k DD k CC k CC DD t u v v c U UC k k u v u v CD DC CD DC v u CD DC C k k k k k A t
                  Q Q Q Q Q Q Q Q EQ Q Q Q Q Q Q 1 (22) Theoretically, 4 4 (1) 4 (2) ( ) ( ) ( ).
c c c A t A t A t  (23)
Similar with that in Case 3 for Scenario 3, all the expressions for 4 () c At will turn to be tiny when computation is done, which will be demonstrated in the numerical example. Thus,

a b c A t A t A t A t    . 4 4 4 4 ( ) ( ) ( ) ( )
Finally, we can get the instantaneous availability for the system as follows,

1 2 3 4 ( ) ( ) ( ) ( )
( ) A t A t A t A t A t     , (25)
and the matrices in this formula have been already obtained in the previous section.

Steady-state availability for MRS-HDS with neglected failures

According to the definition, the steady-state availability can be got from the limitation of instantaneous availability, when time t approaches infinity.

We denote the steady-state availability as () A  , and ( ) lim ( )

t A A t   . ( 26
)
Therefore, after we derive the function for instantaneous availability, we take the limit to get the steady-state availability for the system. It will be shown in the numerical example section later.

Some sojourn time distributions for MRS-HDS with neglected failures

Based on the analysis in Section 3.1, we can present the transition matrix Q for the new Markov process   ( ), 0

X t t  , U U U U U D D D D D UU UC UD C U C C C D C U C C C D DU DC DD         Q Q 0 Q Q Q 0 Q Q Q 0 Q Q Q 0 Q Q . ( 27
)
Since we have defined that

U A U C  , from matrix Q we can get U U U U UU UC AA C U C C      QQ Q QQ , U UD AD CD     Q Q Q ,   U DA DU DC  Q Q Q , DD DD  QQ . (28) And * 1 ( ) ( ) AD AA AD ss   G I Q Q , (29) * 1 ( ) ( )
DA DD DA ss   G I Q Q . ( 30
)
According to what has been discussed in Section 3.1 about Up and Down periods for the system, combining with the meanings of the semi-Markov process, the elements in the density

e WF s G G G G G G G        G , where * * * 0 ( ) ( ) (0) UC CD DD G s s L  GG , * * * * * * * 0 1 ( ) ( ) ( ) ( ) (0) ( ) (0) r UC CU UC CD DD UD DD r G s s s s L s L             G G G G G , * * * * * * * * 0 2 ( ) ( ) ( ) ( ) ( ) (0) ( ) (0) r UD DD DU UC CD DD UD DD r G s S s s s s L s L             G G G G G , * * * * 0 3 ( ) ( ) ( ) ( ) r UC CD DD DC r G s s S s s       G G G   * * * * * * * * ( ) ( ) ( ) (0) ( ) (0) ( ) (0) CU UC CD DD UD DD CD DD s s s L s L s L      G G G G G , 20
r DD DC CD DC UC CD DU r L s s s s s s s             G G G G G G . ( ) ( ) ( ) ( ) ( ) ( ) ( )
Next, some distributions for the Markov history-dependent repairable system can be obtained as follows.

(1) The distribution for the k th working period from the beginning up states We suppose the system is in up states (U states) at the beginning. Let ( 1) k W   be the state occupancy probability vector at the start of the k th working period,

  1, 2,... k  .
Then, the probability density function for the duration of the k th working period is given by [START_REF] Liu | A cold standby repairable system with working vacations and vacation interruption following Markovian arrival process[END_REF] () (2) The distribution for the k th failure period Correspondingly, if we focus on the failure period (Down period of the system) instead of working period (Up period of the system), the probability density function for the duration of the k th failure period is given by [START_REF] Liu | A performance measure for Markov system with stochastic supply patterns and stochastic demand patterns[END_REF]. Consequently, the probability density function for the duration of the k th failure period can be obtained.

e k WF WF t   G1 , (31) where
( ) ( 1)
(3) The distribution for the real working time in an effective working period

In fact, there is some time the system is not at working as the short failure is neglected for the Markov history-dependent repairable system. Thus, we can consider the real working time in a working period.

The Laplace transform of the probability density function for the real working time in the k th working period is

* * * ( 1) * * 0 ( ) (0) (0) ( ) (0) r k AD DA AD A DD DD D r s S s L         G G G 1 , (33) Where ( 1) k A
  can be got from the k th working probability vector  

,

k k k W A W A         . D 1 is a column vector of D ones.
Hence, the probability density function can be obtained after doing inverse Laplace transform and normalazition.

(4) The distribution for the neglected failure time in an effective working period Since the short failure in a working period is neglected, we also concern the distribution for the neglected failure time in the k th working period.

The Laplace transform of the probability density function for the neglected failure time in the k th working period is

* * * ( 1) * * 0 (0) ( ) ( ) (0) (0) r k AD DA AD A DD DD D r S s s L         G G G 1 .
Finally, Inverse Laplace transform and normalazition are needed to do as well.

Numerical example

  ( ), 0 X t t  : 3 1 0 2 3 4 0 1 1 0 2 1 3 0 2 5            Q . ( 34
)
To calculate the instantaneous availability, we first calculate the expressions for At involve the continuous and repeated transitions between specific states, whose probability, compared to other terms in ( 18) and ( 24), is negligible. The results are presented by the solid line in Figure 7. Monte Carlo simulations are conducted to verify the derived analytical expressions. We equally divide   into 500 sub-intervals and use 4 10 random samples to calculate the availability at each sub-interval. The results are also given in Figure 7. From the comparisons, it can be seen clearly that the derived equations accurately describe the availability of the system. ( ) 0.6739.

A 

Next, we calculate the distribution of some sojourn times. For illustration purposes, we only calculate the distribution of the real working times in an effective working period. Other distributions discussed in Section 4.3 can be achieved in a similar way. From the values of matrix Q in equation (34), we can get matrices in equation ( 28

Conclusions

In this paper, the theory of aggregated stochastic processes is employed to develop reliability analysis methods for Markov Repairable System with History-Dependent up and down States (MRS-HDS) considering the effect of neglected failures. A new model for MRS-HDS with neglected failures is developed first, based on the theory of aggregated stochastic processes, which allows for the explicit consideration of both history-dependent states and neglected failures in repairable system models. Based on the developed model, the instantaneous and steady-state availabilities of MRS-HDS with neglected failures are derived. Also, some reliability-related time distributions are obtained, including, distribution for the k th working period, distribution for the k th failure period, distribution for the real working time in an effective working period, distribution for the neglected failure time in an effective working period. All the derived reliability indexes can be expressed in closed-form, which provides an effective and comprehensive way to describe the reliability of MRS-HDS with neglected failures.

In the future, more versatile reliability indexes might be defined and derived so that the behavior of system reliability from different aspects could be better captured. Moreover, considering the influence of system configuration, more system structures can be investigated to achieve a more comprehensive understanding of reliability analysis of MRS-HDS, e.g., series systems, parallel systems, coherent systems and so on.

Figure 2 .

 2 Figure 1. A illustration of changeable states

. 3 . 3 Figure 4 .D

 334 Figure 4. The transition diagram for the new Markov process According to the assumptions of the model, D can be decomposed into two parts:

Figure 5 .

 5 Figure 5. The decomposed transition diagram for the new system Note that the dashed line in the transition diagram presents the difference between our new model and the original model. The new stochastic process  

C

  for D states shorter than  : The bidirectional transitions can happen between S D and other states except L D . What should be mentioned here is the special transition from S D to D C . It happens when the state before S D is a Down state, then this S D is still regarded as in Down period. Consequently, the following changeable state is supposed to be a Down state, which is D C . In addition, there are no transitions between S D and L D . c) Rules for D states longer than  : The transition rules of L D are similar as that of S D .It is impossible that the system runs from for changeable states C : The transitions can occur between changeable states and other states, except that transitions from U to are not allowed.Then based on the new Markov process   ( ), 0 X t t  , a new perspective to analyze the system in view of the state space as S of   ( ), 0 X t t  , we define that U A U C 

Figure 6 ( 2 )

 62 Figure 6(2). The system in

Figure 6 (

 6 Figure 6(3a). The previous state to



  Subcase 1 of case 3 for Scenario 3: the previous state to the first S D state is U , denoted as 3 (1) () c At , which is depicted as figure 6(3c)-1.

Figure 6 .

 6 Figure 6.(3c)-1. The previous state to the first

Figure 6 .

 6 Figure 6.(3c)-2. The previous state to the first S D state is

Figure 6 .

 6 Figure 6.(4b). The previous state to

  handled similarly with 3 () c At . It is again divided into two subcases, the explanation can be reffered to that in Case 3 for Scenario 3.  Subcase 1 of case 3 for Scenario 4: The previous state to the first S D state is U , denoted as 4 (1) () c At , which is depicted as figure 6(4c)-1.

Figure 6 .

 6 Figure 6.(4c)-1. The previous state to the first



  Subcase 2 of case 3: The previous state to the first S D state is U C , denoted as 4 (2) () c At , which is depicted as Figure 6(4c)-2.

Figure 6 .

 6 Figure 6.(4c)-2. The previous state to the first

F 1

 1 is a column vector of F ones.From eqution[START_REF] Wang | Reliability Evaluation for Multi-State Markov Repairable Systems with Redundant Dependencies[END_REF], we can know the Laplace transform of () can get the probability density function for the duration of the k th working period by computation of inverse Laplace transform and normalazition.

  the equations obtained in Section 4.1. It should be noted that we only consider the first two terms in the expressions for

Figure 7 .

 7 Figure 7. The instantaneous availability for the MRS-HDS with neglected failures compared with the result of simulation

.

  critical value is 1   . From equations (6) and (7), we can get the expression Based on equation (29-n33), we calculate the real working times in the first working period with 1 k  ,

Figure 8 .

 8 Figure 8. The probability density function for the real working times in the first working period

  The state in C represents changeable state.(3) The changeable states in C could be observed to be either in Up or Down period, depending on the system's historical state-transition trajectory: If the last non-changeable state prior to the changeable state is in Up period, then the changeable state is classified as an Up state,

	referred to as changeable-up states, denoted as	C . The system will be in Up period when it
				U
	stays in	C . If, on the other hand, the last non-changeable state prior to the changeable state
		U
	is in Down period, then the changeable state is classified as a Down state, referred to as
	changeable-down states, denoted as	C . The notations for changeable states are the same as
				D
	that in [8], which is illustrated in Figure 1.
	(4) If the repair time in D , denoted by	R t , is longer than a predefined threshold  ,	R t is
	still regarded as in Down period; if, on the other hand,	R t is less than  , failures in D

  A numerical example is presented in this section to demonstrate the methods developed in previous sections. For simplicity, we assume that there is only one state in each set U , C , and D . Assume the transition rate matrix for the original Markov process describing the repairable

						3 1 2  
						Q	4 3 2 6 2 5      	,
	and we assume that	 	0.1	and that the initial state of the system is an Up state, so that the initial
	probability vector is	0  	 1, 0, 0		.
	Next, we duplicate the changeable state to yield the transition rate matrix for the aggregated
	Markov process
	system is

Acknowledgments

The research of Dr. Shijia Du is supported by the National Natural Science Foundation of China (No. 71601010) and China Postdoctoral Science Foundation (No. 2016M590030). The research of Dr. Zhiguo Zeng is supported by National Natural Science Foundation of China (61573041 and 71671009). The research of Professor Rui Kang is supported by National Natural Science Foundation of China (61573041).