Amadou Alexandre Boyer
email: alexandre.boyer@insa-toulouse.fr

Ndoye Cisse

Sonia Ben Dhia
email: sonia.bendhia@insa-toulouse.fr

Laurent Guillot
email: laurent.guillot@freescale.com

Bertrand Vrignon
email: bertrand.vrignon@freescale.com

A Cisse Ndoye

Characterization of the Evolution of IC Emissions after Accelerated Aging

Keywords: conducted emissions, electromagnetic compatibility, integrated circuit testing I

établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

coupled with reliability tests to check EMC in harsh environments. EMC research projects mainly focus on low emission and susceptibility design techniques, while little attention is given to the real-life environment of electronic devices (e.g., effects of temperature or electromagnetic environment [START_REF] Armstrong | Specifying lifecycle electromagnetic and physical environments -to help design and test for EMC for functional safety[END_REF] [START_REF] Ben Dhia | Thermal influence on 16-bits microcontroller emission[END_REF]) including the impact of natural aging [START_REF] Parker | The case for combining EMC and environmental testing[END_REF], which can degrade both functionality and the electromagnetic behaviour of an integrated circuit. The evolution over time of emission and susceptibility levels and the relevance of the aging margin used to compensate for the drift of EMC levels are not evaluated during qualification phases. Although these topics are crucial for ensuring sufficient long term safety, EMC standards give no guidance. Different expert groups have identified gaps in usual EMC qualification procedures and pointed out several issues such as the life profile conditions, process dispersion, or aging. In [START_REF]The IET 2008 Guide on Electromagnetic Compatibility for Functional Safety[END_REF], many guidelines are given to improve EMC test plans and choices of EMC mitigation techniques to guarantee the functional safety of electronic systems, e.g., by taking into account actual environmental and physical conditions of devices during EMC tests and their whole lifecycle.

In this context, a new request from electronic equipment suppliers aimed at ensuring the electromagnetic robustness (EMR) of embedded systems appeared recently with the emphasis on going further in electromagnetic behaviour improvement at the IC level, by taking aging factors into account [START_REF] Pfaff | Industrial use of EMC: trends in system development[END_REF]. This concept is an extension of EMC for the full lifetime of a product in order to improve safety and long term reliability. This paper presents an original study of the impact of circuit aging on electromagnetic emissions, in order to provide better insight into the impact of intrinsic circuit degradation due to natural aging on electromagnetic emissions, which can justify the usefulness of further EMC level margins or EMC mitigation techniques. One case study based on a commercial off-the-shelf IC dedicated to automotive applications is proposed. Its emission spectrum is compared prior to and after applicative and accelerated-life tests according to reliability standards. Statistical analyses are performed to identify and clarify the effects of aging on the evolution of parasitic electromagnetic emission.

In the first section of the paper, common IC intrinsic failures are reviewed and the links between circuit reliability and EMC issues are analyzed to highlight the need to characterize EMC on aged components. We then describe the proposed

Characterization of the Evolution of Integrated

Circuit Emissions after Accelerated Aging Alexandre Boyer, Amadou Cisse Ndoye, Sonia Ben Dhia, Member, IEEE, Laurent Guillot, Bertrand Vrignon E methodology for characterizing the evolution of emissions, the component under test, the emission measurement set-up, and the aging tests. In the fourth section, emission measurement results obtained before and after the different aging tests are compared. Resulting emission drifts are analyzed. In the final section, some hypotheses are proposed to explain the drifts of emission level.

II. CONSEQUENCES OF INTEGRATED CIRCUIT AGING ON ELECTROMAGNETIC COMPATIBILITY

A. Reliability Issues in CMOS Circuits

During their lifetime, components are affected by intrinsic failure mechanisms such as electro-migration, gate oxide breakdown, or hot carrier injection [START_REF] White | Microelectronics Reliability: Physics-of-Failure Based Modeling and Lifetime Evaluation[END_REF] [10], mainly activated by harsh environmental conditions such as high or low temperature and electrical overstress. Table I sums up the main failure mechanisms which may affect MOSFET transistors and their characteristics. The most common intrinsic failures of CMOS circuits induce charge trapping and breakdown in gate oxide, and migration in the metal levels, as illustrated in the cross section of a CMOS device presented in Fig. 1. The degree of degradation of a device and, hence, its lifetime depends on the stress level and duration. [START_REF] Gourdon | ASTE[END_REF] Failure mechanism

Cause of failure

Failure mode Acceleration factor

Electromigration

Gate oxide breakdown

Hot carrier Injection

Charge trapping (NBTI) B. Accelerated Life Tests Accelerated life tests consist in applying a high level stress condition (e.g., high temperature or high voltage) for a short period in order to accelerate the damage rate for relevant wearout failure mechanisms [START_REF] Jensen | Electronic Component Reliability[END_REF]. If the acceleration factor (AF) related to a degradation mechanism and an environmental condition is known, then the lifetime of the device under test can be extrapolated, e.g., by using the Arrhenius Law for thermal acceleration (1), as explained in Fig. 2:

N+ N+ P+ P+

Oxide integrity

Mobile ion contamination

                - × = 2 1 1 1 exp T T K E AF A (1)

C. Effect of Intrinsic Failures on CMOS Device Characteristics

Various studies have shown that the first failures affecting a component appear sooner for nanometric technologies than for older technologies [START_REF] Srinivasan | The impact of technology scaling on lifetime reliability[END_REF]. Even if functionality is often preserved especially in digital circuits [START_REF] Kaczer | Impact of MOSFET gate oxide breakdown on digital circuit operation and reliability[END_REF], many circuit characteristics such as operating frequencies, current consumption, leakage current [START_REF] Kaczer | Impact of MOSFET gate oxide breakdown on digital circuit operation and reliability[END_REF], noise [START_REF] Weir | Ultra-thin gate dielectrics: they break down, but do they fail?[END_REF], gate capacitance, or gate oxide resistance [START_REF] Sadat | Analysis and modeling of LC oscillator reliability[END_REF] can be noticeably modified. Fig. 3 presents the typical MOSFET transistor parameters affected by degradation mechanisms which induce performance losses in circuits and (2) a simplified equation for the drain current of a MOS transistor in the saturation region based on the BSIM3 model [START_REF] Liu | MOSFET Models for SPICE Simulation including BSIM3v3 and BSIM4[END_REF]:

D. Effects of Circuit Intrinsic Failures on EMC

EMC can also be affected by the intrinsic failures of CMOS circuits. For example, the modification of MOS transistor parameters (Fig. 3) such as threshold voltage or mobility can directly influence IC transient current and thus the IC conducted emissions [START_REF] Senthinathan | Simultaneous switching ground noise calculation for packaged CMOS devices[END_REF]. It can also modify delay propagations within the digital cores of ICs which make them more sensitive to delays induced by electromagnetic interference [START_REF] Chappel | EMI effects and timing design for increased reliability in digital systems[END_REF].

In practice, the effects of component aging, dispersion, or measurement uncertainties are compensated for by additional compatibility margins, which are typically about 6 -20 dB in many applications [START_REF] Bendhia | Impact du vieillissement des composants sur la CEM : du composant au système[END_REF]- [START_REF] Lawton | Systems guidelines for EMC safety-critical circuits: design, selection and margin demonstrations[END_REF], as explained in Fig. 4. However, the effects of IC aging on EMC performance are not clearly identified. This lack of knowledge could have dramatic consequences on system safety. If the emission levels increase, the aged circuit would cease to comply with EMC requirements. The risk that the emission level exceeds the emission limit and causes interference between neighbouring electronic devices increases (Fig. 5). Conversely, if the emission levels decrease, although interference risks are reduced, the initial EMC requirements become too severe and could lead to over-design and unnecessarily high costs.

Component dispersion margin

Safety margin

Component aging margin

Emission level (dBµV)

Probability Emission margin

Before aging

After aging

Maximum limit for emission

Risk that emission level exceeds emission limit increases Fig. 5. Emission level increase due to circuit aging has an impact on the risk that the emission level exceeds the emission limit

Although commercial ICs have to be qualified for both EMC and reliability to ensure customer requirements, these two areas remain totally disjoined and are tested independently. EMC tests are always performed on "fresh" components, so the effects of aging on EMC performance cannot be accurately evaluated and additional margin and mitigation EMC techniques may be justified. In the following sections, the conducted emission level of a circuit is characterized before and after aging tests to ensure accurate measurements of the drift of emission spectrum induced by IC degradation mechanisms.

III. EXPERIMENT DESCRIPTION

A. EMR Qualification Methodology

In order to exhibit aging effects on emission, emission spectrums were compared before and after an accelerated-life test (Fig. 6). The EMR procedure must be applied to a sufficiently large number of samples to obtain an accurate estimation of EMC variation due to aging, because measurements done on only one sample cannot be representative of the characteristics of a whole device family. The measured drifts between samples can have several origins:

Measurement repeatability for a given sample; EMC-level dispersion due to differences or process variations between tested samples; EMC-level global drift due to technological intrinsic degradation effects.

Measured drifts are consistent only if they are greater than measurement repeatability and are independent of time. Like the global EMC-level drift due to aging, process variations can also be affected by aging and degrade EMC performance. In order to isolate the drift caused by the accelerated aging from the process variations and measurement repeatability, the experimental procedure described in Fig. 7 is used. A reference part allows measurement repeatability to be quantified with regard to variations in measurement devices or test benches. EMC characterization is repeated several times on this reference part. All the others parts are then tested once to evaluate the EMC differences due to the component process variations before aging. The different parts are divided into separate batches, dedicated to one or several specific applicative or accelerated-life tests. Finally, all the samples from each batch are tested again and two values linked to the aging effects are extracted. First, comparison of EMC levels of each sample of a given batch obtained before and after aging gives the global EMC drift due to a particular aging test. Second, dispersion of EMC levels of all parts of a given batch is computed to highlight the effect of aging on the process variations. Statistical analyses such as mean drift σ M (3) or standard deviation σ (4) are required to assess EMC level modification induced by aging.

∑ = - = N i i sample M X X N 1 1 σ (3) ( ) 1 1 2 - - = ∑ = sample N i i M N X X σ (4) i X is the emission level of one sample,
X is the mean emission level of all samples, sample N is the number of samples.

Initial EMC Test

EMC Test after Aging

Initial Process Variation

Process Variation after Aging

EMC test repeatability

- + - + - + - + - + - + - + - +

EMC test repeatability

Evidence of EMC drift ? Effect on functional safety

B. Description of the Component Under Test

The circuit tested was an "eXtreme Switch" (E-Switch), a mixed power circuit designed to replace electrical relays, fuses and discrete components within automotive embedded power applications. Like all electronic devices for automotive applications, the E-Switch must comply with severe reliability and EMC requirements. Maintaining EMC compliance during the vehicle's lifetime is critical to ensure passenger safety.

The E-Switch includes a power circuit with four integrated self-protected 15 mΩ TMOS switches and a 0.25µm BiCMOS control circuit for an extended diagnosis system for vehicle front lighting management. The combination of power and digital parts in a low thermal impedance package requires two separate chips within the same Plastic Quad Flat No lead (PQFN) package. The digital part is clocked by an internal 9 MHz oscillator. An internal charge pump is used to produce the high voltage required on the gate of N-channel TMOS. Fig. 8 gives an overview of the internal architecture of the component.

C. Applicative and Accelerated-life Tests

Applicative and accelerated-life tests used for this experiment are typical of those used in the automotive industry and are described by the AEC-Q100 standard published by the Automotive Electronics Council [START_REF]Automotive Electronics Council, Component Technical Committee, Stress test qualification for integrated circuits[END_REF]. They are required to qualify integrated circuits and guarantee the quality and the robustness of automotive applications. Samples of the studied component underwent the different applicative and accelerated-life tests described in Table II. These different tests do not affect all of the circuit under test in the same way. While HTOL usually has a major impact on intrinsic degradations in the silicon die, TC induces mechanical constraints on the package. Both tests are performed in a climatic chamber to control temperature accurately. HTOL and TC test set-ups are described in Fig. 9. ESD and LD tests reproduce the electrical overstress used for the burn-in of the component that can affect ESD and I/O pad structures.

The different circuits are placed on specific boards during reliability tests. They are then mounted on the EMC test board by a socket for the conducted emission characterization, so that only the circuit is degraded during the reliability tests. Therefore, if a deviation of the emission spectrum is detected, only the aged component will be responsible. For reasons of cost and component availability, a batch of 16 samples was studied. Once the measurement repeatability and the process variations had been characterized, the components were separated into four sub-batches to pass the four different reliability tests described in Table II.

D. Conducted Emission Measurement Setups

Fig. 10 describes the E-Switch conducted emission measurement test bench and the nominal operating conditions of the device under test. The measurement is carried out in accordance with the international standard CISPR25 [START_REF]Radio disturbance characteristics for the protection of receivers used on board vehicles, boats and on devices. Limits and methods of measurement[END_REF]. The noise conducted on the VPWR power supply line is characterized in the 150 kHz -80 MHz band. The measured noise corresponds to interference produced by the circuit activity and conducted along the battery cables. A line impedance stabilizer network (LISN) is used to set the line impedance and isolate the power supply source from the device under test.

Bulbs

IV. RESULT ANALYSIS

A. Characterization of Conducted Emissions

Fig. 11 presents the conducted emission spectrum measured on the VPWR supply of a non-aging component between 150 kHz and 80 MHz. This measurement is representative of a component without external decoupling capacitors. Only components dedicated to HTOL aging are correctly decoupled so that the emission spectrum level is lower. However, this procedure can affect the accuracy of the characterization of aging effects. High amplitude peaks appear at each harmonic of 9 and 18 MHz related to the activity of the digital part, and also at 26 MHz because of the charge pump activity. Only the higher harmonics of these frequencies are taken into account for this study. The harmonic which appears at 3 MHz is due to a parasitic activity of the component so it is not considered.

B. Characterization of Measurement Repeatability and Process Variations

Table III sums up the results for the characterization of measurement repeatability for the six higher harmonics of the emission spectrum and gives an indication of the resulting dispersion due to the repeatability errors of the measurement system. Conducted emission measurements on the VPWR supply were repeated five times on the reference sample 0. Amplitude measurement errors remain negligible up to 26 MHz and increase from 36 MHz. The mean repeatability for the different harmonics is about 0.45 dB. Frequency measurement errors are null whatever the frequency.

Effects of process variations on emission dispersion before aging were also characterized on all 16 available samples. Table IV sums up the effects of the process variations on the amplitude and frequency dispersions of the six higher harmonics. The frequency dispersion is given in terms of the ratio between the frequency drift and the frequency of the harmonics. Dispersions due to process variations on amplitude are higher than measurement errors, but remain within an acceptable margin, proving that the technological process is controlled.

C. Emission Spectrum Drift After Different Reliability Tests

Fig. 12 presents an example of emission spectrum drift on a sample stressed by a set of load dumps. The comparison between the spectrum envelopes of the fresh and aged samples indicates that emission levels increase (up to 2.3 dB at 72 MHz) after a load dump test. Fig. 13 presents other results concerning the samples stressed by HTOL. The peaks of the charge pump harmonic at 26 MHz measured for the fresh and the aged components were compared; the mean values of this peak for the fresh and aged batches also appear on the graph. The amplitude and frequency of the 26 MHz harmonics of each sample have drifted as a result of HTOL stress effects. Both emission level and performance of the charge pump have been affected by HTOL stress. The four aging tests have contrasting consequences on parasitic emissions. While HTOL and TC contribute to reducing the emission level, ESD and LD induce an increase of parasitic emissions which can reach 2 dB for some frequencies. Amplitude measurements done before and after aging tests indicate that dispersions between components remain fairly stable, showing that aging has a negligible effect on process dispersion.

The mean emission amplitude drift for each aging test was also computed to evaluate the global impact on the evolution of conducted emissions, and these are presented in Fig. 15. They are compared with measurement repeatability and process dispersion. Amplitude drifts after TC and ESD tests are of the same order of magnitude as measurement error dispersions, so their effects on the emission spectrum are not consistent. However, amplitude drifts after HTOL and LD tests are not only higher than variations in measurement repeatability but also higher than process dispersion so that the impact of these tests on the emission spectrum is significant.

Conducted emission measurements taken after aging tests show that the harmonic frequencies also drifted. Fig. 16 presents the mean drift for the 6 main harmonics of the spectrum after the four aging tests. As the operating frequencies of the digital core and the charge pump are set by internal oscillators, this measurement gives an indirect indication of the degradation of the E-Switch silicon die and the internal blocks. Unlike HTOL, which induces a systematic reduction of the harmonic frequencies, TC, ESD and LD tests have only a negligible impact on harmonic frequency variations and, hence, on IC functional and physical degradations. The load dump test had the greatest impact on the IC emission drift. Although the emission level rises, the 0.8 dB emission level increase observed after the load dump test could not have serious consequences on the safety of neighbouring components. However, it reduces the emission margin and thus increases the risk that IC emissions may exceed the emission limit (Fig. 5). For critical applications such as in the automotive sector, it can have more serious consequences since the margin is set to ensure that the failure probability is less than 1 ppm. We can illustrate this issue with the emission measurements taken on the E-Switch. If process variations and measurement repeatability errors are considered as Gaussian distribution functions, the necessary emission margin between the measured emission level and an emission limit can be numerically computed. From the measurements taken before load dump burn-in tests, it was found that a 6.2 dB margin reduces the risk that the emission level exceeds the limit to a probability of less than 1 ppm. As the emission limit remains constant over time, a 0.8 dB increase of the emission level leads to a 152 ppm risk, which is no longer acceptable for critical applications. However, other sources of variations have to be considered, such as environmental and EMC test equipment uncertainties which can be larger than 0.8 dB. In practice, larger margins are required to compensate all these effects and reduce the EMI risk at system level.

V. DISCUSSION OF THE ORIGINS OF EMISSION DRIFT

Investigations about circuit intrinsic failures are necessary to determine clearly how the conducted emissions are affected by applicative and accelerated-life tests. Unfortunately, we could not undertake failure analysis procedures on the circuits. However, amplitude and frequency drift of the harmonics of the emission spectrum given by EMC measurements provides some elements on component degradations that allow us to put forward a hypothesis about the origins of emission drift.

The HTOL test accelerates several CMOS intrinsic degradation mechanisms, such as dielectric breakdown or negative bias temperature instability [START_REF] Gourdon | ASTE[END_REF]. These mechanisms induce degradations at the transistor level that affect carrier mobility and transistor threshold voltage and thus tend to decrease transistor saturation current [START_REF] Liu | MOSFET Models for SPICE Simulation including BSIM3v3 and BSIM4[END_REF]. This effect has a direct consequence on the transient current due to switching activity and on the conducted emissions, since the noise source amplitude is reduced.

Although TC induces mainly mechanical strains on the package and has a minor effect on CMOS degradation mechanisms, it contributes to a small reduction in the level of parasitic emissions. IC emissions are not only dependent on the IC switching activity and generated transient current but also on the noise propagation through the power supply decoupling network. The reduction in emission amplitude measured after the TC test can be explained by the degradation of the power decoupling network and the resulting increase of parasitic serial resistance of power supply package pins.

ESD and LD tests produce the strongest effects on parasitic emissions since they tend to increase the emission level. They can induce severe degradation on ESD protections and I/O pads but do not accelerate degradation mechanisms in the IC core, as shown by the frequency drift measurements. The electrical overstress applied on power supply pins could affect the circuit power decoupling network. All the ESD protection structures placed between power and ground rails act as an equivalent capacitive load and thus as an internal decoupling capacitance. ESD and load dump could degrade this capacitance, creating a resistive path between power supply and ground and hence increasing the emission level. Table V summarizes the consequences of aging tests on parasitic emissions and the hypothesis about the origins of emission drift. A quantitative prediction of emission level variations will require physics-of-failure based modeling [START_REF] Pfaff | Industrial use of EMC: trends in system development[END_REF], which is able to predict accurately degradation impact at transistor level and the effects on circuit performances.

VI. CONCLUSION

IC aging has serious consequences on electrical characteristics, which could lead to electromagnetic emission modifications. Therefore, aged circuits embedded within critical electronic systems could increase the electromagnetic interferences level and thus increase the risk to disturb nearby sensitive ICs. Despite the possible dramatic consequences on system functional safety, the impact of the evolution of IC electromagnetic emissions due to aging has rarely been clearly evaluated. This study proposed a characterization of the impact of aging induced by different types of standard applicative and accelerated-life tests on the emission spectrum of a 0.25 µm mixed power device dedicated to automotive applications. After a first round of exposures, some changes concerning the conducted emission level were identified. Although some of these tests induced an increase in the emission level, the measured drifts are not critical for applications if a sufficient emission margin is planned. On the basis of emission measurements, several hypotheses are proposed to explain the observed drifts. Degradation mechanisms related to aging tests affect transient current related to IC switching activity and power supply decoupling networks. These degradations could also affect the noise and delay margins of circuits [START_REF] Fernandez | Effect of oxide breakdown on RS latches[END_REF] and thus the susceptibility to radiofrequency interferences [START_REF] Boyer | Fiabilité des circuits intégrés face aux agressions électromagnétiques[END_REF].

In spite of this qualitative analysis made on one component, a quantitative estimation of EMC level drift and an extrapolation to other circuits in different technologies are difficult without a complete physics-of-failure modelling. This type of modelling can take into account integrated circuit technological parameters and has already been applied to predict performances of functional blocks after stress [27] [28]. However, these approaches are limited for EMC simulation due to the size and the complexity of models. Original prediction methods have to be developed to address this issue.

Fig. 1 .

 1 Fig. 1.Common failure mechanisms in CMOS circuits[START_REF] Segura | CMOS Electronics: How it works, how it fails ?[END_REF]

Fig. 2 .

 2 Fig. 2. Principles of accelerated life test

Fig. 3 .

 3 Fig. 3. Effect of transistor degradations on characteristics of MOS transistor

Fig. 4 .

 4 Fig. 4. Impact of component aging on emission level

Fig. 6 .

 6 Fig. 6. EMR qualification methodology

Fig. 7 .

 7 Fig. 7. Analysis methodology of electromagnetic robustness test

Fig. 8 .

 8 Fig. 8. Internal architecture of the E-Switch

Fig. 9 .

 9 Fig. 9. Climatic chamber for accelerated-life tests

Fig. 10 .

 10 Fig. 10. Power supply conducted emission measurement test-bench

Fig. 11 .

 11 Fig. 11. Conducted emission spectrum measured on VPWR supply of a nonburned component

Fig. 12 . 10 Fig. 13 .

 121013 Fig. 12. Emission level increase after load dump test for sample 10

Fig. 14

 14 Fig. 14 summarizes the effects of the four aging tests on the amplitude drift of the emission level. Histograms describe the mean amplitude drift measured on each sample of the different batches for the six higher harmonics of the emission spectrum. Arrows superimposed on the histograms give the repeatability dispersions in order to evaluate the consistency of measured drifts. For HTOL tests, only the amplitude drifts of three harmonics were characterized because of different experimental conditions compared to other tests.

Fig. 14 .

 14 Fig. 14. Amplitude mean drifts measured at different frequencies for different aging tests

Fig. 15 .

 15 Fig. 15. Emission spectrum mean amplitude drifts after various reliability tests

TABLE I COMMON

 I CMOS INTEGRATED CIRCUIT INTRINSIC FAILURE MECHANISMS AND FAILURE MODE

TABLE II APPLICATIVE

 II AND ACCELERATED-LIFE TEST CONDITIONS[START_REF]Automotive Electronics Council, Component Technical Committee, Stress test qualification for integrated circuits[END_REF]

	Test type	Test name	Conditions	Number of
				samples
	Applicative	Electrostatic	+/-8kV HBM on Power	4
	tests	Discharge	I/Os
	(overstress)	(ESD)	+/-2kV HBM on Digital
			I/Os
		Load Dump	+41V during 400msec	4
		(LD)	Sub-tests:
			Outputs "on"
			Outputs "off"
	Accelerated-	Temperature	Ambient temperature	4
	life tests	Cycling (TC)	cycling from
	(time		-65°C to +150°C,1000
	dependent)		cycles
		High	Ambient temperature	4
		Temperature	+150°C 408 hours
		Operating Life	Typical device operation
		(HTOL)

ACKNOWLEDGMENT

The authors would like to thank Professor Etienne Sicard for his valuable comments and suggestions.


David Bouget

Florent Lalys

Pierre Jannin

Surgical Tools Recognition and Pupil Segmentation for Cataract Surgical Process Modeling

Keywords: Computer vision, video analysis, Bag-of-words, mathematical morphology

In image-guided surgery, a new generation of Computer-Assisted-Surgical (CAS) systems based on information from the Operating Room (OR) has recently been developed to improve situation awareness in the OR. Our main project is to develop an application-dependant framework able to extract highlevel tasks (surgical phases) using microscope videos data only. In this paper, we present two methods: one method to segment the pupil and one to extract and recognize surgical tools. We show how both methods improve the accuracy of the framework for analysis of cataract surgery videos, to detect eight surgical phases.

Introduction

In the context of surgical process modeling, being able to automatically retrieve lowlevel information from the Operating Room (OR) and then extract high-level tasks from these data is a growing need. In previous works [START_REF] Lalys | Automatic phases recognition in pituitary surgeries by microscope images classification[END_REF][START_REF] Lalys | An application-dependant framework for the recognition of high-level tasks in the OR[END_REF], authors developed an application-dependant framework automatically able to extract surgical phases from microscope videos. They first extracted visual cues for each video frame using imagebased analysis. Each frame is therefore composed of binary information forming a semantic signature. These time series are finally used as input for analysis and classification using either Hidden Markov Model or Dynamic Time Warping approaches. As outputs of these two time series analyses, a sequence of surgical phases is proposed.

In this paper, we present two methods: one method to segment the pupil and one to extract and recognize surgical tools. We show how both methods improve the accuracy of the framework for analysis of cataract surgery videos to detect eight surgical phases. Validation studies were performed with a dataset of twenty cataract surgeries.

Materials and Methods

Surgical Tools Recognition

In order to accurately model a surgical process, a vital information needing to be retrieved is the presence of instruments in the surgical field of view. Detecting and recognizing surgical tools can be relatively complex with image-based analysis because they have quite similar shapes. Moreover, they never appear with the same orientations, scales or under the same illumination. The method developed, based on machine learning (see Figure 1), was automatically able to detect any instrument appearing in the video and is composed of three steps:

• Segmentation • Description • Classification
For the segmentation step, the goal was to extract from the image as many regions of interest as surgical tools existing in the image. The better the regions of interest around the tools are, the better the identification will be. This step was based upon the fact there is a distinct color difference surgical tools and the image background. To do so, pre-processing operations were first performed on the input image to create a black and white mask containing all loud outlines. Those operations were respectively: a Gaussian blur transform, a Laplacian transform, along with threshold and dilatation operations. We then refined the mask by applying a connected component method (8 connexity) in order to remove every too small component. Indeed, we can assume that small components can't be outlines of tools and so are considered as noise. From this clean mask, containing only loud outlines, we retrieved the two largest remaining connected components. Indeed, no more than two instruments can be present at a same time within the surgical scene. Lastly, we applied separately each connected component mask extracted on the input image in order to obtain two regions of interest (see Figure 2). Those regions were the most likely to contain a surgical tool. For the description step, the goal was to provide a robust and reproducible way to describe each region of interest previously segmented. Those descriptions were mandatory in order to subsequently perform the classification. Given the fact that the goal of this method was to identify surgical tools, we were willing to extract features invariant to scale, rotation, illumination variations.

In order to create a global description, we first need to obtain locals descriptors (or: key points). We have compared four local features detection methods: Harris [START_REF] Harris | A combined corner and edge detector[END_REF], SIFT [START_REF] Lowe | Object recognition from scale-invariant features[END_REF], SURF [START_REF] Bay | SURF: Speeded Up Robust Features[END_REF], STAR [START_REF] Agrawal | Center surround extremas for realtime feature detection and matching[END_REF]. All of these methods provide a sample of key points, but those key points don't have the same behavior regarding the previously described variations. After detection, a key point (describing a patch of the image) was expressed in a formal way. It is represented by a descriptor vector whose length is variable and depends on the chosen method. Here again, among others, two descriptors methods are generally used: SIFT and SURF descriptors. A SIFT descriptor vector is of length 128 and a SURF descriptor vector is of length 64. In the study, we have decided to use SURF key points as long as SURF descriptors because of the good trade-off between accuracy and computing time.

Using those local descriptors, a global descriptor for a region of interest will be created. To do so, the bag-of-visual-words approach [START_REF] André | Endomicroscopic video retrieval using mosaicing and visual words[END_REF], representing an image as an orderless collection of local features, is used. The bag-of-words model can be defined as an histogram representation based on independent features. This approach usually includes following three steps: feature detection, feature description and codebook generation. From a collection of images, a representative set of patches (or key points) is selected and each is transformed into a description vector. This set of vectors characterizes objects appearing in the images collection. Each vector is called a word and the whole set is called a vocabulary or codebook.

To obtain the global descriptor, the image (represented as a bag of key points) is expressed in the vocabulary space. This bag of key points is expressed as an histogram recounting the number of occurrences of each word in the image.

For the classification step, a database containing the occurrence histograms of each surgical tool needing to be found was used. A smooth classification was then performed by applying a k-nearest neighbor algorithm (k=20). At the end of the classification, for a region of interest, we had its probabilities to contain respectively every surgical tool of the database. A class ''not a tool'' is also considered and called ''background class''.

Pupil Segmentation

In the context of cataract surgery, a step of pupil segmentation can turn out to be very useful to identify some specific visual cues. Those more specific visual cues inside the pupil will improve surgical phases recognition.

This procedure can be divided in three parts and is based upon the assumption of a color difference between the pupil and everything else. The first part led to the creation of a black and white outline mask from the input image, which has been expressed, into the YUV color space. This mask was computed by smoothing, thresholding and performing morphological operations to the input image. Then, we tried to determine circles through the mask using the Hough transform. Sometimes, incomplete circles outlines in the mask may occur, leading to Hough circle detection failure. To tackle the problem, an iterative search was performed on the mask to identify the most probable circular zone. This search was based both on pixel counting and circle radius assumption. Finally, a normalization was performed. We kept the circle center found previously and we set the region of interest radius to a constant value. Following this procedure (Figure 3), a region of interest around the patient pupil has been retrieved.

Validation Studies

In this study, twenty cataract surgeries performed by three different surgeons from the University Hospital of Munich were available. They were recorded using the OPMI Lumera surgical microscope (Carl Zeiss) with an initial resolution of 720x576 at 25fps. In order to speed up processing, each video has been down-sampled to 1 fps and each frame has been spatially down-sampled by a factor 4 with a 5x5 Gaussian kernel, leading to a final resolution of 360x288. Eight surgical phases were defined (Figure 4). As mentioned in the introduction, this work is a part of a higher-level study [START_REF] Lalys | Automatic phases recognition in pituitary surgeries by microscope images classification[END_REF][START_REF] Lalys | An application-dependant framework for the recognition of high-level tasks in the OR[END_REF]. The aim of this framework is to automatically detect phases of a cataract surgery. Each video of the database has been labeled by surgeons, where they have defined the phases' transitions.

The segmentation procedure has been individually and manually validated over the entire video database by testing each frame of each video. The pupil was considered correctly segmented if it was mainly in the segmented region of interest.

For the surgical tools recognition, we first compared the four local key points detection methods along with the optimal number of words. This test was necessary to optimize the bag-of-words approach. Then, we studied results from this module independently from the global framework. To do so, we manually labeled each frame of each video and compared recognized tools and labeled tools. We defined seven classes for the classification, six for the surgical tools and one for everything that is not a tool (also called background class). Each class has been built with 100 representative images.

Finally, we compared results from the whole surgery modeling framework (i.e. detection of the surgical phases) with and without these two modules. The pupil segmentation has been integrated in order to improve the detection of the pupil color shade through histogram-based approach. With this new approach, the recognition of particular color shade within the pupil is now easier. The recognition of surgical tools has been added to the global framework by directly detecting surgical instruments of the surgery. New visual cues were created, corresponding to each instrument to be detected, increasing the number of information composing each semantic frame signature.

Results

For the pupil segmentation module alone, the pupil was correctly detected within 95% of all images (in Table 1). Best result was a segmentation accuracy of 99% and worse result was of 78%.

The results of the bag-of-word approach optimization regarding surgical tools recognition are the following. On the one hand, the number of words did not have a lot of influence on the classification accuracy. However, a number of words ranged between 5 and 15 seems to be the best. The choice of this number will be highly correlated with the key points detection. On the other hand, there are major accuracy variations depending on the key points/description method combination. Of all the combinations tested, the SURF/SURF one seems to give the best classification accuracy (86% for 12 words).

For our study, we decided to use the SURF/SURF combination along with a 15word vocabulary. Within the videos dataset, a surgical tool has been correctly identified as a tool with an accuracy of 84,1% (in Table 2). Given the fact that the classification returns a probability for each tool, no validation study has been conduct to verify if a found tool as been classify as the correct one.

Finally, we have added the new modules to the framework and we now obtain an overall recognition accuracy of 94.4%, results have been slightly improved (in Table 3).

Discussion and Conclusion

In this paper, we proposed to add two new modules of visual cues detection to a framework able to automatically recognize surgical phases of cataract surgery. Even though the framework was created to be application-dependant, each kind of surgical environment has its own particularities and characteristics. As a consequence, the framework has to be tuned for a specific type of surgery in order to be as competitive as possible.

The pupil segmentation method, composed of image-based analysis, detects a region of interest containing the pupil with an accuracy of 95%. This can be considered as preliminary step in order to detect specific visual cues within the pupil. In order to avoid any further detection that could be done within the pupil, we decided to define a constant circumference value for every region of interest. As a drawback, the region of interest is not always perfectly centered on the middle of the pupil. Moreover, automatic segmentation turns out to be difficult when there are interferences in the microscope field of view. For instance, sometimes the pupil is not completely in the image or pupil outlines are too distorted. Retractors can also be too wide and fill too much space, or surgeon's fingers can appear in the field of view.

The surgical tool recognition method, as the pupil segmentation one, has been tuned for this type of surgery. Indeed, a training step is required before the utilization of the framework for each surgical tool likely to appear. Results obtained are promising, we obtain 84,1% of good recognition over all the videos. Some tools are easier to recognize because of their bigger size or because of color gradients more important. Surgical tools information slightly enhanced the framework but they can be far more useful for surgery modeling at a lower granularity level. Surgical tool presence can be directed linked to surgeon's activities. As a drawback, connected components obtained during the first stage of the method do not always contain whole surgical tool. This incomplete detection induces lower recognition rates. Moreover, it is quite difficult to build a complete background class so we improved it as much as possible.

To conclude, the addition of these two modules within the framework leads to better cataract surgery phases recognition. Other visual cues could be extracted in order to further improve phases recognition results. However, in future work, it would be more interesting to focus on lower level information. Surgical tools information could be used to detect surgeon's gestures and thus extract activities within major surgical phases of surgeries.

Figure 1 .

 1 Figure 1. Process pipeline for the surgical tools detection module.

Figure 2 .

 2 Figure 2. Different detection steps in the tools recognition module. From left to right: input image, mask, final connected components images.

Figure 3 .

 3 Figure 3. Pupil segmentation steps. From left to right: input image, mask, final segmentation (blue circle).

Figure 4 .

 4 Figure 4. Typical images for the eight surgical phases: 1-Preparation, 2-Betaisodona injection, 3-Corneal incision, 4-Capsulorhexis, 5-Phaco-emulsification, 6-Cortical aspiration, 7-IOL implantation, 8-IOL adjustment and wound sealing.

Table 1 .

 1 Mean accuracy, minimum and maximum of the pupil segmentation over the entire video database.

		Accuracy (Std)	Minimum	Maximum
	Detection	95,00%	78,00%	99,00%

Table 2 .

 2 Tool detection percentage with the tool recognition module alone.

		Accuracy (Std)	Std
	Detection	84.1%	8.6%

Table 3 .

 3 Percentage of surgical phases correctly recognized with and without the new modules.

		Average (%)	Std (%)	Min (%)	Max (%)
	Without modules	90.2	8.4	78.1	99.9
	With modules	94.4	3.1	90.6	99.9

Corresponding Author; E-mail: david.bouget@irisa.fr.

Acknowledgments

The authors would like to acknowledge the financial support of Carl Zeiss Surgical GmbH, as well as Dr. L. Riffaud for his help on the clinical aspect.