Alexandre Gaffet

Pauline Ribot

Elodie Chanthery

Nathalie Barbosa Roa

Christophe Merle

A Health Monitoring Method for Automotive Surface Mount Technologies

Keywords: AI methods for FDI, Statistical process control, Condition Monitoring, Extreme Value Theory, Gaussian Mixture Method, Process Capability

In this paper, we propose a two-stage online data-based diagnostic method that detects issues in the In-Circuit Test (ICT) equipment from a Surface-Mount Technology (SMT) production line. The first stage performs anomaly detection in a univariate stream of test values. The second stage achieves fault detection and isolation based on the process capability and a Gaussian mixture clustering method. Combining the two stages allows improving the online cost of the second stage and improving the confidence and interpretability of the first stage, anomaly detection. Two solutions are compared for the first stage, anomaly detection by Extreme Value Theory (EVT) and a sliding window method. The comparison is done with an automotive, industrial database and shows that using EVT delivers almost the same performance in detection with less computation time.

INTRODUCTION

Nowadays, the production of electronic boards is mainly done with Surface Mount Technologies (SMT). The increase in product complexity and competitiveness requires limiting the production time losses and the number of faults. To avoid quality issues and false alarms, health monitoring of the testing equipment, present at different locations in the production line, is crucial. Health monitoring is performed with Fault Detection and Isolation (FDI) method. This work focuses on particular testing equipment: the In-Circuit Test (ICT) machine, which tests each component of an electronic board to check its presence and correct test value. This machine is composed of a pneumatic bed-of-nails wired to fit the specific design of each type of product and to test its electronic components. Furthermore, the production of electronic boards uses substrates of similar sizes, called panels. Panels are composed of identical duplicates of the same product, leading to testing one or several products simultaneously. The health monitoring is realized with the components test values provided by the ICT equipment.

Three main challenges are found in this industrial dataset:

(1) The uncertainty of the maintenance operation database that disqualifies the use of a supervised monitoring approach;

Supported by Vitesco Technologies SAS France.

(2) The large number and diversity of components tested by the ICT machine leading to the need for a generalized monitoring approach for all components;

(3) The difference of test values depending on production parameters (testing equipment, supplier of components, the temperature of the oven...) that creates different groups in one component test values.

In [START_REF] Gaffet | Data-driven capability-based health monitoring method for automative manufacturing[END_REF] a FDI method that takes into account all the previous challenges that have already been defined. The method treats each component test separately, mainly because the relation between test values of components changes even if the ICT machine is functioning correctly. The FDI method is based on a clustering stage using Gaussian Mixture Model (GMM) and the process capability index (Cpk) [START_REF] Wu | An overview of theory and practice on process capability indices for quality assurance[END_REF]) a wellknown health indicator in industry. This method detects faults linked to equipment and a batch of products.

The previous FDI method has a relatively important computation cost if used online (for instance, using a sliding window) due mainly to the many different components tests that need to be controlled. The main improvement proposed in this paper is to introduce a first anomaly detection stage in the method to monitor the stream of test values. Then it launches the FDI method, when relevant, limiting the time cost of the global monitoring method.

The anomaly detection method focuses only on the point anomaly category as defined in [START_REF] Chandola | Anomaly detection: A survey[END_REF]:

"if an individual data instance can be considered as anomalous with respect to the rest of data, then the instance is termed as a point anomaly". A FDI method is used to assess the condition and to detect abnormal behavior of the testing equipment and products (electronic boards). In our study, a fault is a collection of component test values with a high probability of having out-of-bounds samples with respect to the design limits. Consequently, the concept of false alarm does not exist in our study. These limits are specified for each component of the electronic boards. The two-stage method is validated on a real-world diagnostic case with a production dataset of ICT test values from electronic boards used in the automotive industry.

Anomaly detection has been an active field in these past years. This article focuses on the streams of univariate time series of point anomaly. Some statistical methods based on hypothesis testing [START_REF] Rosner | Percentage points for a generalized esd many-outlier procedure[END_REF]) and singular value decomposition [START_REF] Mahimkar | Rapid detection of maintenance induced changes in service performance[END_REF]) have been proposed. A lot of the anomaly detection methods rely on assumptions on the distribution of data according to [START_REF] Chandola | Anomaly detection: A survey[END_REF]). These assumptions are, for instance, Normal or Poisson distributions. They are only realistic in the static case and with sufficient knowledge about the data. In the case of streams of data, these assumptions may no longer be true, leading to unexpected cases in practice. Some other methods are based on autoencoders [START_REF] Xu | Unsupervised anomaly detection via variational auto-encoder for seasonal kpis in web applications[END_REF], [START_REF] Mirsky | Kitsune: an ensemble of autoencoders for online network intrusion detection[END_REF]). However, autoencoders require a large amount of data to be fitted and because of that, they are not able to monitor the beginning of the production of a new type of product. [START_REF] Ren | Time-series anomaly detection service at microsoft[END_REF] proposes an approach based on timefrequency metrics. Nonetheless, as there is no periodicity within our test values, this method category does not seem appropriate for our industrial case. Finally, [START_REF] Siffer | Anomaly detection in streams with extreme value theory[END_REF] it proposes two methods based on Extreme Value Theory (EVT). Both methods have no assumption about data distribution and are updated online at each new point. The first method, Streaming Peaks Over Threshold (SPOT), considers stationary stream data. The second, Drift SPOT (DSPOT), deals with a stream of data that can be subjected to drifts. This paper uses the SPOT algorithm in the first stage of our monitoring method to decide when the FDI method has to be launched.

The paper is organized as follows. Section 2 presents an overview of the global two-stage health monitoring method based on test values of components. In Section 3, the anomaly detection method used in the first stage is detailed. Section 4 is dedicated to the application of the proposed method to a real-world case using test values coming from various ICT equipment of a French plant. The method used in the anomaly detection stage is validated by analyzing the computing cost and the number of detected faults using this stage against the use of an online sliding window. To conclude, the last section highlights the results and gives some perspectives.

TWO-STAGE HEALTH MONITORING METHOD

An overview of the proposed two-stage health monitoring method is presented in Fig. 1. This paper focuses on the first stage of the monitoring method, while the second stage has been precisely described in [START_REF] Gaffet | Data-driven capability-based health monitoring method for automative manufacturing[END_REF].

The next sections condense the main concepts of both stages.

The goal of the first stage is to detect a period of interest to launch the second stage of the monitoring method. Two methods are studied and compared for this first stage. The first one is a sliding window method. The second one is a period selection method performing anomaly detection.

As illustrated in Fig. 1, a stream of test values is received, and the method returns a period of interest for the second stage. In the case of anomaly detection, a trigger and the fixed window of previous test values are returned and nothing otherwise. This detection uses the SPOT anomaly detection algorithm presented in Section 3.

The aim of the second stage is to confirm or deny the occurrence of the fault related to the anomaly detected by the first stage of the method. The selected period of interest is first divided into different clusters. Due to a large number of combinations on the operational production points (systems and component batches), it is inefficient to group data samples per combination. Instead, we propose to cluster the samples into meaningful clusters.

A GMM is fitted by the expected maximization algorithm and used for the clustering stage of the FDI method.

A Maximum A Posteriori (MAP) approach is applied for the selection of the best number of clusters, and there are no constraints on the variances and means of the fitted Gaussian models. Nonetheless, in practice, the created clusters do not correspond perfectly to Normal distributions. The best mixture model for our case is not the one that fits the majority of test values but instead the one that creates the most meaningful clusters. Accordingly, the Normalized Entropy Criteria (NEC) is used as the selection criterion for the MAP approach.

The NEC measures the overlapping between the different clusters of the GMM using entropy. A set of K GMM(k), for the component k ∈ [1 : K], are fitted to 1000 test values. This number is chosen as a starting point and further analysis could be made to optimize this parameter.

According to the NEC, the chosen mixture GMM(k * ) is the one with the lower entropy, i.e., less overlapping between clusters. The initialization of the GMM is based on the first k-means iterations and justified in [START_REF] Gaffet | Data-driven capability-based health monitoring method for automative manufacturing[END_REF].

After the clustering stage, a health index is computed for each cluster in the indexes extraction stage. The process capability index Cpk is used because of its high generalization properties for normal processes and according to expert knowledge, the test values of components are supposed to follow a normal distribution. The process capability index Cpk is defined as follows:

Cpk(X) = min U L -µ(X) 3σ(X) , µ(X) -LL 3σ(X) (1)
where X is a sample of points and µ, σ, U L, LL are respectively the mean, the standard deviation, the upper design limit and the lower design limit. The limits U L, LL are defined in the design phase of the products.

During the detection stage, the health index is compared to a threshold fixed by expert knowledge. This health index measures the criticality of each cluster indeed the Finally, the isolation stage is used for each cluster with a Cpk below the fixed threshold. The isolation stage is specific to our industrial case, while the rest of the method remains generic for a data-based monitoring approach. In our industrial case, a fault is related to either the testing equipment or to a batch issue (products). The idea is to compare the values of Cpk between different positions in the panel and other equipment to isolate the fault in the production line. An example of the output provided by this last stage is illustrated in Section 4.

PERIOD SELECTION METHODOLOGY

Anomaly detection based on Extreme Value Theory

This section describes the method used to detect point anomaly in the first stage of the monitoring method. Many methods already exist to find a statistical threshold based on quantile. In order to compute the quantile, two solutions are possible: to assume a distribution or to compute the distribution empirically. Since we want to avoid assumptions about the distribution, the first solution will not be chosen. Besides, if the quantile computation is empirical, the results can be wrong. Indeed, the risk of having out-of-bounds test values is very low compared to the number of products required to estimate the quantile for automotive applications. Moreover, the probability of extreme values can be null in some cases (e.g infrequent anomalies), leading to wrong distribution estimation.

The EVT infers the distribution of the extreme values without making strong assumptions about it. The EVT states that under weak conditions [START_REF] Fisher | Limiting forms of the frequency distribution of the largest or smallest member of a sample[END_REF]), the next extreme events have the same distribution as the original one. These extreme laws are called Extreme Value Distributions (EVD). The EVD are denoted G γ , with γ the parameter of the distribution. They are defined as follows:

G γ : x → exp(-(1 + γ • x)) -1 γ , γ ∈ R, 1 + γ • x > 0 (2)
F θ (x) = P (X -θ > x|X > θ) ∼ 1 + γ • x σ(θ) -1 γ ( 3
)
where γ is a parameter of the EVD and F θ (x) = 1-F θ (x).

and with F = 1 -F and q = F (z q ), we finally obtain the threshold z q :

z q ∼ θ + σ(θ) γ q F (θ) -γ -1 (4)
The SPOT method, proposed in [START_REF] Siffer | Anomaly detection in streams with extreme value theory[END_REF], is designed for streams of univariate data following any stationary distribution. Only the test values between θ and z q are saved, and the anomaly threshold is modified with each new value inside this interval, leading to lower memory consumption. It uses past and current samples, assuming that the extreme values follow a GPD ( 4). An overview of the SPOT algorithm is presented in Fig. 2. Before being able to use this method for monitoring a calibration stage should be performed. We use the first values of the time series to fit the GPD. This function depends on the risk parameter q, given by the user, and returns the anomaly threshold z q . Then, for each new value, if the value X i is larger than the anomaly threshold, an alarm is triggered. If the value is lower than the anomaly threshold z q and larger than θ then the GPD is fitted with the previous set of test values between θ and z q and the new value using likelihood optimization described in [START_REF] Siffer | Anomaly detection in streams with extreme value theory[END_REF]. Then, using (4), the new anomaly threshold z q is computed.

Fig. 2. Overview of the SPOT algorithm

This SPOT algorithm is used for detecting a point anomaly in the stream of test values. The unique parameters of the method that needs to be fixed are the risk q and θ. These parameters are chosen by experts in particular based on fault characteristic knowledge. In Section 4, the results obtained using different risk parameters are compared.

Evaluation metrics

Several metrics are used to evaluate the benefits of the first stage of the monitoring method and to compare different risk parameters. In order to evaluate the efficiency of using EVT for the first global stage (Fig. 1) we compare it against the use of sliding windows. We call M 0 the algorithm using only sliding windows and M q the one using EVT, where q is the risk parameter fixed in the first stage. Four evaluation metrics are proposed.

The first metric corresponds to the Ratio of Detected Faults (RDF) by M q compared to M 0 . The comparison is made over a set of P components tests and n number of sliding windows.

RDF (M q ) = P p=1 n k=1 δ(M q ) k,p P p=1 n k=1 δ(M 0 ) k,p (5)
with δ(M q ) k,p = 1 if M q detects a fault in the sliding window k for the test p 0 else.

The best result for this metric is 1, meaning that all the faults have been discovered by M q .

The second metric is computed test by test and corresponds to the Maximum Ratio of Faults (MRF) in the component tests. It is defined as the number of windows detected as a fault by M 0 over the total number of windows for each test.

M RF p = n k=1 δ(M 0 ) k,p (6)
We defined the Loss (L) as third metric. It corresponds to the relative computation time of the monitoring method that has to be minimized. It includes the anomaly detection time cost of the first stage denoted C 1 and the fault detection cost of the second stage denoted C 2 :

L(M q ) = P p=1 n k=1 λ(M q ) k • C 2,k,p + C 1 (M q ) k,p n p=1 n k=1 λ(M 0 ) k • C 2,k,p + C 1 (M 0 ) k,p (7)
with λ(M q ) k = 1 if M q starts the fault detection method for the sliding window k and test p 0 else. where C 2,k is the computation time of the second stage for the sliding window k. The Loss L is the ratio of the computation time of M q compared to the one of M 0 . L is between 0 and 1 with value 0 if the time cost of the method M q is null.

The last metric corresponds to the Ratio of Window Investigated (RW I) by M q compared to M 0 and is defined as:

RW I(M q ) = n k=1 λ(M q ) k n k=1 λ(M 0 ) k (8)
The metric boundaries are [0, 1] with 0 being the best result possible. Three of the four metrics are used to demonstrate the interest of the first stage in Section 4, the remaining metric (M RF ) is only used to demonstrate the impact of the number of faults.

EXPERIMENTAL RESULTS AND VALIDATION

Experiments description

In order to extract and treat large amount of data, a cloud instance is used with 32 CPUs of 8 GB of memory based on AMD EPYC 7000 with 2.5 GHz processors. For the experiments, the test values of one type of products are extracted for two different ICT equipment. From the 824 component tests, we choose to remove one (to be shown in Section 4.3) since it triggers the algorithm for all analyzed windows. The remaining 823 tested components are kept for experimentation. For each of these tested components and ICT equipment, the two-stage monitoring algorithm is launched using as parameter Cpk = 1.3, we also add a minimum size of cluster of 10 samples to be used.

The real dataset has two categories of faults. The first category corresponds to faults of the ICT equipment. It can be identified because for identical cards the same component exhibits different test values, with only one having a mean close to the limit or a large variance. Fig. 3 describes such a fault category: the test values for one component and two positions are presented by panel. Position 2 has some values very close to the lower limit for a number of products around 1000 while position 1 does not have the same values for these panels.

The second fault category is related to what we call "batch issue" and is linked to other parameters of production. According to experts, this type of fault is frequently explained by the supplier of components. For this type of faults, all panel tested positions have test values with a low Cpk. to the limit in all positions and then move simultaneously towards the center afterward.

Performance evaluation

This section highlights the performance evaluation of the different values of the risk parameter q with the three metrics defined in Section 3. Fig. 5 presents the cost metric L and RW I of the algorithms depending on two parameters: the risk q and the M RF metric. Then, to plot the two metrics, we exclude the tests with a M RF greater than a fixed value. The maximum M RF is about 21% for all the tests of our dataset. We denote "M RF : All" when all the tests are used for the metric computation. "M RF : β" represents the cases such that M RF ≤ β.

The two cost metrics are very close due to the low-cost time of the SPOT algorithm. They do not seem to be afflicted by the number of faults inside one component test due to the few number tests with a large number of faults in our dataset. The impact of the risk parameter is far more critical, as a q of 0.001 reduces by 70% the time cost of the global method.

Fig. 6 presents the score RDF depending on the risk parameter q and the maximal ratio of faults (M RF ). A large parameter q, (e.g. 0.01), results in almost 100% of fault detection with a computation time reduction of 20%.

Fig. 5. L and RW I metrics with MRF ranging from 0.01 to All and q ∈ {0.001, 0.003, 0.005, 0.01} Fig. 6. RDF metric with MRF ranging from 0.01 to All and q ∈ {0.001, 0.003, 0.005, 0.01}

Table 1 presents the different numerical values used to compute the evaluation metrics for three interesting values of the risk parameter q. These q values have RDF metric value between 96% and 70% which is an acceptable boundary for our case. The computing time of the first stage of the method is really short compared to the fitting time, of the GMM, it represents between 0.01 and 0.003 of the total computation time for the chosen q values. The use of EVT in the first stage allows to drastically reduce the computing time of our global monitoring method (between 65% and 30% of time reduction) since it triggers the second stage only for suspicious windows. In practice, the final choice of the risk parameter depends on the available computing power.

Application of the two-stage monitoring method

This section applies the proposed two-stage monitoring method to three examples. The first example presents the different step results of the method, and the second example emphasizes the need for an adaptive threshold. Finally, the third example presents a solution to a practical limitation of the method over the excluded test.

For the first example, we consider component tests whose expected values are 100kΩ. The design limits are fixed as ±10% of the expected value. For these products the panel is composed of two positions. Fig. 7 presents these values along with the anomaly threshold z 0.003 for a risk q = 0.003. Several products are detected as anomalies by the first stage, as they go above or below the adaptive anomaly threshold. These points are then used as starting points for the second stage of the method. A special focus on one anomaly detection for this case is shown in Fig. 8. After having detected the anomaly (first point in Fig. 8), a window, embedded between dashed lines, composed of the previous test values is extracted. As a starting point, we choose the previous 100 test values. A deeper analysis could be made to optimize this parameter. According to the method described in Section 2, this set of test values is then grouped into clusters using GMM. For the first position, two clusters are found. One of these clusters has a capability index below the chosen target of the FDI method, 1.3. This target is fixed by the process experts.

The capability index of these found clusters is presented in Table 2. Since the capability index of one cluster is below the FDI target, the second position is also investigated, finding only one cluster. This cluster has a Cpk larger than the target accordingly we can identify this anomaly as an equipment issue. The second example demonstrates the interest in an adaptive threshold. It is a test of capacitance whose nominal value is 10 -10 F with design limits fixed to 5 × 10 -11 and 1.5 × 10 -10 F . The panel of products also has two positions. Fig. 9 presents the anomaly detection thresholds for different values of the risk parameter q. The first set of test values is faulty and the first threshold is fixed using these values. The calibration step is done using a set of test values considered as a fault. The SPOT method allows getting a more adjusted anomaly threshold for the product after a maintenance operation (around the product number 8000).

Fig. 9. Anomaly threshold z q for the risk q ∈ {0.001, 0.003, 0.005} for the Example 2 Fig. 10 highlights the necessity of an adaptive anomaly threshold. Indeed the two anomalies (points in Fig. 10) linked to faults are only detected by the new value of z q after the product number 3400. The second stage of the monitoring method is then launched for each detected anomaly, the third point is chosen here as we want to highlight the benefits of choosing an adaptative threshold, in Fig. 10. We use as input the window embedded by dashed lines. The tested position in the panel is 1. Two clusters are found and the Cpk value of the clusters are provided by Table 3. As one cluster has a Cpk value lower than the target Cpk = 1.3 the isolation is performed. Clusters are created for the test values of the second position and the results of the Cpk for these clusters are also provided by Table 3. As only one position has a Cpk lower than 1.3 the fault isolation result is also an equipment issue. The third example treats the excluded test. Fig. 11 presents the test values and the threshold of the SPOT algorithm. For this component test, the Cpk is always under 1.3 and the fault is permanent. This is mainly due to a mean value too close to the lower limit. Since the distribution of the test values does not change, then there are no extreme values from the calibration distribution and the first stage is likely to fail. Nonetheless, the second stage of the method detects a fault for all windows of this component test. That is why a practical solution will verify that the set of test values used for the calibration is not a fault if used as input of the second stage. The introduction of this first stage reduces the total computation time of the monitoring method by at least 30% compared to the previously proposed method, which uses only sliding windows. The second stage of the method detects and isolates a fault using GMM clustering and the capability index Cpk as a health indicator. This twostage monitoring method is tested on an industrial dataset for the ICT equipment. In spite of being able to detect faults, the algorithm sensitivity against faults is reduced compared to M 0 . Further analysis is needed in order to verify that the method improves the production KPI. The proposed method has the advantage that it can apply to all types of component tests.

For future work, one idea is to link the result of the SPOT algorithm to the diagnosis in such a way that faulty samples do not alter the data distribution. Accordingly, samples between θ and z q that belong to a fault time window will not be used to fit the GP D. Another idea is to group the component tests using their design characteristics and link these groups to a specific value of q reducing the computation time of some groups.

Fig. 1 .

 1 Fig. 1. Two-stage monitoring method more the test values are close to the design limits, the more the Cpk is low and the more the variance is large, the more the Cpk is low.

  As explained in[START_REF] Ma | Robust and rapid adaption for concept drift in software system anomaly detection[END_REF], this distribution is, unfortunately, hard to estimate. Nevertheless, the Theorem 1, given in Pickands[START_REF] Pickands | Statistical inference using extreme order statistics[END_REF] makes it possible to estimate the distribution. It states that the excess X -θ over a threshold θ follows a Generalized Pareto Distribution (GPD) with parameters σ and γ. In practice, the GPD is easier to fit than the EVD. Theorem 1.(Pickands-Balkema-De Haan ). The extrema of the cumulative distribution function F converges in distribution G γ , if and only if a function σ(t) exists, ∀x ∈ R and 1 + γ • x > 0:

  Fig. 4 illustrates this second fault category: the test values for the same component recorded on each of the three product positions. The test values start close

Fig. 3 .

 3 Fig. 3. Fault related to the equipment

Fig. 7 .Fig. 8 .

 78 Fig. 7. Anomaly threshold Z for the risk q = 0.003 for Example 1

Fig. 10 .

 10 Fig. 10. First stage z values adaptation for Example 2 with a focus on the product ranging from 3300 and 3700

Fig. 11 .

 11 Fig. 11. Test values for the excluded test 5. CONCLUSION AND FUTURE WORK This paper presents a two-stage health monitoring method with a first anomaly detection stage based on EVT.The introduction of this first stage reduces the total computation time of the monitoring method by at least 30% compared to the previously proposed method, which uses only sliding windows. The second stage of the method detects and isolates a fault using GMM clustering and the capability index Cpk as a health indicator. This twostage monitoring method is tested on an industrial dataset for the ICT equipment. In spite of being able to detect faults, the algorithm sensitivity against faults is reduced compared to M 0 . Further analysis is needed in order to verify that the method improves the production KPI. The proposed method has the advantage that it can apply to all types of component tests.

Table 1 .

 1 Experiments for different values of the risk parameter q and M RF : All

	q	0.001	0.003	0.005
	Mq Computing time (h)	704	1235	1452
	M 0 (sliding window) computing time (h)	2046	2046	2046
	L	0.344	0.603	0.709
	RDF	0.696	0.909	0.956
	RW I	0.335	0.497	0.706
	Number windows checked by the second stage with Mq	1271521 1886392 2679063
	Number of windows checked by the second stage with M 0	3795344 3795344 3795344
	Number of faults found by Mq	4316	5637	5929
	Number of faults found by M 0 (sliding window)	6197	6197

Table 2 .

 2 Cpk values for the window of the first anomaly

	Cpk	Cluster	N points Position
	3.80	1	67	1
	0.91	2	33	1
	2.52	1	100	2

Table 3 .

 3 Cpk values for the blue window in Example 2

	Cpk	Position	Cluster	N points
	17.37 2	1	13
	8.30	2	2	22
	17.65 2	3	13
	3.78	2	4	25
	7.14	2	5	27
	3.30	1	1	92
	0.82	1	2	8