Jyoti Maggu
email: jyotim@iiitd.ac.in

Angshul Majumdar
email: angshulm@iiitd.ac.in

Emilie Chouzenoux
email: emilie.chouzenoux@univ-mlv.fr

Giovanni Chierchia
email: giovanni.chierchia@esiee.fr

Deeply Transformed Subspace Clustering

Subspace clustering assumes that the data is separable into separate subspaces; this assumption may not always hold. For such cases, we assume that, even if the raw data is not separable into subspaces, one can learn a deep representation such that the learnt representation is separable into subspaces. To achieve the intended goal, we embed subspace clustering techniques (locally linear manifold clustering, sparse subspace clustering and low rank representation) into deep transform learning. The entire formulation is jointly learnt; giving rise to a new class of methods called deeply transformed subspace clustering (DTSC). To test the performance of the proposed techniques, benchmarking is performed on image clustering problems. Comparison with state-of-the-art clustering techniques shows that our formulation improves upon them.

Introduction

Clustering has been a classical problem in machine learning. It studies how signals are naturally grouped together. Perhaps the simplest and most widely used clustering technique is the K-means [START_REF] Hartigan | Algorithm AS 136: A k-means clustering algorithm[END_REF]. It groups the samples such that the total (Euclidean) distance of the data points within the clusters is minimized. The problem is NP hard, and hence is usually solved greedily. One of the limitations of Kmeans is that it may fail to capture non-linear relationships. The simple fix to that is the kernel K-means [START_REF] Dhillon | Kernel k-means: spectral clustering and normalized cuts[END_REF]. The concept remains the same as in any kernel trick; operationally instead of Euclidean distances between the samples, its kernelized version is used for Kmeans. Related to the kernel K-means is spectral clustering [START_REF] Dhillon | Kernel k-means: spectral clustering and normalized cuts[END_REF][START_REF] Ng | On spectral clustering: Analysis and an algorithm[END_REF]; the later generalizes over the former by replacing kernelized data matrix to any similarity measure (not restricting to Mercer kernels) calling it the 'affinity matrix'. Subspace clustering techniques [START_REF] Vidal | Subspace Clustering[END_REF][START_REF] Oktar | A review of sparsity-based clustering methods[END_REF] are based on a slightly different model; it assumes that the samples from the same cluster will lie in the same subspace. Finding the clusters boils down to finding the different subspaces. There can be different ways to find the subspaces -these will be discussed later.

Subspace clustering operates on the raw data; the data might be such that it is not separable into subspaces in the original domain. In such a case, one may be able to learn a projection, such that the data is separable in the projected domain. Several studies in the past have shown that indeed may be the case. For example, in [START_REF] Patel | Latent Space Sparse Subspace Clustering[END_REF][START_REF] Patel | Latent Space Sparse and Low-Rank Subspace Clustering[END_REF] a tight-frame was learnt from the data along with the subspace clustering formulation; even though the original data was not separable into subspaces, the projection by the learnt tight-frame made it separable.

In [START_REF] Tian | Learning deep representations for graph clustering[END_REF][START_REF] Peng | Deep Sub-space Clustering with Sparsity Prior[END_REF] representations were learnt via autoencoders and fed into clustering algorithms (K-means in [START_REF] Tian | Learning deep representations for graph clustering[END_REF] and subspace clustering in [START_REF] Peng | Deep Sub-space Clustering with Sparsity Prior[END_REF]).

The assumptions in these studies remained the same, i.e. even if the original data is not separable into subspaces, their corresponding representations will be. The difference between [START_REF] Patel | Latent Space Sparse Subspace Clustering[END_REF][START_REF] Patel | Latent Space Sparse and Low-Rank Subspace Clustering[END_REF] and [START_REF] Tian | Learning deep representations for graph clustering[END_REF][START_REF] Peng | Deep Sub-space Clustering with Sparsity Prior[END_REF] is that the former are shallow techniques while the later are based on deep learning.

Our work is based on similar assumptions. We assume that even if the data is not separable into subspaces in the original space, we can learn deep representation from the data such that it is tailored to be separable in the representation space. Our preliminary work on this topic can be found in [START_REF] Maggu | Transformed Locally Linear Manifold Clustering[END_REF]; we showed that by learning a single layer of transform jointly with the locally linear manifold clustering, better results can be obtained. It was a shallow technique involving only one layer of transform learning. Our proposed work is an extension of this basic approach.

• The first extension is to learn deeper representations via deep transform learning (DTL) [START_REF] Maggu | Unsupervised Deep Transform Learning[END_REF].

• The second extension is to incorporate three variants of subspace clustering -i) Locally linear manifold clustering (LLMC), ii) sparse subspace clustering (SSC), and iii) low rank representation (LRR) into the DTL model.

Operationally, we embed the clustering formulation into the deep transform learning paradigm, and jointly solve it using variable splitting.

The assumption here is that, even if the original data is not separable into subspaces, by non-linearly transforming the data (via DTL), we will learn a representation which will be separable into subspaces.

Background

Subspace Clustering

Subspace clustering assumes that the data is naturally segregated into subspaces. To find these clusters, the first task is to identify these subspaces. The general formulation, be it locally linear manifold clustering (LLMC) [START_REF] Goh | Segmenting Motions of Different Types by Unsupervised Manifold Clustering[END_REF], sparse subspace clustering (SSC) [START_REF] Elhamifar | Sparse Subspace Clustering: Algorithm, Theory, and Applications[END_REF] or low rank representation (LRR) [START_REF] Liu | Robust Recovery of Subspace Structures by Low-Rank Representation[END_REF], remains the same; in order to identify the subspaces each sample is expressed as a linear combination of other samples.

, in {1,..., }

c i i i x X c i n = ∀ (1)
Here

i i i i c x X c R c i n - + ∀ (2)
Here R is the regularization term. Depending on its nature there are three formulations. For LLMC there is no regularization. For sparse subspace clustering, R is a sparsity promoting l1-norm. For LRR, it is a low-rank penalty usually in the form of nuclear norm.

Once the coefficients are estimated, the next step is to segment the data. This requires application of spectral clustering. For that, the affinity matrix is defined from the coefficient matrix

1 [ | ... | ] n C c c = ⌣ ⌣ , obtained from all n samples. Note that i c ⌣ ( n ∈ ℝ ) is defined from i c ( 1 n-
∈ ℝ ) by putting zero in the i th position. There is no unique definition to the affinity matrix; the only requirement is that it needs to be symmetric. Several variants have been proposed for constructing it from C; but the most commonly used is the following -

T A C C = + (3)
Once the affinity matrix is defined (by using any suitable formula), the third step is to segment the clusters. Usually spectral clustering algorithm (Normalized-Cuts on A) is used for this purpose.

Transformed Subspace Clustering

The basic formulation for transformed subspace clustering was proposed by the authors in [START_REF] Maggu | Transformed Locally Linear Manifold Clustering[END_REF]. Instead of learning the clusters from the raw data, they are learnt from the transform coefficients. The learning proceeded in a joint fashion. The complete formulation is given as follows -

( ) 2 2 2 1 2 , , Transform Learning Subspace Clustering min + log det + ( ) c i i F F i T Z C i TX Z T T Z z Z c R C λ µ γ - - + - +  (4)
Note that in [START_REF] Maggu | Transformed Locally Linear Manifold Clustering[END_REF], the LLMC formulation was used so there was no regularization. The first portion of the formulation (4) corresponds to that of transform learning [START_REF] Ravishankar | Learning Sparsifying Transforms[END_REF]. It is the analysis equivalent of dictionary learning; transform learning analyses the data by learning a transform / basis to produce coefficients. Mathematically this is expressed as,

TX Z = ( 5
)
Here T is the transform, X is the data and Z the corresponding coefficients. Learning proceeds by solving the following optimization problem -first part of (4).

( )

2 2 1 , min + log det + F F T Z TX Z T T Z λ µ - - (6)
T and Z are updated alternatingly till the solution converges [START_REF] Ravishankar | Online Sparsifying Transform Learning-Part II: Convergence Analysis[END_REF]. The update for Z has a closed form -soft thresholding; the closed form update for T has also been given in [START_REF] Ravishankar | Closed-form solutions within sparsifying transform learning[END_REF].

The solution for our prior formulation (4) proceeds via alternating between the three variables T, Z and C. Although the initial work [START_REF] Maggu | Transformed Locally Linear Manifold Clustering[END_REF] did not use any regularization, one can easily extend it to incorporate SSC and LRR formulations.

The idea behind this approach is motivated by [START_REF] Patel | Latent Space Sparse Subspace Clustering[END_REF][START_REF] Patel | Latent Space Sparse and Low-Rank Subspace Clustering[END_REF][START_REF] Tian | Learning deep representations for graph clustering[END_REF][START_REF] Peng | Deep Sub-space Clustering with Sparsity Prior[END_REF], i.e. even if the raw data X is not separable into subspaces, the transform coefficients (Z) will lie in different subspaces. However, the learnt transform (T) is linear; therefore the projection it learns is mostly as good as the raw data as far as separability into subspaces is concerned.

Deep Clustering

So far, there has been only a single work that incorporates subspace clustering into a deep learning framework. In [START_REF] Peng | Deep Sub-space Clustering with Sparsity Prior[END_REF], they incorporate the sparse subspace clustering formulation into the features from the deepest layer of a stacked autoencoder. Mathematically the formulation is as follows, (

( )

2 2 1 2 3 3 2 1 3 2 1 3 2 1 1 , , 2
Stacked Autoencoder Subspace Clustering min

c i i i i i i W W C F i X W W W W W W X W W W X W W W X c c ϕ ϕ ϕ ϕ ϕ γ ϕ ϕ ϕ ϕ λ ′ ′ ′ ′ - + - +  (7)
Here are showing the formulation for three layers. i W ′ denotes the i th level of decoder and i W denotes the i th level of decoder. The l1norm on ci promotes sparsity. The idea is similar to that of transformed subspace clustering; the clustering formulation is incorporated into the representation learning model.

There have been other clustering formulations embedded into the deep learning framework. In [START_REF] Tian | Learning deep representations for graph clustering[END_REF], a naïve solution where stacked autoencoders are used for representation learning followed by separate K-means / spectral clustering was proposed. A more sophisticated version of it [START_REF] Yang | Towards k-means-friendly spaces: Simultaneous deep learning and clustering[END_REF], jointly learns the representation from a stacked autoencoder while incorporating K-means clustering formulation. A slightly different version was proposed in [START_REF] Xie | Unsupervised deep embedding for clustering analysis[END_REF], where instead of using the standard Euclidean distance as the measure for K-means clustering, the Kullback-Leibler divergence was used. In [START_REF] Gheche | OrthoNet: Multilayer Network Data Clustering[END_REF], the encoder-decoder structure was replaced by a simple neural network trained to have orthogonal outputs, thus yielding a deep version of spectral clustering.

In all of the aforesaid formulations, the underlying assumption remains the same. The deep network learns non-linear projections such that the learnt representation is separable into subspaces even when the original data is not. There is another class of deep learning techniques which are deeper extensions of the well known non-negative matrix factorization (NNMF) based clustering framework. In [START_REF] Trigeorgis | A Deep Semi-NMF Model for Learning Hidden Representations[END_REF] this is extended to deeper layers. They argue that representations from each layer form clusters based on different attributes, e.g. if one is clustering faces, one layer may cluster based on gender, another layer may cluster based on ethnicity etc.

Deeply Transformed Sub-space Clustering

The idea of deep transform learning has been recently introduced [START_REF] Maggu | Unsupervised Deep Transform Learning[END_REF]. Just like any other deep learning model, we learn deep representation by repeatedly applying transform learning on the data. Mathematically this is expressed as follows -

3 2 1 T T T X Z = (8)
Although we have not explicitly shown any activation function between the layers, we will implicitly impose rectified linear unit (ReLU) type activation. We have shown the formulation for three layer. The generalization to more number of layers will be straightforward.

The optimization problem is posed as -( )

3 2 2 3 2 1 ' , 1 1 2 1
min log det . . 0 and 0

i i i F F T s Z i T T T X Z T T s t T X T T X λ = - + - ≥ ≥  (9)
Note that we have dropped the sparsity promoting term on the coefficients. Usually in deep learning, the dimensionality of the coefficients reduce in each layer; therefore the representation is naturally compact.

Using the unsupervised formulation, one can in principle take the learnt representation and put into any classifier for segmentation.

However, it will be shown in this work, that such a greedy piecemeal formulation usually does not perform very well. A better approach is to learn the deep representation tailored for projections. For this, we embed the subspace clustering formulation into the deep transform learning to jointly learn the representation and clustering. The notion is similar to prior studies [START_REF] Yang | Towards k-means-friendly spaces: Simultaneous deep learning and clustering[END_REF][START_REF] Xie | Unsupervised deep embedding for clustering analysis[END_REF][START_REF] Gheche | OrthoNet: Multilayer Network Data Clustering[END_REF], where the goal is to learn a deep projections (with deep autoencoders) that are conducive to clustering.

We reiterate the core idea behind this work. In subspace clustering, it is assumed that the data is naturally separated into certain subspaces. This may not always hold. Here, we are assuming that even if the original data is not separable into subspaces, by learning a non-linear representation of it (via deep transform learning) tailored for clustering, the representation will fall into separate subspaces.

Mathematically our formulation can be expressed as, ( )

1 2 3 2 3 3 2 2 2 3 2 1 2 , , , , , , 1 1 2 1
min log det ( )

. . 0 and 0

c i i i i F F i T T T X X Z C i i T T T X Z T T z Z c R C s t T X T T X λ γ = - + - + - + ≥ ≥   (10)
To solve [START_REF] Maggu | Transformed Locally Linear Manifold Clustering[END_REF], we follow the popular variable splitting strategy. After introducing the proxy variable, the augmented Lagrangian is solved via alternating direction method of multipliers (ADMM). In our case [START_REF] Maggu | Transformed Locally Linear Manifold Clustering[END_REF] we introduce two proxy variables

2 1 3 1 2 and T T X X T X X = =
. The augmented Lagrangian becomes, ( )

1 2 3 2 3 3 2 2 2 2 2 3 3 1 2 2 3 2 1 2 2 , , , , , , 1 3 2
min log det ( )

. . 0 and 0

c i i i i F F F F i T T T X X Z C i i T X Z T X X T X X T T z Z c R C s t X X µ µ λ γ = - + - + - + - + - + ≥ ≥   (11)
In [START_REF] Maggu | Unsupervised Deep Transform Learning[END_REF], the hyper-parameters μ1 and μ2 correspond to the representation in shallower layers. We argue that there is no reason to prefer one layer over the other, therefore we assign μ1=μ2=1. With this slight simplification, we have -( )

1 2 3 2 3 3 2 2 2 2 2 3 3 2 2 3 1 2 2 , , , , , , 1 3 2
min log det ( )

. . 0 and 0

c i i i i F F F F i T T T X X Z C i i T X Z T X X T X X T T z Z c R C s t X X λ γ = - + - + - + - + - + ≥ ≥   (12)
The problem (12) can be solved using ADMM [START_REF] Hong | Convergence analysis of alternating direction method of multipliers for a family of nonconvex problems[END_REF]. Each of the variables are updated separately from the following sub-problems.

( )

1 2 2 1 2 1 1 P1: min log det F F T T X X T T λ - + - ( ) 2 2 2 2 2 3 2 2 P2 : min log det F F T T X X T T λ - + - ( ) 3 2 2 3 3 3 3 P3 : min log det F F T T X Z T T λ - + - 2 2 3 3 2 2 3 3 P4 : min . . 0 F F X T X Z T X X s t X - + - ≥ 2 2 2 2 2 3 1 2 2 P5 : min . . 0 F F X T X X T X X s t X - + - ≥ 2 2 3 3 2 P6 : min c i i F i Z i T X Z z Z c γ - + -  2 2
P7 : min ( )

c i i i C i z Z c R C - +
 Sub-problems P1 to P3 are standard transform updates whose closed form solution is given in [START_REF] Ravishankar | Closed-form solutions within sparsifying transform learning[END_REF]. P4 and P5 are least square problems with closed form updates -one first needs computing the pseudoinverse followed by imputing all the negative values to zero. P6 is a simple least squares problem. The solution to P7 will depend on the regularization used. With no regularization (locally linear manifold clustering), it will have a closed form update via the pseudoinverse. With l1-norm regularization P7 will have to solved via some kind of iterative soft thresholding such as [START_REF] Zeng | Sparse solution of underdetermined linear equations via adaptively iterative thresholding[END_REF]; this case pertains to sparse subspace clustering. When the regularizer in P7 is a nuclear norm, one needs to solve it via singular value shrinkage [START_REF] Lin | Accelerated reweighted nuclear norm minimization algorithm for low rank matrix recovery[END_REF].

This concludes the derivation of the main algorithm. Once, ( 10) is solved, our work proceeds in the same fashion as standard subspace clustering. Given C, we compute the affinity matrix using (3), which is then segmented / clustered by Normalized cut.

Experimental Results

In this section we compare our method with three deep clustering benchmarks -deep sparse subspace clustering (DSC) [START_REF] Peng | Deep Sub-space Clustering with Sparsity Prior[END_REF], deep Kmeans clustering (DKM) [START_REF] Yang | Towards k-means-friendly spaces: Simultaneous deep learning and clustering[END_REF] and deep matrix factorization (DMF) [START_REF] Trigeorgis | A Deep Semi-NMF Model for Learning Hidden Representations[END_REF]. The said studies have been published recently and have compared with traditional clustering techniques like matrix factorization, spectral clustering, subspace clustering, hierarchical clustering etc. However for the sake of completeness, we compare with four classical methods as well -sparse subspace clustering (SSC), kernel SSC (KSSC), low rank representation (LRR) and kernel LRR (KLRR); the kernel methods use RBF kernel.

We follow the experimental protocol from [START_REF] Peng | Deep Sub-space Clustering with Sparsity Prior[END_REF]. Experiments were carried out on the COIL20 1 (object recognition) and Extended YaleB 2 (face recognition) datasets. The COIL20 database contains 1,440 samples distributed over 20 objects, where each image is with the size of 32×32. The used YaleB consists of 2,414 samples from 38 individuals, where each image is with size of 192×168. For both the datasets DSIFT (dense scale invariant feature transform) features were extracted. They were further reduced by PCA to a dimensionality of 300. Since the ground truth (class labels) for these datasets are available, clustering accuracy was measured in terms of Accuracy, NMI (normalized mutual information), ARI (adjusted rank index), Precision and F-score. The results are shown in Table 1 (COIL20)

and Table 2 (YaleB). Since the last stage of all the clustering algorithms involve K-means, we ran the experiments 100 times and report the mean.

The parametric settings for the methods compared against have been taken from the respective papers. For our proposed technique, we have kept λ=0.1 and γ=1; these are the standard values used in transform learning. TLLMC does not require specification of any other parameter. TSC has τ=0.1 as the sparsity promoting term and TLLR has τ=0.01 as the rank deficiency term. The algorithms are robust to these parametric values; changes by an order of magnitude to either side do not affect the results statistically. The results show that our proposed method with sparse subspace clustering yields the best results on an aggregate. The results from the LLMC based formulation are comparable to the existing benchmarks. Our formulation with LRR yields the worst results. But this is in tune with the observations in [START_REF] Peng | Deep Sub-space Clustering with Sparsity Prior[END_REF] -LRR formulation does not yield good results on these datasets. This may be because, LRR is sensitive to outliers; most LRR based clustering formulations have an explicit outlier rejection term; we have not incorporated it here; we used the vanilla formulation. Perhaps this is the reason, the results are poor. We find that the classical techniques (SSC, KSSC, LRR and KLLR) perform worse than the rest of the techniques. This observation is in tune with prior studies.

Our method requires specification of very few parameters. The values of λ=0.1 and γ=1 have been used from transform learning literature [START_REF] Ravishankar | Learning Sparsifying Transforms[END_REF][START_REF] Ravishankar | Online Sparsifying Transform Learning-Part II: Convergence Analysis[END_REF][START_REF] Ravishankar | Closed-form solutions within sparsifying transform learning[END_REF]; we did not tune it. The only parameter, we tuned is the regularization term corresponding to the subspace clustering.

Since, LLMC does not have any regularization, no analysis could be carried out. For the LRR and SSC variants, variation of ARI is plotted. From these run-times we find that our method is actually the fastest among all the deep learning formulations. Among the three variants we have proposed, DTLLMC is the fastest; this is because it does not have any regularization term that needs iterative steps. DTSSC and DTLRR are slower that DTLLMC owing to the l1-norm and nuclear norm regularizations. Among these two, DTLRR is the slowest since it requires solving a singular value decomposition, which is a computationally expensive step. The linear shallow methods are the fastest; but their kernelized versions are comparatively slower. This is because of the increases size of the kernel matrix compared to the original dimensions.

So far we have shown the best results from our proposed method. In the next set of experiments we show the results of variation in number of layers and the results of joint versus greedy solution. In the greedy solution, we learn the deep transform separately and feed the features into subspace clustering. Since the sparse subspace clustering yields the best results, we are showing the results on this formulation. The results are shown in Tables 4 and5. We find that the results improve from one to three layers, but once we go beyond three layers the results deteriorate. This is because, with more layers the number of parameters to learn increases; with limited volume of training data (as is the case), this leads to overfitting and subsequent deterioration of results. Between the joint and greedy formulations, the joint formulation yields better results. This is expected, because this formulation learns the weights with the goal of clustering. The greedy unsupervised formulation does not have this advantage.

Conclusion

In this work we propose deeply transformed subspace clustering. Several layers of transforms are used to analyze the data with such that the learnt representations are separable into subspaces. This stems from the assumption that even if the data is not separable into subspaces in the original space, their non-linearly learnt representations will be. We have compared our proposed technique on two benchmark clustering datasets and compared with three contemporary deep clustering techniques. Our method shows the best results.

Fig. 1 .Fig. 2 .

 12 Fig. 1. Variation of DTSC (a) and DTLRR (b) with regularization value τ

  xi (

		m ∈ ℝ ) denotes the i th sample and c i X (	∈ ℝ	1 × -m n	) all other samples; ci (	1 n-∈ ℝ ) is the corresponding linear weight vector. The
	information about the subspaces is embedded in the coefficients ci. The coefficients are solved by the following,
	min i	c	2 2	( ), in {1,..., }

Table 1 .

 1 Comparison with benchmarks on COIL 20

	Metric	SSC	KSSC	LRR	KLRR	DSC	DKM	DMF	DTLLMC	DTSC	DTLRR
	Accuracy	.79	.72	.72	.71	.85	.88	.86	.93	.98	.81
	NMI	.89	.79	.84	.80	.91	.94	.92	.97	.98	.89
	ARI	.76	.64	.65	.63	.84	.86	.85	.89	.90	.80
	Precision	.70	.63	.65	.61	.82	.85	.84	.88	.90	.72
	F-measure	.78	.65	.66	.63	.85	.87	.84	.91	.93	.81

Table 2 .

 2 Comparison with benchmarks on Extended YaleB

	Metric	SSC	KSSC	LRR	KLRR	DSC	DKM	DMF	DTLLMC	DTSC	DTLRR
	Accuracy	.70	.70	.71	.70	.88	.91	.89	.96	.99	.84
	NMI	.83	.83	.80	.80	.90	.92	.90	.97	.98	.89
	ARI	.64	.65	.63	.63	.83	.90	.83	.95	.96	.77
	Precision	.65	.67	.62	.61	.79	.91	.80	.95	.99	.71
	F-measure	.66	.68	.65	.65	.83	.90	.84	.93	.95	.76
	1 www.cs.columbia.edu/CAVE/software/softlib/coil-20.php
	2 https://computervisiononline.com/dataset/1105138686

Table 3 .

 3 Comparison of run-times in seconds

	Technique	Coil 20	Yale B
	SSC	11	9
	KSSC	22	20
	LRR	24	22
	KLRR	48	50
	DSC	62	61
	DKM	87	83
	DMF	57	54
	DTLLMC	44	38
	DTSSC	50	42
	DTLRR	58	48

Table 4 .

 4 Analysis of proposed method (DTSC) on COIL 20.

	Metric	1 layer	2 layer		3 layer	4 layer
		Greedy	Joint	Greedy	Joint	Greedy	Joint	Greedy	Joint
	Accuracy	.96	.97	.96	.98.	.96	.98	.94	.99
	NMI	.85	.92	.86	.94	.88	.98	.84	.95
	ARI	.86	.92	.87	.94	.87	.90	.83	.96
	Precision	.82	.87	.82	.88	.83	.90	.81	.88

Table 5 .

 5 Analysis of proposed method (DTSC) on Extended YaleB

	Metric	1 layer	2 layer		3 layer	4 layer
		Greedy	Joint	Greedy	Joint	Greedy	Joint	Greedy	Joint
	Accuracy	.73	.98	.76	.99	.76	.99	.74	.99
	NMI	.39	.94	.41	.94	.43	.98	.42	.96
	ARI	.38	.95	.43	.95	.44	.96	.45	.95
	Precision	.44	.98	.58	.99	.59	.99	.59	.99
	F-measure	.42	.94	.51	.95	.53	.96	.52	.95

Acknowledgement

This work is supported by the Infosys Center for Artificial Intelligence @ IIIT Delhi and by the Indo-French CEFIPRA grant DST-CNRS-2016-02.


Keywords: Transformed Subspace Clustering Jyoti Maggu, Angshul Majumdar, Emilie Chouzenoux

come

Introduction

The problem of clustering is well known. It studies how signals are naturally grouped together. Perhaps the simplest and most widely used clustering technique is the Kmeans [START_REF] Hartigan | Algorithm AS 136: A k-means clustering algorithm[END_REF]. It groups the samples such that the total distance of the data points within the cluster are minimized. The problem is NP hard, and hence is usually solved greedily.

One of the limitations of K-means is that it operates on the raw data and hence fails to capture non-linear relationships. The simple fix to that is the kernel K-means [START_REF] Dhillon | Kernel k-means: spectral clustering and normalized cuts[END_REF]. The concept remains the same as in any kernel trick; operationally instead of Euclidean distances between the samples, its kernelized version is used for K-means.

Related to the kernel K-means is spectral clustering [START_REF] Dhillon | Kernel k-means: spectral clustering and normalized cuts[END_REF][START_REF] Ng | On spectral clustering: Analysis and an algorithm[END_REF]. The kernelized data matrix is the same as the affinity matrix for spectral clustering. It just generalizes the kernels to be any similarity measure and not necessarily based on Mercer kernels.

Subspace clustering techniques [START_REF] Vidal | Subspace Clustering[END_REF] assume that the samples from the same cluster will lie in the same subspace. Operationally, it involves expressing each data point as a linear combination of other data points. These linear weights serve as inputs for creating the affinity matrix. The factorization based techniques prevalent in clustering (such as [START_REF] Zhao | Robust Dual Clustering with Adaptive Manifold Regularization[END_REF][START_REF] Chen | Non-Negative Matrix Factorization for Semisupervised Heterogeneous Data Coclustering[END_REF][START_REF] Liu | Constrained Concept Factorization for Image Representation[END_REF]) technically belong to the subspace clustering paradigm.

In the past, it has been found that instead of applying subspace clustering on the raw data, a projection can be learnt such that the clustering is carried out in the projected domain. In [START_REF] Patel | Latent Space Sparse Subspace Clustering[END_REF][START_REF] Patel | Latent Space Sparse and Low-Rank Subspace Clustering[END_REF] a tight-frame was learnt from the data along with the subspace clustering formulation. The fundamental assumption behind such formulations is that even if the original data do not fall on separate sub-spaces, their projected versions will.

Our work is based on similar assumptions. Instead of applying subspace clustering on the original space, we will learn the subspace clustering in the transformed space. Therefore, even if the data cannot be segmented / clustered in the original domain, its transformed representation can be clustered into separate subspaces. The work proposes to incorporate three variants of subspace clusteringi) Locally linear manifold clustering (LLMC), ii) sparse subspace clustering (SSC), and iii) low rank representation (LRR).

We compared our proposed formulations with state-ofthe-art representation learning and clustering techniques. We show that our method improves over the rest by a considerable margin in terms the clustering metrics used here.

Literature Review

Subspace Clustering

Subspace clustering techniques like locally linear manifold clustering (LLMC) [START_REF] Goh | Segmenting Motions of Different Types by Unsupervised Manifold Clustering[END_REF][START_REF] Blumensath | Directional Clustering Through Matrix Factorization[END_REF], sparse subspace clustering (SSC) [START_REF] Elhamifar | Sparse Subspace Clustering: Algorithm, Theory, and Applications[END_REF][START_REF] Tschannen | Noisy Subspace Clustering via Matching Pursuits[END_REF][START_REF] Chen | Active Orthogonal Matching Pursuit for Sparse Subspace Clustering[END_REF] and low rank representation (LRR) [START_REF] Liu | Robust Recovery of Subspace Structures by Low-Rank Representation[END_REF][START_REF] He | Information Theoretic Subspace Clustering[END_REF] express the samples as a linear combination of other samples. This is expressed as, , in {1,..., }

c ii i x X c i n = (1) Here xi ( m 
) denotes the i th sample and

c i X ( 1 mn -  ) all other samples; ci ( 1 n-



) is the corresponding linear weight vector.

For all the sub-space clustering techniques the general learning formulation can be expressed as follows,

c i i i i i c x X c R c i n - +  ( 2
)
Here R is the regularization term. Depending on its nature there are three formulations. For LLMC there is no regularization. For sparse subspace clustering, R is a sparsity

Transformed Subspace Clustering

Jyoti Maggu 1 , Angshul Majumdar 1 and Emilie Chouzenoux 2 1 Indraprastha Institute of Information Technology email: {jyotim, angshul}@iiitd.ac.in 2 CentraleSupelec email: emilie.chouzenoux@centralesupelec.fr promoting l1-norm [START_REF] Elhamifar | Sparse Subspace Clustering: Algorithm, Theory, and Applications[END_REF] or l0-norm [START_REF] Tschannen | Noisy Subspace Clustering via Matching Pursuits[END_REF][START_REF] Chen | Active Orthogonal Matching Pursuit for Sparse Subspace Clustering[END_REF]. For LRR, R is a low-rank penalty usually in the form of nuclear norm. For each formulation, once the coefficient matrix

1 [ | ... | ] n C c c =
is obtained for all n samples, the affinity matrix is computed. Note that i c

( n  ) is defined from i c ( 1 n-



) by putting zero in the i th position. There is no unique definition to the affinity matrix; the only requirement is that it needs to be symmetric. Several variants have been proposed [START_REF] Vidal | Subspace Clustering[END_REF]. For example one option can be -

T A C C =+ (3)
This is usually used in SSC.

Another option for LRR is to form the affinity matrix from the scaled left singular values of C; this is defined as, ( )

2 T ij ij A UU  =  (4)
A C C C C = + - (5)
Once the affinity matrix is defined (by using any suitable formula), one needs to segment the clusters. Usually spectral clustering algorithm (Normalized-Cuts) [START_REF] Shi | Normalized cuts and image segmentation[END_REF] is used for this purpose.

Related to subspace clustering are the matrix factorization based approaches. Some of them have been referred before. For a review one can peruse [START_REF] Türkmen | A Review of Nonnegative Matrix Factorization Methods for Clustering[END_REF]. The basic idea there in, is to identify arbitrary subspaces from the data by factoring it.

Transform Learning

Transform learning analyses the data by learning a transform / basis to produce coefficients. Mathematically this is expressed as, TX Z = (6) Here T is the transform, X is the data and Z the corresponding coefficients. The following transform learning formulation was proposed in [START_REF] Vidal | Subspace Clustering[END_REF] -

( ) 22 1 , min
+ log det + FF TZ TX Z T T Z  -- (7)
The parameters (λ and μ) are positive. The factor log det T imposes a full rank on the learned transform; this prevents the degenerate solution (T=0, Z=0). The additional penalty 2 F T is to balance scale. In [START_REF] Vidal | Subspace Clustering[END_REF], an alternating minimization approach was proposed to solve the transform learning problem. This is given by -

2 1 min F Z Z TX Z Z   - + (8a)
( )

22 min + log det FF T T TX Z T T   - - (8b)
Updating the coefficients (8a) is straightforward using one step of soft thresholding, ( )

( ) max 0, ( ) Z signum TX abs TX    - (9)
Here ''  indicates element-wise product. The update for the transform (8b) also has a closed form solution. This is given as -

TT XX I LL  += (10a) 1 TT L XZ USV - = (10b)
( )

2 1/ 2 1 0.5 ( 2 ) T T U S S I V L  - = + + (10c)
The proof for convergence of such an alternating update algorithm can be found in [START_REF] Ravishankar | Online Sparsifying Transform Learning-Part II: Convergence Analysis[END_REF].

Proposed Formulations

We have discussed in the previous sections the existing concepts of subspace clustering and transform learning. Our contribution in this paper is to embed three subspace clustering formulations, associated to three distinct choices for the regularization term R, into the transformed space, that is, instead of learning the affinity matrix from the raw data, we propose to learn it from the (transform) coefficient space.

The trend of learning representations for machine learning has been used rampantly in supervised problems. For example consider [START_REF] Yang | Sparse representation based fisher discrimination dictionary learning for image classification[END_REF][START_REF] Jiang | Label Consistent K-SVD: Learning a Discriminative Dictionary for Recognition[END_REF]. In [START_REF] Yang | Sparse representation based fisher discrimination dictionary learning for image classification[END_REF] it is assumed that even if the data cannot be discriminated in the original domain, they can be learnt to be discriminative in the feature domain using dictionary learning. Similarly in [START_REF] Jiang | Label Consistent K-SVD: Learning a Discriminative Dictionary for Recognition[END_REF], it is assumed that eventhough one cannot linearly project the raw samples to their corresponding class labels, the learnt dictionary coefficients can be. Similar ideas are echoed in [START_REF] Guo | Discriminative Analysis Dictionary Learning[END_REF] using the transform learning approach. As mentioned before, similar ideas have been explored for clustering as well [START_REF] Shi | Normalized cuts and image segmentation[END_REF][START_REF] Türkmen | A Review of Nonnegative Matrix Factorization Methods for Clustering[END_REF]. In [START_REF] Peng | Deep Subspace Clustering with Sparsity Prior[END_REF], it has been assumed that even if the data is not clustered on subspaces, its deeply (stacked autoencoder) learnt representation will be. Our work follows the same trend; we assume that even if the original data do not lie on separate subspaces, the (transform) learnt coefficients will lie on different subspaces and hence the ensuing affinity matrix can be segmented for clustering.

A naïve solution would be to learn the transform on the data and then use the coefficients as inputs for subspace clustering. But such a piecemeal formulation will not yield the best results. This can be seen from [START_REF] Peng | Deep Subspace Clustering with Sparsity Prior[END_REF] and [START_REF] Tian | Learning deep representations for graph clustering[END_REF]. In [START_REF] Tian | Learning deep representations for graph clustering[END_REF], a deep representation is first learnt and then a thirdparty clustering algorithm is used on the learnt representation; the results improve in [START_REF] Peng | Deep Subspace Clustering with Sparsity Prior[END_REF] when the deep representation and the clustering are jointly learnt.

We propose to formulate a joint solution instead. Mathematically, our formulation is expressed as, ( )

+ log det + ( ) c FF T Z C ii i i TX Z T T Z z Z c R C   -- + - +  22 ,, 2 1 2 min
Alternating minimization [START_REF] Attouch | Proximal alternating minimization and projection methods for nonconvex problems: An approach based on the Kurdyka-Łojasiewicz inequality[END_REF] approach is used for solving [START_REF] Goh | Segmenting Motions of Different Types by Unsupervised Manifold Clustering[END_REF]. It can be segregated into the following subproblems.

( )

22 P1:min + log det FF T TX Z T T  -- 2 2 1 2 P2:min + c ii F i Z i TX Z Z z Z c  - + -  2 2 P3:min ( ) c ii i C i z Z c R C -+ 
The update for the transform (P1) remains the same as in [START_REF] Ravishankar | Learning Sparsifying Transforms[END_REF]. The update for the transform coefficients can also be shown to be the same as in [START_REF] Patel | Latent Space Sparse and Low-Rank Subspace Clustering[END_REF]. Indeed we have,

( ) ( ) ( ) 2 2 1 2 2 1 2 1 min + ) min + min + 0 F F Z T T T T T T F Z F TT TT T Z F TX Z Z Z I C X T Z Z I C Z I XT ZZ IC     - + -  - + -   -   -   (12)
The update for P3 will be dependent on the regularization term R(C). For transformed locally linear manifold clustering (TLLMC) there is no regularization, hence it takes the form -

2 2 min c ii i C i z Z c -  (13)
This can be solved for each ci via the pseudo-inverse.

For transformed sparse subspace clustering (TSSC), the regularization is a sparsity enhancing penalty. In particular if one uses the l1-norm, P3 becomes equivalent to -

1 min C Z ZC C  -+ (14)
This is a standard l1-minimization for which we have used the iterative soft thresholding algorithm [START_REF] Daubechies | An iterative thresholding algorithm for linear inverse problems with a sparsity constraint[END_REF].

For transformed low rank representation (TLLR) formulation, a low rank penalty is imposed on C. Here we will consider the nuclear norm penalty (defined as the sum of singular values of the matrix), so that P3 reads -

* min C Z ZC C  -+ (15)
The above problem can be solved using singular value shrinkage [START_REF] Majumdar | Some Empirical Advances in Matrix Completion[END_REF].

Once the matrix C is obtained, the affinity matrix can be created using either of (3), ( 4) or ( 5); here we have retained (3) since it appears to yield the best practical results. Spectral clustering is then applied to the affinity matrix in order to segment the data X.

Kernelization

An efficient strategy to handle non-linearity in data in machine learning is to employ the so-called kernel trick. If the (transform) learnt coefficients are not linearly separable into subspaces, we assume that projecting them nonlinearly to a higher dimension can make them separable into subspaces. This can be achieved by kernelizing the transform subspace clustering formulation. Kernelizing the basic transform learning has been proposed in [START_REF] Maggu | Kernel transform learning[END_REF]. In this work, we incorporate subspace clustering into the kernel transform learning formulation.

In kernel transform learning, a non-linear version of data is represented in terms of a transform made up of linear combination of non-linear version of itself. This is expressed as, ( ) ( )

T transform B X X Z  = (17)
One can immediately identify the kernel defined by ( ) ( )

T K X X  = .
This allows us to express the kernel version of transform learning in the known (transform learning) form -BK Z = (18) Here, the kernel matrix K plays the role of the data matrix X in the original transform learning formulation; B is the linear weights (similar to transform) that needs to be learnt and Z is the corresponding coefficient matrix.

Under this notation, we can formulate all our versions of transformed subspace clustering using the generic notation -

( ) 22 ,, 2 1 2 min
+ log det + ( ) c FF B Z C ii i i BK Z B B Z z Z c R C   -- + - +  (19)
The solution for [START_REF] Türkmen | A Review of Nonnegative Matrix Factorization Methods for Clustering[END_REF] remains the same as in the previous linear case. We can expect improvements over the basic transform learning version with the kernel trick, since the later accounts for non-linearities.

Experimental Evaluation

Image Clustering

In this section we compare our method with three state-ofthe-art deep clustering benchmarksdeep sparse subspace clustering (DSC) [START_REF] Peng | Deep Subspace Clustering with Sparsity Prior[END_REF], deep K-means clustering (DKM) [START_REF] Yang | Towards k-means-friendly spaces: Simultaneous deep learning and clustering[END_REF] and deep matrix factorization (DMF) [START_REF] Trigeorgis | A Deep Matrix Factorization Method for Learning Attribute Representations[END_REF]. The said studies have been published recently and have compared with traditional clustering techniques like matrix factorization, spectral clustering, subspace clustering, hierarchical clustering etc. Therefore, we do not compare with the traditional ones. We also do not compare with [START_REF] Tian | Learning deep representations for graph clustering[END_REF], since it has been conclusively shown in [START_REF] Peng | Deep Subspace Clustering with Sparsity Prior[END_REF] that the later excels over the former by a large margin. We have not also compared with recent techniques such as [START_REF] Caron | Deep clustering for unsupervised learning of visual features[END_REF], since it is not a pure clustering formulation; it uses label information. We also do not compare against unpublished non-reviewed archival studies.

We follow the experimental protocol from [START_REF] Peng | Deep Subspace Clustering with Sparsity Prior[END_REF][START_REF] Chen | Active Orthogonal Matching Pursuit for Sparse Subspace Clustering[END_REF]. Experiments were carried out on the COIL20 (object recognition) [34] and Extended YaleB (face recognition) [35] datasets. The COIL20 database contains 1,440 samples distributed over 20 objects, where each image is with the size of 32×32. The used YaleB consists of 2,414 samples from 38 individuals, where each image is with size of 192×168. For both the datasets DSIFT (dense scale invariant feature transform) features were extracted. They were further reduced by PCA to a dimensionality of 300. We experiment on these input features. Since the ground truth (class labels) for these datasets are available, clustering accuracy was measured in terms of Accuracy, NMI (normalized mutual information), ARI (adjusted rand index), Precision and F-score. The results are shown in Table 1 (COIL20) and Table 2 (YaleB). Since the last stage of clustering involves K-means, we ran the experiment 100 times and report the average values.

The parametric settings for the methods compared against have been taken from the respective papers. For our proposed technique, we have kept λ=μ=0.1 and γ=1. TLLMC does not require specification of any other parameter. TSC has μ=0.1 as the sparsity promoting term and TLLR has μ=0.01 as the rank deficiency term. The algorithms are robust to these parametric values; changes by an order of magnitude to either side do not affect the results statistically.

In Tables 1 and2, we compare our proposed techniques TLLMC, TSSC and TLLR with DSC, DKM and DMF. Our proposed version comes in two flavors. In the joint (Jt.) formulation (proposed in this work), SSC is embedded in the transform learning; this is the one proposed in this paper. The second formulation is the simplistic piecemeal (PM) formulation. Here features obtained by transform learning are fed into subspace clustering in a piecemeal fashion; the two (transform learning and clustering) are not learnt jointly. As one can expect, the joint formulation yields much better results than the piecemeal formulation. This has been seen in prior studies as well. In [START_REF] Peng | Deep Subspace Clustering with Sparsity Prior[END_REF], the clustering formulation was embedded in a deep autoencoder, where as in [START_REF] Tian | Learning deep representations for graph clustering[END_REF] the formulation was piecemealthe deep autoencoder was learnt separately and the features from the bottleneck layer fed into a separate clustering algorithm. It was shown in [START_REF] Tian | Learning deep representations for graph clustering[END_REF] that the joint formulation yields much better results than the piecemeal [START_REF] Tian | Learning deep representations for graph clustering[END_REF] formulation.

From Tables 1 and2, we also observe that the TLLR formulation does not yield good results. This is not surprising; it follows from the findings in [START_REF] Peng | Deep Subspace Clustering with Sparsity Prior[END_REF]; there in it was find that LRR based formulations yield poor results on these datasets. Our joint TSSC and TLLMC formulations always yields the best results.

Next we show the empirical convergence plot. The plots are generated for the COIL20 dataset. On the X-axis is the iteration number and on the Y-axis is the normalized cost function. All the plots show that our algorithm converges very fast, in less than 10 iterations. We have discussed in III.A how our method can be kernelized. In Tables 3 to 6, we show results of kernelized versions for COIL 20 and Yale B respectively. Note that these results are shown for the jointly learnt variants since they yield better results than their corresponding piecemeal versions. In Tables 3 and4, we show the results for kernelized variants of TLLMC and in Tables 5 and6, we show results for kernelized variants of TSSC. We do not show the results for TLLR since it performs poorly.

The results are shown for some typical kernels like linear, Gaussian, Laplacian and polynomial kernels of order 2, 3 and 4. The RBF kernel yields the best results almost always; followed by the Laplacian kernel. The polynomial kernels yield the worst results consistently. They are worse than the simple linear technique; the results deteriorate with the increase in the order of the polynomial.

All the experiments were run on an Intel i7 processor with 32 GB RAM running a 64 bit Windows 10. The proposed techniques and DMF was based on Matlab; DSC, DKM were based on Python. The run-times are shown in Table 5. The runtimes show that our methods are the fastest. The linear versions are obviously faster than the kernelized ones. This is because the size of the kernelized data matrix is much larger than the original data matrix (input to the linear versions). Of the proposed three variants, TLLMC is the fastest as it has no regularization term. TSSC is slightly slower owing to the requirement of the thresholding step in every iteration. TLRR is even slower, because one needs to threshold the singular values; and computing singular values in every iteration is time consuming. All the compared deep learning based techniques are slow; this is because they need to learn multiple layers.

Document Clustering

Our second example focuses on document clustering. For this application, we follow the protocol defined in a recent work [START_REF] Pei | Concept Factorization With Adaptive Neighbors for Document Clustering[END_REF]. For our experiments, we use three data sets TDT2 corpus [37], Reuters-21578 corpus [37], and 20 Newsgroup [37].

The TDT2 English document data set includes six months of material drawn on a daily basis from six English language news sources. In this set, the total number of samples is 9394, the feature dimension is 36771, and the number of clusters is 30.

The Reuters-21578 document set is a collection of manually categorized newswire stories from Reuters Ltd. In this set, the total number of samples is 8293, the feature dimension is 18933, and the number of clusters is 65.

The 20 Newsgroups data set is a collection of approximately 20,000 newsgroup documents. In this set, the total number of samples is 18846, the feature dimension is 26214, and the number of clusters is 20.

Following [START_REF] Pei | Concept Factorization With Adaptive Neighbors for Document Clustering[END_REF], we report the result in terms of two metrics namely entropy and purity. For good clustering, one requires small entropy and high purity [START_REF] Yang | Linear and Nonlinear Projective Nonnegative Matrix Factorization[END_REF]. In [START_REF] Pei | Concept Factorization With Adaptive Neighbors for Document Clustering[END_REF], the metrics are reported by varying the number of clusters from 2 to 10. We follow the same protocol.

Suppose there is ground truth data that labels the samples by one of classes. Purity is given by,

l q r l k k kl k n entropy n n q n == =  (21)
where l kk l nn =  . Generally, a smaller entropy value cor- responds to a better clustering quality Our proposed method has been compared with [START_REF] Pei | Concept Factorization With Adaptive Neighbors for Document Clustering[END_REF] and with [START_REF] Allab | A Semi-NMF-PCA Unified Framework for Data Clustering[END_REF]; to the best of our knowledge these are two of the most recent works in this area and are known to yield the best possible results on document clustering.

We found that the kernelized versions yield better results than the linear counterparts; and the best results are obtained from the Gaussian kernel. These results are shown here. The results show that only for the easiest dataset (TDT2), CFAN yields slightly better results than some of our proposed variants. Note that, by easiest we mean the dataset where all the algorithms yield good results. But even for this dataset the best result is obtained from TSSC. For the more difficult datasets, all our proposed methods excel over the ones compared against; that too by a significant margin.

We show the empirical convergence plot of our proposed algorithms in Figure 1. The results are shown for the Reuters dataset with 10 clusters. The X-axis shows the iteration numbers and the Y-axis the normalized objective function. By normalized, we mean the cost at the current iterate divided by the initial cost. The plots show that all our algorithms converge very fastwithin 20 iterations. The results for the other datasets are similar; given the limitations of space, we do not show them here.

Finally we show the runtimes for all the techniques. The configuration of the machine remains the same as before. All the algorithms have been implemented in Matlab. The timing comparison shows that our proposed methods are slower than the benchmarks. This is because, unlike image clustering, the state-of-the-art in document clustering are based on matrix factorization and not deep learning. Matrix factorization is a matured area with fast algorithms. Similar to matrix factorization, our proposed techniques are all shallow. However, we require computing SVDs for the updating the transform in every iteration. This is time consuming and hence is slower than matrix factorization.

Conclusion

In this work, we have incorporated subspace clustering formulations into the transform learning framework. This results in three variantstransformed locally linear manifold clustering, transformed sparse subspace clustering and transformed low rank representation. We have also propose kernelized versions of the aforesaid three variants.

Experiments have been carried out on two benchmark problemsimage and document clustering. For each problem, state-of-the-art techniques are compared against. In all cases, our proposed method excels over these in terms of the metrics used here.

In future, we would like to build on the proposed approach in two ways. One way will be to try to incorporate other clustering cost functions. For example in [START_REF] Yang | Towards k-means-friendly spaces: Simultaneous deep learning and clustering[END_REF], it has been shown how K-means can be embedded into deep autoencoders. We would like to explore a similar avenue.

The other enhancement to this work would be to try to make the proposed approach deeper. In a recent study, the concept of deep transform learning has been proposed [START_REF] Maggu | Unsupervised deep transform learning[END_REF]. We would like to see if it would be possible to embed clustering formulations into the said deep formulation.

TT

Figure 1 .

 1 Figure 1. Empirical Convergence Plot for Image Clustering All the aforesaid results are shown for linear kernels.We have discussed in III.A how our method can be kernelized. In Tables3 to 6, we show results of kernelized versions for COIL 20 and Yale B respectively. Note that these results are shown for the jointly learnt variants since they yield better results than their corresponding piecemeal versions. In Tables3 and 4, we show the results for kernelized variants of TLLMC and in Tables5 and 6, we show results for kernelized variants of TSSC. We do not show the results for TLLR since it performs poorly.The results are shown for some typical kernels like linear, Gaussian, Laplacian and polynomial kernels of order 2, 3 and 4. The RBF kernel yields the best results almost always; followed by the Laplacian kernel. The polynomial kernels yield the worst results consistently. They are worse than the simple linear technique; the results deteriorate with the increase in the order of the polynomial.All the experiments were run on an Intel i7 processor with 32 GB RAM running a 64 bit Windows 10. The proposed techniques and DMF was based on Matlab; DSC, DKM were based on Python. The run-times are shown in Table5.

  number of samples in the cluster k that belong to original class l. A larger purity value indicates better clustering performance.Entropy measures how classes are distributed on various clusters. The entropy of the entire clustering solution is computed as

Figure 2 .

 2 Figure 2. Empirical Convergence for Document Clustering

TABLE 5

 5

			TLRR-Kernel	58	48
	Comparison of Runtime in Seconds
	Technique	Coil 20	Yale B
	DSC	62	61
	DKM	87	83
	DMF	57	54
	TLLMC	11	9
	TLLMC-Kernel	44	38
	TSSC	12	11
	TSSC-Kernel	50	42
	TLRR	16	13

TABLE 6

 6

		Comparison showing improvement of proposed method over existing techniques on TDT2
	Clusters		Entropy (lower is better)				Purity (higher is better)
		CFAN	SNMF-PCA	TLLMC TSSC TLLR	CFAN	SNMF-PCA	TLLMC	TSSC	TLLR
	2	.0000	.0000	.0000	.0000	.0000	1.0000	1.0000	1.0000	1.0000 1.0000
	4	.0000	.0159	.0000	.0000	.0159	1.0000	.9956	1.0000	1.0000	.9229
	6	.0526	.0889	.0063	.0013	.0431	.9435	.8954	.9481	.9963	.8917
	8	.0552	.0941	.0329	.0465	.0506	.9476	.8801	.9490	.9013	.9661
	10	.0808	.0685	.0188	.0178	.0631	.9153	.9224	.9265	.9775	.9254
	Avg.	.0312	.0582	.0126	.0103	.0682	.9682	.9285	.9673	.9736	.9527
						TABLE 7
		Comparison showing improvement of proposed method over existing techniques on REUTERS
	Clusters		Entropy (lower is better)			Purity (higher is better)
		CFAN	SNMF-PCA	TLLMC TSSC TLLR	CFAN	SNMF-PCA	TLLMC TSSC TLLR
	2

TABLE 5

 5

	Comparison of Runtime in Seconds
	Technique	TDT2	Reuters	NewsGroup
	CFAN	308	221	682
	SNMF-PCA	276	195	349
	TLLMC	482	303	794
	TSSC	507	346	733
	TLLR	793	515	901

Acknowledgement

The first author is partially supported by TCS Research. All the authors are supported by the Indo-French CEFIPRA grant DST-CNRS-2016-02.


Robert Soriano

Gilles Cunge

Nader Sadeghi

VUV-absorption spectroscopy and delocalized plasma-induced emission used for the species detection in a down-stream soft-etch plasma reactor Autors

Vacuum ultraviolet-absorption spectroscopy and delocalized plasma-induced emission used for the species detection in a down-stream soft-etch plasma reactor

Introduction

Downstream etchers known as remote plasma sources (RPS) were introduced in the early 70's for resist stripping applications [1,2]. However, until recently, they have drawn relatively little scientific interest compared to other plasma sources because they inevitably lead to isotropic etching, which restrict their use to specific applications. But as the size and thickness of transistors continue to shrink, new processes are required to etch high aspect ratio (AR) nano-features. For example, ultrahigh selective isotropic etching in RPS is needed for specific steps involved in the patterning of FinFet [3] transistors and 3D Nand memories [4]. RPS processes have also started to be investigated [5] to replace wet processes that show severe limitation to remove materials in high aspect ratio (AR) structures due to capillary forces. Finally, plasma induced damage (mostly caused by energetic ions and photons) has become a major concern and new etching technologies (involving RPS in combination with typical plasmas) such as the Smart Etch [6] and other ALE processes [7] have been introduced to etch materials anisotropically without damages. As a matter of fact, the interest of RPS lies in their capabilities to produce purely chemical etching with radicals leading to high selectivity and absence of structural damages in underlayers.

In many of the above mentioned applications, the goal is to etch selectively a Si alloy respectively to another Si alloy (e.g. Si, SiN and SiO2), which can be achieved in F-based RPS plasmas. NF3 is typically used as F source due to its high cross section for dissociative electron attachment [8].

Further, F atoms density in the discharge can be controlled by adding H or O containing gases into NF3. For example, downstream plasmas in NF3/O2 mixtures have been used to etch selectively SiN to SiO2 [9], and remarkable performances were obtained recently [4] to remove SiN from trenches with AR of 100, thus performing better than in wet processes.

On the other hand, H addition to NF3 in RPS leads to new etching possibilities. Initially, such processes were introduced [10] to deoxidize silicon surfaces, i.e. etch SiO2 selectively to Si prior to epitaxy, or to clean STI trenches. These processes were achieved in NF3/H2 [11,12] or NF3/NH3 chemistry [13]. The addition of hydrogen to NF3 does not serves only to reduce F atoms but also promotes the formation of new radicals such as HF or NH4F, which are believed to be the precursor to the formation of ammonium salts (NH4)2SiF6 selectively on SiON and SiOx surface [14,15,16]. In this case, there is no direct etching of the surface during the RPS step: the salt forms selectively with material to be etched and must then be sublimated to readily etch the material.

Several innovative Etching processes have been developed recently based on this phenomenon, including the so called Smart Etch process [6] and several Atomic Layer Etching processes [7,17] all of which are two step processes: the material is first modified anisotropically on a wellcontrolled depth by a given plasma and the modified layer is then etched selectively in a second step by a RPS in NF3/H2 or NF3/NH3. The "Smart Etch" process is using energetic H2 or He ions generated in a CCP plasma for the first step, thus modifying several nm of SiN material by ion implantation (the use of light ions prevents SiN sputtering) [18,19]. Initially, the modified SiN* layer was removed selectively to pristine SiN by a wet HF [6] or by gaseous HF [20]. But this doesn't allow cycling the two step of the process with a good throughput. By contrast, by removing the modified SiN* layer with an RPS in NF3/NH3 mixtures [21], it becomes possible to perform the implantation and RPS steps without air exposure, either on two plasma chambers in the same cluster, or even in the same specifically design chamber [22]. For industrial applications, the salt formed during the RPS step can be sublimated either by heating [17], or simply by reducing the pressure before the next implantation step [22]. Recent work [22] has demonstrated that (NH4)2SiF6 salt formation is efficient only on oxidized surfaces, explaining the SiO2/Si and SiOx/SiN etching selectivity as well as the SiN*/SiN selectivity in the Smart Etch process: even if oxygen free H2 or He plasma are used to modify the SiN there was actually a considerable amount of O implantation in the SiN* originating from plasma impurities. Interestingly, while the salt formation has been widely analyzed through surface diagnostics, the origin of radicals that are precursors to their formation remains highly speculative; since the gas phase of NH3/NF3 and NF3/H2 RPS has never been analyzed. The consensual [10,15,21,22,23,24] mechanism for the salt formation is the reaction :

SiO2 + 2NH4F + 4HF  (NH4) 2 SiF6 + 2H2O (1)
Therefore, NH4F, HF and NH4F(HF) are typically assumed to be formed in the plasma and hence they react with the surface. This assumption remains speculative even if the work from Ogawa [15] indirectly suggests that NH4F could be produced from the reaction of NH3 gas with some fragment of NF3 (formed by heating NF3). As a matter of fact, the formation of HF in the plasma can be expected to play a significant role in the overall chemistry and we can expect HF to be formed by the chain reactions (2-4) (well known in HF laser physics) [25,26,27,28,29,30]:

H2+F HF+H (2) H+NFX (x=1,2)  HF+NFx-1 (3) NH3+F NH2+HF (4)
In the present paper, we apply the VUV Broad-Band absorption spectroscopy (BBAS) in the 120-200 nm range [31,32] with a deuterium lamp as light source for the real time monitoring of etching agents and etch products in the downstream of NF3-NH3-H2 containing plasmas of an AMAT RPS reactor especially designed to perform Smart Etching and described in details in ref [22]. Besides the VUV-BBAS, we also have implemented a probe pulsed DC discharge to follow, by actinometry, the density of F and H atoms in the downstream chamber. By directly measuring the densities of NH3, NF3 and HF, as well as the variation of F and H densities as a function of the operating condition, we could get insights into the complex chemistry of these RPS plasmas.

Experimental set-up

Plasma reactor

The experiments are carried out in an industrial 50 cm diameter remote plasma reactor (frontiers from Applied Materials) used for the soft etching of 30 cm diameter wafers. It is composed of two chambers separated by an about 50 mm thick shower head. As shown in figure 1, the gas, whose main component is helium and contains a few percent of H2, NF3 and NH3, enters the top compartment, flows through the shower head to expand into the downstream compartment and is then pumped out with a 200 l/s dry primary pump. The total gas flow rate is about 2000 sccm and the gas pressure in the downstream chamber (DSC), measured with a capacitance gauge, is maintained to 1 Torr with a throttle valve located on the pumping pipe. A Capacitively Coupled Plasma (CCP) is generated in the top compartment with up to 500 W power of 13.56 MHz radiofrequency discharge. Radicals and atoms produced by the discharge are transported by the gas flow into the DSC but the plasma production zone is limited by the shower head, blocking the extension of the discharge into the downstream compartment. Thus, only radicals are responsible for the soft etching of the wafer positioned in the DSC, without any assistance by ion bombardment, as is the case in usual reactive ion etching processes [33]. Due to the absence of viewports in the CCP zone of the reactor, any diagnostic technique can be implemented to characterize the CCP plasma. However, atoms and molecules produced by the dissociation of injected molecules in this plasma and transported to the DSC have been monitored in this chamber by the vacuum ultraviolet broad-band absorption spectroscopy (VUV-BBAS) and by optical emission spectroscopy (OES). But, as a consequence of the absence of energetic electrons in the downstream plasma, any optical radiation is emitted from this region. Thus, to monitor the density of H and F atoms present in the downstream chamber, a small tube (about 4 cm inner diameter, 10 cm long) is connected to the DSC through a 4 cm diameter hole on the reactors' wall and an auxiliary plasma (AP) is generated inside this tube by applying a positive high voltage (HV) to an isolated, 0.2 cm diameter, 3 cm long copper road which penetrate into the tube by one of its end, as shown in figure 1. The opposite end of the tube is sealed by a quartz window for the OES and the inner walls of the tube act as a cathode for the auxiliary discharge.

Actinometry with auxiliary plasma

To follow the variation of H and F atoms densities produced by the CCP discharge, the optical emission of the auxiliary discharge is collected by an optical fibre, whose other end is set on the entrance slit of a 50 cm focal length monochromator (Acton 500 i), equipped with a 600 groves/mm grating and backed by a 1024 elements, 25 µm pitch photodiode array. A 112 kΩ resistor, placed in series with the HV supplier, limits the peak discharge current of the AP to about 8 mA with 1 kV applied voltage. And moreover, the HV is modulated at 120 Hz, with 5% duty cycle to keep as low as possible the dissociation of the gas inside the auxiliary discharge tube.

Thus, we assume that the recorded emission intensities from the excited states of H, F and He atoms, at 656.3, 703.7 and 706.5 nm lines, respectively, are proportional to the densities of these atoms in the downstream chamber. The actinometry method [34,35], with 706.5 nm helium line for the reference intensity, is used to follow changes on H and F atoms' densities when varying the amount of the added gases to helium. We are aware that the large difference between the excitation energies of helium line (22.7 eV) and that of H line (12 eV) and F line (14.7 eV) render questionable the use of helium as actinometer gas [34]. But we consider that with helium remaining the main component of the feed gas, the electron temperature in the auxiliary plasma shouldn't significantly change with the variation of the minor components, which are H2, NF3, NH3 and radicals produced by their dissociation in the CCP plasma. Thus, assuming constant excitation cross-section of species and neglecting the quenching of excited states by the gas, we apply the simplified actinometry equations [35,36]:

𝐼 706 = 𝐺. 𝐾 706 . 𝑛 𝑒 . [𝐻𝑒] (5)
𝐼 𝑋 = 𝐺. 𝑛 𝑒 . (𝐾 𝑋 . [𝑋] + 𝐾 𝑃𝑋 . [𝑃𝑀]) (6)
Where G is a geometric factor, [He] and [X] (with X=F or H) are the densities of helium and atoms X, with I706 and IX being their respective recorded intensities and [PM] is the density of the primary precursor molecule (NF3, H2 or NH3) added to helium whose dissociative excitation in the AP can contribute to the recoded intensity IX and Ki are the respective rate coefficients for the production of X atom in the observed excited state from species "i". To write (6), the contribution to the IX intensity of the other dissociation products of the gas in the CCP, excepted X atoms, is neglected.

The excitation coefficient KPM of the precursor molecule is deduced by measuring IX when known amount of the PM, diluted in helium, is introduced into the reactor in the absence of the CCP discharge. And, as will be described in the following section, the density of precursor molecules NF3 and NH3 in the downstream chamber, thus in the auxiliary plasma, can be measured by the VUV-BBAS. Hence the contribution of PM to the IX can be evaluated and subtracted for obtaining 𝐼 𝑋 𝑐 , the intensity originated from the sole direct excitation of the ground state X atom in the auxiliary plasma. And finally, the variation of the density of X atoms in the downstream chamber, when changing the gas mixture and CCP discharge conditions, can be followed by using the simplified equation:

[𝑋]~[𝐻𝑒] 𝐼 𝑋 𝑐 𝐼 706 (7)

Broad-band absorption spectroscopy

The absolute densities of NH3 and NF3 molecules in the downstream chamber, thus their dissociation rates in the CCP chamber, as well as the density of HF molecules formed from the dissociation products, have been measured by the broadband absorption technique [37,38,39]. A schematic of the top view of the reactor, with the VUV-BBAS set-up is shown in Figure 2. Shortly, the light source is a high-pressure deuterium (D2) lamp (X2D2 from Hamamatsu) that emits a continuum radiation down to 120 nm. A VUV grade parabolic mirror produces a collimated, 1 cm diameter, light beam at 90° from the incident light. This VUV beam crosses the downstream chamber, through two optical ports facing each other and equipped with MgF2 windows, at about 5 cm below the shower head (see Fig. 1). At the exit of the second window, another VUV grade parabolic mirror focuses the VUV beam onto the entrance slit of a 20 cm focal length VUV spectrometer (Jobin-Yvon H20-UVL), equipped with a 1200 groove/mm concave VUV grating.

The exit port of the spectrometer was modified to accept a VUV CCD camera (Newton DO940 for X-ray & VUV from Andor) with 2048x512 pixels of 13.5 µm pitch size. In the central part of the CCD (between pixels 600 to 1500), the linear dispersion is 0.047 nm/pixel and the ultimate spectral resolution of the system is about 0.14 nm with 20 µm entrance slit width. But the spectral resolution is degraded in the edges of the CCD. Turbomolecular pumps maintain a background pressure below 10 -4 Torr in the entire optical path of the VUV light outside the reactor chamber (D2 lamp to the reactor input window and the exit window to the CCD). As usual in BBAS experiments [38], the absorption spectra of the gas in the downstream chamber is obtained by recording three spectra with the CCD: i) the background with the lamp off (BG), ii) the emission of the lamp without the gas flow, L0 and iii) the transmitted light of the lamp (IP(λ)) with the gas flow On. Remark that the absence of energetic electrons in the DSC results in the absence of plasma emission, which can be neglected. The absorbance spectra (A), which is related to the wavelength dependent absorption cross-section (λ) of the molecules through the Beer-Lambert law, is obtained by:

[𝐴] 𝜆 = 𝐿𝑛 ( 𝐼 0 (𝜆) 𝐼 𝑇 (𝜆) ) = 𝐿𝑛 ( 𝐿 0 (𝜆)-𝐵𝐺 𝐼 𝑃 (𝜆)-𝐵𝐺 ) = ∑ 𝜎 𝑖 (𝜆). 𝑙. 𝑖 𝑁 𝑖 (8)
where I0 and IT are the intensity of the incoming and transmitted VUV light, index "i" refers to different absorbing species present in the chamber with density Ni and l is the absorption length.

Although the absorption cross-sections of NH3 [40,41,42,43] and NF3 [44,45,46] have been reported in the literature, for the determination of their partial pressures in the downstream chamber we have used our own collection of absorbance spectra recorded when the reactor was filled with these gases at different known pressures in the absence of discharge. This collection of spectra constitutes some sort of abacus. As an example, recorded absorbance spectra of NH3 and NF3 are reported in figures 3 and 4 for a few indicated pressures. This way of proceeding has the advantage of being insensitive to changes on the spectral profile of NH3 with the spectral resolution of the spectrograph + CCD. Opposite to the NF3 case, for which the VUV photon absorption ends up in a pre-dissociated continuum, the upper states of the VUV transitions in NH3 are Rydberg states, with vibrational and rotational levels. In Fig. 3, transitions to these electronic states and corresponding vibrational levels are identified, but the rotational structure cannot be resolved. The absorption profile is thus composed of many narrow ro-vibrational transitions and important modifications on the behavior of the absorbance curves can be observed with the spectral resolution of the detection system (compare spectra in [41] and [43]). It should be pointed out that the entrance slit of the spectrometer was kept unchanged and consequently, the spectral resolution was identical in all spectra recorded in plasma conditions and for the abacus. Also, using the calibration spectra of the abacus for the determination of NH3 partial pressure under plasma conditions eliminates a possible error source by saturation of the peak absorbencies [39].

Results and discussion

Characterisation of HF spectrum

Dissociation of NF3 in CCP plasma produces F atoms that can rapidly react with H containing molecules to form HF molecule [26,28]. Strong absorption spectra of HF in the VUV spectral range have been reported in the literature [47,48] and its detection by VUV-BBAS was thus highly expected. The experiments have been carried out in the afterglow of NF3/H2 mixture in which HF is produced by the reaction of F atoms resulting from the plasma dissociation of NF3 with H2 molecule (reaction (2)), for which a rate coefficient k2=2-3 x 10 -17 m 3 .s -1 has been reported [26,28]. Absorption spectra recorded in the downstream chamber without and with a discharge in the CCP chamber are shown in figure 5. The experimental steps are:

1) A mixture of 4% NF3/ 9% H2 diluted in helium is flown through the reactor and the exit throttle vale is adjusted for obtaining 1 Torr total pressure in the downstream chamber.

2) The recorded VUV spectra is identical to the one obtained at 40 mTorr of NF3 from the abacus of figure 4. As NF3 doesn't react with H2, we conclude that the partial pressure of NF3 in the DSC is 40 mT in the absence of plasma.

3) A 250 W plasma is stuck in the CCP compartment and after a few seconds of stabilisation time, a new VUV absorption spectrum is recorded. As seen in Fig. 5, the shape of this new spectrum is totally different from the previously recorded in the absence of CCP plasma. This modification revels absorption by new species formed under the CCP plasma action, which, as previously discussed, is HF molecule. However, the high wavelength tail of the new spectrum can be fitted by the absorption spectrum of NF3 at a pressure of 24 mTorr, indicating that about the half of the NF3 molecules was dissociated by the CCP plasma. The new spectrum, obtained when subtracting the absorption curve of 24 mTorr NF3 from the recorded spectra with the plasma ON, corresponds to the A 1 ← X 1  + absorption spectrum of HF molecules. This attribution is justified by the similarity of this spectrum with the one obtained by Nee et al from synchrotron radiation absorption by HF gas [47]. These authors also provide the wavelength dependent absorption cross section, from which the absorbance can be converted to the density of HF in the DSC by using Eq. ( 8). The deduced partial pressure of HF in the example presented in Fig. 5 is 40 mTorr. Given that A 1  state is pre-dissociated, because its energy is above the D°=5.87 eV dissociation limit of HF [START_REF] Huber | Molecular Spectra and Molecular Structure: Constants of Diatomic Molecules[END_REF], the HF(A 1 ← X 1  + ) absorption spectrum is a continuum, with its absorbance being scaled with the HF density, for fixed absorption length. We will thus use in the following the absorbance spectrum deduced from NF3/H2 plasma, depicted in figure 5, for the determination of the amount of HF molecules produced in other plasma conditions.

As an example, in figure 6 The partial pressures of NF3, NH3 and HF in all gas mixtures and plasma conditions have been deduced following the same procedure than in Fig. 6, by the best adjustment of the recorded absorption spectrum with the sum of three absorbance curves of these molecules with adequate amplitudes. Considering the reproducibility of results from several experiments, the uncertainty on determined partial pressures is estimated to be about 7% of the evaluated values + 1 mTorr.

Kinetics of NF3/H2 plasmas

Dissociation of NF3 in the CCP plasma produces also NF2 and NF radicals that rapidly react with H atoms, produced by the dissociation of H2 in the CCP, but also reaction (2). The chain reactions which follow are:

H + NF2 → NF + HF (9) H + NF → N + HF ( 10
)
NF + NF → N2 + 2 F ( 11
)
for which the rate coefficients reported in the literature are: k9=1.9 x 10 -17 m 3 .s -1 [START_REF] Cheah | [END_REF]; k10=2.5 x 10 -17 m 3 .s -1 [START_REF] Cheah | [END_REF]51] and k11=7 x 10 -17 m 3 .s -1 [START_REF] Cheah | [END_REF]51]. These fast chain reactions, whose main final product is HF molecule, can continue as far there are enough H2 molecules and/or CCP generated F atoms in the chamber. molecules, which depends on RF power. Higher is the RF power, larger is the dissociation rate of NF3 and larger is the amount of produced HF. However, the limiting factor can also be the amount of H atoms, produced by the plasma dissociation of H2 molecules or by the reaction (2), available for the chain reactions. In figure 8 are reported the partial pressures of NF3 and HF in the DSC versus the amount of H2 in the feed gas, composed of 100 sccm of NF3, x sccm of H2 and 1900-x sccm of helium. The RF power in the CCP is fixed at 250 W and the total pressure in the DSC is 1 Torr. At the beginning, with increasing H2 flow rate, the partial pressure of HF increases almost linearly to reach its maximum at about 100 sccm of H2, while NF3 density remains almost constant. In fact, the dissociation rate of NF3, thus the amount of F atoms in the DSC is fixed by the RF power in the CCP and up to 100 sccm of H2 the total number of F atoms resulting from the dissociation of NF3 exceeds the double of the number of introduced H2 molecules. Thus, every H2 of the feed gas produces two HF molecules. This is seen from the linear increase of the ratio (HF/dissociated NF3) during the initial phase of the H2 flow rate rise, also shown in figure 8. The actinometric emission intensity of the 703.7 nm F line in the auxiliary plasma, which is indicative of the partial pressure of fluorine atoms in the DSC, is also plotted in Fig. 8. Its behaviour indicates that, as expected, the density of F atoms in the DSC decreases as the H2 flow rate increases up to about 100 sccm. Above this value, there are enough H2 molecules in the reactor to consume all F atoms of the dissociated NF3 molecules, and the intensity of the 703.7 fluorine line from the auxiliary plasma remains zero.

In conclusion, below 100 sccm of H2, this is the amount of H2 molecules of the feed gas which limits the quantity of HF produced inside the reactor but above 100 sccm, the dissociation rate of NF3 will become the limiting factor. However, as the amount of H2 in the feed gas becomes significant, its vibrational and rotational excitation will lead to a slight lowering of the electron temperature in the CCP plasma, resulting in a small diminution of the dissociation rate of NF3. This is the reason for the small enhancement of the partial pressure of NF3 after its minimum density at 100 sccm of H2 and the simultaneous lowering of the HF density, as seen in Fig. 8. However, the ratio of the partial pressures of HF over the dissociated NF3 remains close to 3, attesting that any of NF3 molecules that starts to be dissociated by losing a flour atom, is fully consumed by the chain reactions (2 and 9 to 11) to form, in the end, three HF molecules.

Kinetics of NF3/ NH3 plasmas

F and H atoms produced in the CCP chamber by the dissociation of NF3 and NH3, trigger a series of chain reactions which leads to the production of large amount of HF molecule. In table 1 are reported the measured partial pressures of NF3, NH3 and HF in the gas introduced into the reactor chamber (plasma off, 0 W) and for 3 different RF powers. The introduced gas mixture is NF3/ NH3/ He (50/50/1950 sccm) and the pressure in the DSC is fixed to 1 Torr. In the absence of plasma, the partial pressures of NF3 and NH3 are 25 and 27 mTorr, respectively. Striking the CCP plasma in the upper chamber with 150 W RF power, results in a drastic decrease (by factor 7) of the NH3 partial pressure in the down-stream chamber whereas that of NF3 is only divided by 2. This different behaviour is opposite to an expectation based on binding energy of F atom in NF3 (2.9 eV) and H atom in NH3 (4.1 eV), which should favour a higher dissociation rate of the former. But the difference reflects the fact that electron impact is the only process responsible for the dissociation of NF3 molecules whereas the so produced F atoms can also react with NH3, according to the reactions ( 4) and ( 12) [52,53,54,15]:

HF + NH3 → NH4F (12)
Reaction ( 4), which is very fast with k4= 1-4 x10 -16 m -3 .s -1 according to the temperature [30], can explain why the newly generated HF molecule, with its partial pressure of 28 mTorr, becomes the most abundant molecule in the DSC at the lowest 150 W of CCP power With increasing RF power, thus increasing plasma density in the CCP chamber, NF3 and NH3 are more dissociated but the enhancement of their dissociated parts, also reported in columns 5 and 7 of table 1, is much slowed down. Accordingly, the amount of the produced HF increases, but it seems to levels off above 300 W RF power. We note that when RF power exceeds 150 W, due to the excess of plasma generated H atoms, all F atoms produced by the dissociation of NF3 are converted to HF molecule.

Figure 9 shows the impact of the NH3 dilution (for a fixed NF3 flow rate of 50 sccm and total flow rate of 2050 sccm at 250 W CCP power) on the densities of NH3, NF3, HF and also on the amount of NH3 lost (compared to the plasma OFF). The variation of F is not shown because it drops rapidly below the detection limit as soon as 25 sccm of NH3 is added to NF3. The density of NF3, whose gas flow rate is constant at 50 sccm, is increasing slowly with the amount of NH3 injected in the discharge. This is attributed to the small decay of the electron temperature/density with increasing NH3 density in the mixture. However, these variations remain small for NH3 Furthermore, the amount of NH3 lost would also be expected to be constant for the same reason, although it shows a significant increase with the NH3 flow rate. As a matter of fact, there seems to be a correlation between the amount of NH3 lost and the amount of HF decreases after its maximum at 50 sccm of NH3 when its flow rate increases above 75 sccm. This is strongly suggesting that reaction 12 is taking place efficiently under our conditions, i.e. that NH3 reacts with HF to produce NH4F, the main precursor for the formation of salts on the wafer. The NH4F density can then be roughly estimated from the loss of HF or from the loss of NH3, as shown in Figure 10. There is a relatively good agreement between the two estimations, although a significant divergence is observed at high NH3 dilution where the NH4F density deduced from NH3 loss is significantly larger than that deduced from HF decrease. This is probably due to the assumption that the NH3 loss is caused solely by reactions with HF: the NH3 density is actually driven by several processes including for example the recombination NH2+HNH3 which depend on the H density and thus on the H recombination rate on the reactor walls to form H2 (which may be plasma chemistry dependent). Nevertheless, our results show that significant amount of NH4F is produced in the discharge at large NH3 dilutions, i.e. the processing conditions used to form salts selectively on the oxydized silicon or nitride surfaces.

conclusions

The chemistry of downstream plasmas used for isotropic etching of Si alloys is investigated by VUV absorption spectroscopy and OES both in NF3/H2 and NF3/NH3 chemistries. In both cases VUVAS indicate the formation of large amounts of HF molecules which are expected to play a role in oxide etching. F atoms are shown to react rapidly with H2 and NH3 to produce HF until all the fluorine is consumed. H also react rapidly with NF2 and NF, producing HF and consuming NFx radicals, which couldn't be detected by UVAS in the downstream chamber. In the NF3/NH3 chemistry, the HF density is shown to decrease when the NH3 flow rate is increased above the NF3 flow rate. This decay scales with the amount of NH3 that is lost, which is attributed to an efficient reaction between NH3 and HF to form the NH4F molecule that is supposed to be the precursor for salt formation on the wafer. The NH4F density could be estimated based on the quantity of HF an NH3 lost and the high NH3 dilution that is required to etch selectively oxidized material correspond to the largest values of the ratio NH4F/HF.

Figure Caption

Fig. 1: Schematic of the experimental set up showing the downstream chamber with the small deported plasma (used for OES). Absorption spectrum recorded with the plasma ON (250 W). This spectrum can be fitted in its high wavelength part by the green curve, which correspond to NF3 gas at 24 mTorr. By subtracting this NF3 contribution to the overall spectrum, we obtain the VUV absorption spectrum of HF molecules (blue line), corresponding to a pressure of 40 mTorr HF according to its absorption cross section.

  are shown absorption spectra recorded in the downstream chamber without and with CCP plasma in 1.6% NF3/ 2% NH3 diluted in helium gas at a total flow rate of 2 l/min and 1 Torr pressure in the DSC. The spectrum recorded in the absence of CCP plasma results from absorption by only NF3 and NH3 and can be reconstructed by adding absorbance spectra from 16 mT of NF3 and 20 mT of NH3, which indicates their partial pressures in the DSC. Striking a 350 W, CCP plasma results in and important diminution of the partial pressures of NF3 and NH3, which are highly dissociated by the discharge, and hence in weaker absorbance. The new absorption spectra with the CCP plasma ON can be reconstructed by adding absorbance spectra of 7.7 mT of NF3, 4 mT of NH3 and 20 mT of HF. Thus about 50% of NF3 and 80% of NH3 of the introduced gas have been dissociated by the plasma and, as will be discussed in following sections, the chain reactions of the dissociation products lead to the formation of HF molecules.

Figure 7

 7 illustrates the consumption of plasma generated F atoms and production of HF in NF3/ H2/ He (100/100/1900 sccm) gas mixture and pressure fixed at 1 Torr in the DSC. With increasing RF power of the CCP, the steady-state partial pressure of NF3 in the downstream chamber drops gradually from its initial value of 52 mTorr to 20 mTorr at 350 W. Simultaneously, the partial pressure of produced HF molecule increases, reaching 90 mTorr at 350 W. It is also shown in this figure the ratio between the partial pressure of HF and the missing partial pressure of NF3. This ratio clearly shows that three HF molecules are produced for one missing NF3 molecule, indicating that once a NF3 molecule starts to be dissociated, producing the first F atom, the chain reactions (2 and 9 to 11) will consume the other by-products of the dissociation (NF2 and NF) to form HF molecules. In the above presented example, the limiting factor of the chain reactions is the amount of the plasma dissociated NF3

Fig. 2 :

 2 Fig.2: Top view of the reactor showing the VUVAS experiment with the D2 light source and the VUV monochrmator equipped with the CCD camera.

Fig. 3 :

 3 Fig.3: VUV absorption spectrum of NH3 gas (no plasma) at several reactor filling pressures.

Fig. 4 :

 4 Fig.4: VUV absorption spectrum of NF3 gas (no plasma) at several reactor filling pressures.

Fig. 5 :

 5 Fig.5 : Black cross (+): absorption spectra recorded in a the NF3 (4%)/H2 (9%)/He gas and adjusted with a pure NF3 spectrum recorded at 40 mTorr (cyan line). Black open cirle (°):

Fig. 6 :Fig. 7

 67 Fig.6: Black aster (): absorption spectra recorded in a NF3/NH3/He gas mixture gas/He gas and adjusted (green line) with the spectra from the abaques withe the sum of 16 mtorr of NF3 and 20 mTorr of NH3. Black open circle: Absorption spectrum recorded with the plasma ON (250 W).This spectrum can be fitted by the sum of three spectra: HF (20 mTorr), NF3 (7.7mTorr) and NH3(4 mTorr)

Fig. 8 :Fig. 9 :

 89 Fig. 8 : impact of the H2 dilution in NF3 on the densities of NF3, HF, F and on the ratio of produced HF per NF3 lost

Fig. 10 : 4 F

 104 Fig.10: variation of the HF density as a function of the NH3 dilution in the NF3(50sccm)/NH3 plasma. The magenta and green curves indicate the density of NH4F deduced respectively from the loss of HF and from the loss of NH3 (assuming that NH3 reacts with HF to form NH4F)

Table 1 -

 1 Measured partial pressures of NF3, NH3 and HF at different RF power of CCP, together with dissociated parts of NF3 and NH3 and the ratios of HF over dissociated parts.

  flow rate above 75 scmm: the amount of NF3 loss (i.e. dissociated to produce F atoms) is almost constant in the range of 100 to 400 sccm NH3 flow rate. The density of HF rises rapidly to reach a maximum value at 50 sccm NH3, while at the same time the NH3 density remains negligibly low. This behaviour is in good agreement with the results of Table1and with reaction (4): F atoms produced from NF3 dissociation reacts rapidly with NH3 to produce HF. As far there is enough F available, all the injected NH3 molecules are consumed by reaction (4) and the HF density rise accordingly. The NH3 is fully dissociated for its flow up to approximately 50 sccm, at which point there is no more F atoms available: the NH3 density then increase and above 75 sccm it rises almost linearly with the NH3 flow rate. Surprisingly the HF density appears to decay significantly when the NH3 flow rate overpass about 50 sccm : once all F atoms available are consumed by NH3 to form HF, one would expect the HF density to reach a steady state.

ACKNOWLEDGMENTS

This work was partly supported by LabEx Minos (No. ANR-10-LABX-55-01) and by the French RENATECH network. The authors would like to acknowledge Applied Materials for their technical support.