Pavel Tokmakov

Martial Hebert

Cordelia Schmid

Unsupervised Learning of Video Representations via Dense Trajectory Clustering

Keywords:

This paper addresses the task of unsupervised learning of representations for action recognition in videos. Previous works proposed to utilize future prediction, or other domain-specific objectives to train a network, but achieved only limited success. In contrast, in the relevant field of image representation learning, simpler, discrimination-based methods have recently bridged the gap to fully-supervised performance. We first propose to adapt two top performing objectives in this class -instance recognition and local aggregation, to the video domain. In particular, the latter approach iterates between clustering the videos in the feature space of a network and updating it to respect the cluster with a non-parametric classification loss. We observe promising performance, but qualitative analysis shows that the learned representations fail to capture motion patterns, grouping the videos based on appearance. To mitigate this issue, we turn to the heuristic-based IDT descriptors, that were manually designed to encode motion patterns in videos. We form the clusters in the IDT space, using these descriptors as a an unsupervised prior in the iterative local aggregation algorithm. Our experiments demonstrates that this approach outperform prior work on UCF101 and HMDB51 action recognition benchmarks 3 . We also qualitatively analyze the learned representations and show that they successfully capture video dynamics.

Introduction

The research on self-supervised learning of image representation has recently experienced a major breakthrough. Early approaches carefully designed objective functions to capture properties that the authors believed would result in learning rich representations [START_REF] Doersch | Unsupervised visual representation learning by context prediction[END_REF][START_REF] Noroozi | Unsupervised learning of visual representations by solving jigsaw puzzles[END_REF][START_REF] Gidaris | Unsupervised representation learning by predicting image rotations[END_REF][START_REF] Zhang | Colorful image colorization[END_REF]. For instance, Doersch et al. [START_REF] Doersch | Unsupervised visual representation learning by context prediction[END_REF] proposed to predict relative positions of two patches in an image, and Zhang et al. [START_REF] Zhang | Colorful image colorization[END_REF] trained a network to colorize images. However, they have achieved only limited success. The methods that have brought the performance of self-supervised image representations close to those learned in a fully-supervised way, rely on a different principle instead. They use the standard cross-entropy loss and either treat each image as an individual class [START_REF] Dosovitskiy | Discriminative unsupervised feature learning with convolutional neural networks[END_REF][START_REF] Wu | Unsupervised feature learning via nonparametric instance discrimination[END_REF][START_REF] Oord | Representation learning with contrastive predictive coding[END_REF], or switch between clustering images in the feature space of the network, and updating the model to classify them into clusters [START_REF] Caron | Deep clustering for unsupervised learning of visual features[END_REF][START_REF] Zhuang | Local aggregation for unsupervised learning of visual embeddings[END_REF]. The resulting representations effectively capture discriminative image cues without having to manually separate images into categories.

Self-supervised feature learning for videos has so far mostly relied on manually designed objective functions. While some works adopted their objectives directly from the image-based methods, such as predicting video rotation [START_REF] Jing | Self-supervised spatiotemporal feature learning via video rotation prediction[END_REF], or relative position of space-time patches [START_REF] Kim | Self-supervised video representation learning with space-time cubic puzzles[END_REF], others utilize video-specific cues, such as predicting feature representations of video patches in future frames [START_REF] Han | Video representation learning by dense predictive coding[END_REF]. Very recently, Sun et al. [START_REF] Sun | Contrastive bidirectional transformer for temporal representation learning[END_REF], have proposed a variant of the instance classification objective for videos.

In this work we first investigating whether the recent, classification-based objectives proposed for image representation learning can be applied to videos. We introduce a video variant of the non-parametric Instance Recognition approach of Wu et al., [START_REF] Wu | Unsupervised feature learning via nonparametric instance discrimination[END_REF] (Video IR). It simply treats each video as its own class and trains a 3D ConvNet [START_REF] Tran | Learning spatiotemporal features with 3D convolutional networks[END_REF][START_REF] Hara | Can spatiotemporal 3D CNNs retrace the history of 2D CNNs and ImageNet?[END_REF] to discriminate between the videos. We observe that this naive approach is already competitive with prior work in the video domain.

To further improve the results, we capitalize on the observation of Zhuang et al. [START_REF] Zhuang | Local aggregation for unsupervised learning of visual embeddings[END_REF] that embedding semantically similar instances close to each other in feature space is equally important to being able to discriminate between any two of them. We adapt their Local Aggregation approach to videos (Video LA). As shown in the top part of Figure 1, this method first encodes a video using a 3D ConvNet, and the resulting embeddings are clustered with K-means. A non-parametric clustering loss proposed in [START_REF] Zhuang | Local aggregation for unsupervised learning of visual embeddings[END_REF] is then used to update the network and the algorithm is iterated in an Expectation-Maximization framework. This approach results in an improvement over Video IR, but the gap between the two objectives remains smaller than in the image domain.

We identify the reasons behind this phenomenon, by examining the video clusters discovered by the algorithm. Our analysis shows that they mainly capture appearance cues, such as scene category, and tend to ignore the temporal information, which is crucial for the downstream task of action recognition. For instance, as shown in the top right corner of Figure 1, videos with similar background, but different activities are embedded closer than examples of the same action. This is not surprising, since appearance cues are both dominant in the data itself, and are better reflected in the 3D ConvNet architecture.

To mitigate this issue, we turn to the heuristic-based video representations of the past. Improved Dense Trajectories (IDT) [START_REF] Wang | Action recognition with improved trajectories[END_REF] were the state-of-the-art approach for action recognition in the pre-deep learning era, and remained competitive on some datasets until very recently. The idea behind IDT is to manually encode the cues in videos that help to discriminate between human actions. To this end, individual pixels are first tracked with optical flow, and heuristics-based descriptors [START_REF] Dalal | Histograms of oriented gradients for human detection[END_REF][START_REF] Dalal | Human detection using oriented histograms of flow and appearance[END_REF][START_REF] Wang | Dense trajectories and motion boundary descriptors for action recognition[END_REF] are aggregated along the trajectories to encode both appearance and motion cues.

In this work, we propose to transfer the notion of similarity between videos encoded in IDTs to 3D ConvNets via non-parametric clustering. To this end, we first compute IDT descriptors for a collection of unlabeled videos. We then cluster these videos in the resulting features space and use the non-parametric classification objective of [START_REF] Zhuang | Local aggregation for unsupervised learning of visual embeddings[END_REF] to train a 3D ConvNet to respect the discovered clusters (bottom part of Figure 1). The network is first trained until convergence using the fixed IDT clusters, and then finetuned in the joint IDT and 3D ConvNet space with the iterative Video LA approach. The resulting representation outperforms the baselines described above by a significant margin. We also qualitatively analyze the clusters and find that they effectively capture motion information.

Following prior work [START_REF] Han | Video representation learning by dense predictive coding[END_REF][START_REF] Jing | Self-supervised spatiotemporal feature learning via video rotation prediction[END_REF][START_REF] Sun | Contrastive bidirectional transformer for temporal representation learning[END_REF], we use the large-scale Kinetics [START_REF] Carreira | Quo vadis, action recognition? a new model and the kinetics dataset[END_REF] dataset for self-supervised pretraining, ignoring the labels. The learned representations are evaluated by finetuing on UCF101 [START_REF] Soomro | UCF101: A dataset of 101 human actions classes from videos in the wild[END_REF] and HMDB51 [START_REF] Kuehne | HMDB: a large video database for human motion recognition[END_REF] action recognition benchmarks. To gain a better insight into the quality of the representations, we additionally provide an evaluation in a few-shot regime, using the model as a fixed feature extractor.

Related work

In this section, we first briefly review previous work on image-based unsupervised representation learning. We then discuss various approaches to video modeling, and conclude by presenting relevant video representation learning methods.

Image representation learning from unlabeled data is a well explored topic. Due to space limitations, we will only review the most relevant approaches here. The earliest methods were built around auto-encoder architectures: one network is trained to compress an image into a vector in such a way, that another network is able to reconstruct the original image from the encoding [START_REF] Hinton | A fast learning algorithm for deep belief nets[END_REF][START_REF] Lee | Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations[END_REF][START_REF] Kingma | Auto-encoding variational bayes[END_REF][START_REF] Donahue | Adversarial feature learning[END_REF][START_REF] Goodfellow | Generative adversarial nets[END_REF]. In practice, however, the success of generative methods in discriminative representation learning has been limited.

Until very recently, manually designing self-supervised objectives has been the the dominant paradigm. For example, Doersch et al. [START_REF] Doersch | Unsupervised visual representation learning by context prediction[END_REF] and Noroozi and Favaro [START_REF] Noroozi | Unsupervised learning of visual representations by solving jigsaw puzzles[END_REF] predict relative positions of patches in an image, Zhang et al. [START_REF] Zhang | Colorful image colorization[END_REF] learn to colorize images, and Gidaris et al. [START_REF] Gidaris | Unsupervised representation learning by predicting image rotations[END_REF] learn to recognize image rotations. While these methods have shown some performance improvements compared to random network initialization, they remain significantly below a fully-supervised baseline. The most recent methods, instead of designing specialized objective functions, propose to use the standard cross-entropy loss and either treat every image as its own class [START_REF] Dosovitskiy | Discriminative unsupervised feature learning with convolutional neural networks[END_REF][START_REF] Oord | Representation learning with contrastive predictive coding[END_REF][START_REF] Wu | Unsupervised feature learning via nonparametric instance discrimination[END_REF], or switch between clustering the examples in the feature space of the network and updating the network with a classification loss to respect the clusters [START_REF] Caron | Deep clustering for unsupervised learning of visual features[END_REF][START_REF] Zhuang | Local aggregation for unsupervised learning of visual embeddings[END_REF]. These methods exploit the structural similarity between semantically similar images, to automatically learn a semantic image embedding. In this paper we adapt the methods of Wu et al. [START_REF] Wu | Unsupervised feature learning via nonparametric instance discrimination[END_REF] and Zhuang et al. [START_REF] Zhuang | Local aggregation for unsupervised learning of visual embeddings[END_REF] to the video domain, but demonstrate that they do not perform as well due to the structural priors being less strong in videos. We then introduce explicit prior in the form of IDT descriptors and show this indeed improves performance.

Video modeling has traditionally been approached with heuristics-based methods. Most notably, Dense Trajectories (DT) [START_REF] Wang | Dense trajectories and motion boundary descriptors for action recognition[END_REF] sample points in frames and track them with optical flow. Then appearance and motion descriptors are extracted along each track and encoded into a single vector. The discriminative ability of DT descriptors was later improved in [START_REF] Wang | Action recognition with improved trajectories[END_REF] by suppressing camera motion with the help of a human detector, and removing trajectories that fall into background regions. The resulting representation focuses on relevant regions in videos (humans and objects in motion) and encodes both their appearance and motion patterns.

More recently, the success of end-to-end trainable CNN representation has been extended to the video domain. Simonyan et al. [START_REF] Simonyan | Two-stream convolutional networks for action recognition in videos[END_REF] proposed to directly train 2D CNNs for action recognition, fusing several frames at the first layer of the network. Their approach, however, had a very limited capacity for modeling temporal information. This issue was later addressed in [START_REF] Tran | Learning spatiotemporal features with 3D convolutional networks[END_REF] by extending the 2D convolution operation in time. Introduction of the large scale Kinetcis dataset for action recognition [START_REF] Carreira | Quo vadis, action recognition? a new model and the kinetics dataset[END_REF] was a major step forward for 3D CNNs. Pretrained on this dataset, they were finally able to outperform the traditional, heuristicbased representations. Several variants of 3D ConvNet architectures have been proposed since, to improve performance and efficiency [START_REF] Carreira | Quo vadis, action recognition? a new model and the kinetics dataset[END_REF][START_REF] Hara | Can spatiotemporal 3D CNNs retrace the history of 2D CNNs and ImageNet?[END_REF][START_REF] Xie | Rethinking spatiotemporal feature learning for video understanding[END_REF]. In this work, we demonstrate how the IDT descriptors can be used to improve unsupervised learning of 3D ConvNet representations.

Video representation learning from unlabeled data is a less explored topic. This is largely because the community has only recently converged upon the 3D ConvNets as thr standard architecture. Early methods used recurrent networks, or 2D CNNs, and relied on future-prediction [START_REF] Srivastava | Unsupervised learning of video representations using LSTMs[END_REF], as well as various manually designed objectives [START_REF] Mobahi | Deep learning from temporal coherence in video[END_REF][START_REF] Misra | Shuffle and learn: unsupervised learning using temporal order verification[END_REF][START_REF] Lee | Unsupervised representation learning by sorting sequences[END_REF][START_REF] Gan | Geometry guided convolutional neural networks for self-supervised video representation learning[END_REF][START_REF] Fernando | Self-supervised video representation learning with odd-one-out networks[END_REF]. In particular, several works utilized temporal consistency between consecutive frames as a learning signal [START_REF] Misra | Shuffle and learn: unsupervised learning using temporal order verification[END_REF][START_REF] Lee | Unsupervised representation learning by sorting sequences[END_REF][START_REF] Mobahi | Deep learning from temporal coherence in video[END_REF], whereas Gan et al. [START_REF] Gan | Geometry guided convolutional neural networks for self-supervised video representation learning[END_REF] used geometric cues, and Fernando et al. [START_REF] Fernando | Self-supervised video representation learning with odd-one-out networks[END_REF] proposed the oddone-out objective function.

With 3D ConvNets, generative architectures [START_REF] Kim | Self-supervised video representation learning with space-time cubic puzzles[END_REF][START_REF] Vondrick | Generating videos with scene dynamics[END_REF], as well as some selfsupervised objectives have been explored [START_REF] Jing | Self-supervised spatiotemporal feature learning via video rotation prediction[END_REF][START_REF] Kim | Self-supervised video representation learning with space-time cubic puzzles[END_REF][START_REF] Wang | Self-supervised spatio-temporal representation learning for videos by predicting motion and appearance statistics[END_REF]. For example, Jing et al. [START_REF] Jing | Self-supervised spatiotemporal feature learning via video rotation prediction[END_REF] train a model to predict video rotation, Kim et al. [START_REF] Kim | Self-supervised video representation learning with space-time cubic puzzles[END_REF] use relative spatio-temporal patch location prediction as an objective, and Wang et al. [START_REF] Wang | Self-supervised spatio-temporal representation learning for videos by predicting motion and appearance statistics[END_REF] regress motion and appearance statistics. In another line of work, future frame colorization was explored as a self-supervision signal [START_REF] Vondrick | Tracking emerges by colorizing videos[END_REF]. Recently, Han et al. [START_REF] Han | Video representation learning by dense predictive coding[END_REF] proposed to predict feature representations of video patches in future frames. Most similarly, Sun et al. [START_REF] Sun | Contrastive bidirectional transformer for temporal representation learning[END_REF] use a variant of the instance discrimination loss. In this work, we demonstrate that simply adapting instance discrimination [START_REF] Wu | Unsupervised feature learning via nonparametric instance discrimination[END_REF] and local aggregation [START_REF] Zhuang | Local aggregation for unsupervised learning of visual embeddings[END_REF] objectives from the image to the video domain already achieves competitive results, and augmenting local aggregation with IDT priors further improves the results, outperforming the state-of-the-art.

Method

Our goal is to learn an embedding function f θ that maps videos V = {v 1 , v 2 , ..., v N } into compact descriptors f θ (v i ) = d i in such a way, that they can be discriminated based on human actions, using unlabeled videos. For instance, as shown in Figure 1, we want the two videos of people to doing handstands to be close to each other in the embedding space, and well separated from the video of a person training a dog. Below, we first introduce the two objective functions used in our work -instance recognition [START_REF] Wu | Unsupervised feature learning via nonparametric instance discrimination[END_REF] and local aggregation [START_REF] Zhuang | Local aggregation for unsupervised learning of visual embeddings[END_REF], and then describe our approach of using IDT [START_REF] Wang | Action recognition with improved trajectories[END_REF] descriptors as unsupervised priors in non-parametric clustering.

Video instance recognition

This objective is based on the intuition that the best way to learn a discriminative representation is to use a discriminative loss. And, in the absence of supervised class labels, treating each instance as a distinct class of its own is a natural surrogate.

Using the standard softmax classification criterion, the probability of every video v with the feature d belonging to its own class i is expressed as:

P (i|d) = exp(w T i d) N j=1 exp(w T j d) , (1)
where w j is the weight vector of the j'th classifier. In this case, however, every class contains only a single example, thus w j can be directly replaced with d j .

The authors of [START_REF] Wu | Unsupervised feature learning via nonparametric instance discrimination[END_REF] then propose the following formulation of the class probability:

P (i|d) = exp(d T i d/τ ) N j=1 exp(d T j d/τ ) , ( 2
)
where τ is a temperature parameter that controls the concentration level of the distribution, and helps convergence [START_REF] Wang | Normface: L2 hypersphere embedding for face verification[END_REF][START_REF] Hinton | Distilling the knowledge in a neural network[END_REF]. The final learning objective is the standard negative log likelihood over the training set. Recall that training is done in batches, thus a memory bank of encodings D = {d 1 , d 2 , ..., d N } has to be maintained to compute Equation 2.

Video local aggregation

While being able to separate any two instances is a key property for an image or video embedding space, another, complementary and equally desirable property is minimizing the distance between semantically similar instances. To this end, Zhuang et al. [START_REF] Zhuang | Local aggregation for unsupervised learning of visual embeddings[END_REF] proposed to use clusters of instances instead of individual examples as class surrogates. We adapt their approach to the video domain, and briefly describe it below. Firstly, the video embedding vectors

d 1 , d 2 , ..., d N are grouped into K clusters G = {G 1 , G 2 , .
., G K } using K-means. The embedding function f θ is then updated to respect the cluster, using the non-parametric clustering objective proposed in [START_REF] Zhuang | Local aggregation for unsupervised learning of visual embeddings[END_REF], and the two steps are iterated in an EM-framework. In particular, for every instance v i together with its embedding d i , two sets of neighbours are identified: close neighbours C i (shown with a dashed circle in Figure 1) and Fig. 1. Our approach for unsupervised representation learning from video collections. Directly applying a non-parametric clustering objective results in a representation that groups videos based on appearance (top right corner). To mitigate this issue, we propose to first cluster the videos in the space of IDT descriptors (bottom right corner), which results in a grouping that better reflects video dynamics. We then apply the non-parametric clustering loss to transfer the properties of this embedding to a 3D ConvNet.

background neighbours B i . Intuitively, close neighbours are those examples that fall into the same cluster as v i and background neighbors are simply those that have a small distance to d i in the feature space (they include both close neighbors and hard negative examples). Please see [START_REF] Zhuang | Local aggregation for unsupervised learning of visual embeddings[END_REF] for more details on how C i and B i are constructed.

The objective is then to minimize the distance between d i and its close neighbours (instances in the same cluster), while maximizing the distance to those background neighbors that are not in C i (hard negatives). The authors formulate this objective in a probabilistic way as minimizing the negative log likelihood of d i being recognized as a close neighbor, given that it is recognized as a background neighbor:

L(C i , B i |d i , θ) = -log P (C i ∩ B i |d i ) P (B i |d i ) , (3)
where the probability of d being a member of a set A is defined as:

P (A|d) = i∈A P (i|d), (4)
and the definition of P (i|d) is adapted from Equation 2. Despite the involved formulation, one can see that this objective does exactly what it is intended to do -minimizes the distance between examples inside a cluster and maximize it between those belonging to different clusters in a non-parametric way. Intuitively, the Local Aggregation objective relies on the structural similarity between semantically similar images, together with deep image prior in CNN architectures [START_REF] Ulyanov | Deep image prior[END_REF], to form meaningful clusters in the embedding space. In videos, however, both structural and architectural priors are less strong. Indeed, pixels that are close to each other in the spatio-temporal volume of a video are not always strongly correlated due to the presence of object and camera and motion. On the architecture side, 3D ConvNets are also worse at capturing spatio-temoral patterns, compared to CNNs at capturing spatial patterns. To mitigate this lack of implicit priors, we propose to introduce an explicit one in the form of IDT descriptors.

IDT descriptors as priors for video representation learning

While state-of-the-art architectures for action recognition [START_REF] Tran | Learning spatiotemporal features with 3D convolutional networks[END_REF][START_REF] Carreira | Quo vadis, action recognition? a new model and the kinetics dataset[END_REF][START_REF] Hara | Can spatiotemporal 3D CNNs retrace the history of 2D CNNs and ImageNet?[END_REF] simply extend 2D CNN filters into the temporal dimension, treating videos as spatio-temporal cuboids of pixels, classical approaches [START_REF] Wang | Dense trajectories and motion boundary descriptors for action recognition[END_REF][START_REF] Wang | Action recognition with improved trajectories[END_REF] explicitly identified and encoded spatio-temporal interest points that are rich in motion patterns relevant to action classification. In our experiments, we use the original implementation of IDT [START_REF] Wang | Action recognition with improved trajectories[END_REF] to compute video descriptors for unlabeled videos (shown in the lower part of Figure 1). We supply the IDT extractor with human detection form the state-ofthe-art Mask-RCNN [START_REF] He | Mask R-CNN[END_REF] model trained on MS COCO [START_REF] Lin | Microsoft COCO: Common objects in context[END_REF] for improved camera stabilization (see [START_REF] Wang | Action recognition with improved trajectories[END_REF] for details).

This method, however, produces thousands of descriptors x ∈ X per video. To encode them into a compact vector we follow prior work [START_REF] Wang | Action recognition with improved trajectories[END_REF][START_REF] Wang | Hallucinating IDT descriptors and I3D optical flow features for action recognition with CNNs[END_REF] and first apply PCA to reduce the dimensionality of each individual trajectory descriptor x i . We then utilize Fisher vector coding [START_REF] Perronnin | Improving the fisher kernel for large-scale image classification[END_REF], which is based on a Gaussian Mixture Model (GMM) with K components G(w k , µ k , σ k ), parameterized by mixing probability, mean, and diagonal standard deviation. The encoding for a trajectory descriptor x is then computed by stacking the derivatives of each components of the GMM with respect to mean and variance:

φ * k (x) = p(µ k |x) √ w k [φ k (x), φ k (x) √ 2 ], (5)
where the first-and second-order features φ k , φ k ∈ R D are defined as:

φ k (x) = (x -µ k ) σ k , φ k (x) = φ k (x) 2 -1, (6)
thus, the resulting Fisher vector encoding φ

(x) = [φ * 1 (x), φ * 2 (x), ..., φ * k (x)
] is of dimensionality 2KD. To obtain the video-level descriptor ψ, individual trajectory encodings are averaged ψ = avg x∈X φ(x), and power- [START_REF] Koniusz | A deeper look at power normalizations[END_REF] and l2-normalization are applied. Finally, to further reduce dimensionality, count sketching [START_REF] Weinberger | Feature hashing for large scale multitask learning[END_REF] is used: p(ψ) = P ψ, where P is the sketch projection matrix (see [START_REF] Weinberger | Feature hashing for large scale multitask learning[END_REF] for details).

The resulting encoding p(ψ) is a 2000-dimensional vector, providing a compact representation of a video, which captures discriminative motion and appearance information. Importantly, it is completely unsupervised. Both the PCA projection and the parameters of the Gaussian mixture model are estimated using a random sample of trajectory encodings, and matrix P is selected at random as well.

To transfer the cues encoded in IDTs descriptors to a 3D ConvNet, we first cluster the videos in the p(ψ) space with K-means, to obtain the clusters G. We then use G to compute the sets of neighborhoods (C i , B i ) for each video v i in an unlabeled collection (shown in the bottom right corner on Figure 1), and apply the objective in Equation 3 to train the network. This forces the learned representation to capture the motion patterns that dominate the IDT space (note that IDTs encode appearance cues as well in the form of HOG descriptors).

Finally, we construct a joint space of IDT and 3D ConvNet representations by concatenating the vectors d and p(ψ) for each video. We further finetune the network in this joint space for a few epochs. This step allows the model to capitalize on appearance cues encoded by the the expressive 3D ConvNet architecture. We analyze the resulting model quantitatively and qualitatively, and find that it both outperforms the state-of-the-art, and is better at capturing motion information.

Experiments

Datasets and evaluation

We use the Kinetics [START_REF] Carreira | Quo vadis, action recognition? a new model and the kinetics dataset[END_REF] dataset for unsupervised representation learning and evaluate the learned models on UCF101 [START_REF] Soomro | UCF101: A dataset of 101 human actions classes from videos in the wild[END_REF] and HMDB51 [START_REF] Kuehne | HMDB: a large video database for human motion recognition[END_REF] in a fully-supervised regime. Below, we describe each dataset in more detail.

Kinetics is a large-scale, action classification dataset collected by querying videos on YouTube. We use the training set of Kinetics-400, which contains 235 000 videos, for most of the experiments in the paper, but additionally report results using fewer as well as more videos in Section 4.7. Note that we do not use any annotations provided in Kinetics.

UCF101 is a classic dataset for human action recognition, which consists of 13,320 videos, covering 101 action classes. It is much smaller than Kinetics, and 3D ConvNets fail to outperform heuristic-based methods on it without fully-supervised pretraining on larger datasets. Following prior work [START_REF] Jing | Self-supervised spatiotemporal feature learning via video rotation prediction[END_REF][START_REF] Han | Video representation learning by dense predictive coding[END_REF], we use UCF101 to evaluate the quality of representations learned on Kinetics in an unsupervised way via transfer learning. In addition to using the full training set of UCF101, we report few-shot learning results to gain more insight into the learned representations. We use the first split of the dataset for ablation analysis, and report results averaged over all splits when comparing to prior work.

HMDB51 is another benchmark for action recognition, which consists of 6,770 videos, collected from movies, and split into 51 categories. Due to the small size of the training set, it, poses an even larger challenge for learning-based methods. As with UCF101, we report ablation results on the first split, and use the results averaged over all splits for comparison to prior work.

Following standard protocol, we report classification accuracy as the main evaluation criteria on UCF101 and HMDB51. However, this makes direct comparison between different approaches difficult, due to the differences in network architectures. Thus, whenever possible, we additionally report the fraction of the fully-supervised performance for the same architecture.

Implementation details

Self-supervised objectives We study three self-supervised objective functions: Video Instance Recognition (Video IR), Video Local Aggregation (Video LA) and Video Local Aggregation with IDT prior. For Video IR we follow the setting of [START_REF] Wu | Unsupervised feature learning via nonparametric instance discrimination[END_REF] and set τ in Equation 2 to 0.07. We use 4096 negative samples for approximating the denominator of Equation 2.

In addition to the parameters described above, Local Aggregation requires choosing the number of clusters K, as well as the number of runs of K-means that are combined for robustness. The authors of [START_REF] Zhuang | Local aggregation for unsupervised learning of visual embeddings[END_REF] do not provide clear guidelines on selecting these hyperparameters, so we choose to take the values used in their ImageNet experiments and decrease them proportionally to the size of Kinetics. As a result, we set K to 6000 and the number of clusterings to 3. We validate the importance of this choice in Appendix B.

For experiments with with IDT priors we use exactly the same hyperparameters for the LA objective as described above. We use the original implementation of [START_REF] Wang | Action recognition with improved trajectories[END_REF] to extract IDT descriptors. Human detections are computed with ResNet101 variant of Mask-RCNN [START_REF] He | Mask R-CNN[END_REF] model pretrained on MS COCO [START_REF] Lin | Microsoft COCO: Common objects in context[END_REF]. We evaluate the importance of human detections for the final performance of our approach in Appendix A. When computing Fisher vector encoding, we generally follow the setting of [START_REF] Wang | Hallucinating IDT descriptors and I3D optical flow features for action recognition with CNNs[END_REF]. In particular, we set the feature importance to 90% when computing PCA, and the number of components in GMM to 256. When fitting the PCA and GMM models we randomly choose 3500 videos from Kinetics and 500 IDT descriptors from each video, to get a representative sample. Note that extracting IDTs and encoding them into Fisher vectors does not require GPUs, and thus the code can be efficiently run in parallel on a CPU cluster. As a result, we were able to compute the descriptors for Kinetics in just 5 days. Network architecture and optimization Following most of the prior work, we use a 3D ResNet18 architecture [START_REF] Hara | Can spatiotemporal 3D CNNs retrace the history of 2D CNNs and ImageNet?[END_REF] in all the experiments, but also report results with deeper variants in Appendix C. The embedding dimension for selfsupervised objectives is set to 128, as in [START_REF] Zhuang | Local aggregation for unsupervised learning of visual embeddings[END_REF]. We use SGD with momentum to train the networks, and apply multi-scale, random spatio-temporal cropping for data augmentation, with exactly the same setting as in [START_REF] Hara | Can spatiotemporal 3D CNNs retrace the history of 2D CNNs and ImageNet?[END_REF]. We also perform the standard mean subtraction. All the models are trained on 16 frames clips of spatial resolution of 112 × 112, unless stated otherwise.

During self-supervised learning we follow the setting of [START_REF] Zhuang | Local aggregation for unsupervised learning of visual embeddings[END_REF] and set the learning rate to 0.03, and momentum to 0.9, with batch size of 256. All the models are trained for 200 epoch, and the learning rate is dropped by a factor 0.1 at epochs 160 and 190. As in [START_REF] Zhuang | Local aggregation for unsupervised learning of visual embeddings[END_REF], we initialize the LA models with 40 epoch of IR pretraining.

When finetuning on UCF101 and HMDB51, we set the learning rate to 0.1 and momentum to 0.9, using batch size 128. We drop the learning rate by a factor of 0.1 when the validation performance stops improving. Following [START_REF] Jing | Self-supervised spatiotemporal feature learning via video rotation prediction[END_REF], we freeze the first ResNet block when finetuning on UCF101, and the first two blocks on HMDB51 to avoid overfitting. During inference, for every video we sample five clips at random, using the center crop. The final prediction is obtained by averaging softmax scores over the five clips. For few-shot experiments, we use the protocol of [START_REF] Chen | A closer look at few-shot classification[END_REF] and freeze the entire network, only learning a linear classifier.

Analysis of self-supervised objectives

We begin by comparing different variants of self-supervised objectives described in Section 3. They are used to learn a representation on Kinetics-400 in a selfsupervised way, and the resulting models are transferred to UCF101 and HMDB51. We additionally evaluate two baselines -Supervised, which is pretrained on Kinetics using ground-truth labels, and Scratch, which is initialized with random weights. The results are reported in Table 1.

Firstly, we observe that supervised pretraining is indeed crucial for achieving top performance on both datasets, with the variant trained from scratch reaching only 50.2% and 30.3% of the accuracy of the fully supervised model on UCF101 and HMDB51 respectively. The gap is especially large on HMDB51, due to the small size of the dataset. Using the video variant of the Instance Recognition objective (Video IR in the table), however, results in a 27.6% accuracy improvement on UCF101 and 22.8% HMDB51, reaching 82.9% and 70.7% of the supervised accuracy respectively. Notice that this simple method already outperforms some of the approaches proposed in prior works [START_REF] Jing | Self-supervised spatiotemporal feature learning via video rotation prediction[END_REF][START_REF] Han | Video representation learning by dense predictive coding[END_REF][START_REF] Kim | Self-supervised video representation learning with space-time cubic puzzles[END_REF].

Next, we can see that the Local Aggregation objective (Video LA in the table) further improves the results, reaching 84.6% and 73.9% of the fully-supervised performance on UCF101 and HMDB51 respectively. This shows that despite the higher-dimensionality of the video data, this method is still able to discover meaningful clusters in an unsupervised way. However, the gap to the IR objective is smaller than in the image domain [START_REF] Zhuang | Local aggregation for unsupervised learning of visual embeddings[END_REF].

Finally, our full method, which uses IDT descriptors as an unsupervised prior when clustering the videos (Video LA + IDT prior in the table), is indeed able to further boost the performance, reaching 86.3% and 78.0% of fully supervised performance on the two datasets. The improvement over Video LA is especially significant on HMDB51. We explain this by the fact that categories in UCF101 are largely explainable by appearance, thus the benefits of better modeling the temporal information are limited on this dataset. In contrast, on HMDB51 capturing scene dynamics is crucial for accurate classification.

Few-shot evaluation

When finetuning a model, even on a datasets of modest size, like UCF101, the effect of self-supervised pretraining is confounded by the effectiveness of the adaptation strategy itself. Indeed, it has been show recently that, on several tasks that were traditionally used to measure the effectiveness of image-based unsupervised learning approaches, fully supervised performance can be achieved with no pretraining at all, by simply better utilizing the existing data [START_REF] He | Rethinking ImageNet pre-training[END_REF]. Thus, to gain more insight into our objectives, we propose to use pretrained models as feature extractors, and learn linear classifiers in a few-shot regime. The results on UCF101 are reported in Table 2.

The most important observation here is that the gap between fully-supervised and unsupervised representations increases as the data becomes scarcer. This shows that, despite being useful in practice, unsupervised pretraining is still far from making large datasets obsolete. Among the objectives studied in our work, however, Video LA with IDT prior shows the strongest performance across the board, and is especially effective in the low-data regime.

Qualitative analysis of the representations

To gain further insight into the effect of our IDT prior on representation learning, we now visualize some of the clusters discovered by the vanilla LA, and the variant with the prior in Figures 2 and3 respectively. Firstly, we observe that, in the absence of external constraints LA defaults to using appearance, and primarily scene information to cluster the videos. For instance, the first cluster (top left corner) corresponds to swimming pools, the one on the top right seems to focus on grass, and the two clusters in the bottom row capture vehicles and backyards, irrespective of the actual scene dynamics. This is not surprising, since appearance cues are both more dominant in the data itself, and are better reflected by the 3D ConvNet architecture. Table 3. Evaluation of the effect of clip length on the Video LA objective with and without IDT prior on the first split of UCF101 and HMDB51 using classification accuracy. Scratch and Supervised baselines are also reported. All models use a 3D ResNet18 backbone, and take frames with resolution of 112 × 112 as input. Both self-supervised and fully-supervised variants benefit from longer sequences, but the model trained from scratch is not able to capitalize on more information. In contrast, the model learned with IDT prior is better at capturing motion cues. For example, the cluster in the top left corner of Figure 3 is characterized by forward-backward hand motion, such as observed during cleaning or barbecuing. The cluster in the top-right captures humans spinning or rotating. The bottom left cluster mostly contains videos with very fast actor motion, and the one in the bottom right closely corresponds to the action 'riding'.

Method

Importantly, neither set of clusters is perfectly aligned with the definition of actions in popular computer vision dataset. For instance, despite having a clear motion-based interpretation, the top left cluster in Figure 3 combines Kinetcis categories 'cleaning window', 'cleaning floor', and 'barbecuing'. Indeed, the actions vocabulary used in the literature is defined by a complex combination of actor's motion and scene appearance, making automatic discovery of wellaligned clusters challenging, and partially explaining the remaining gap between clustering-based methods and fully-supervised pretraining.

Learning long-term temporal dependencies

Next, we experiment with applying our Video LA objective with IDT prior over longer clips. Recall that this approach attempts to capture the notion of similarity between the videos encoded in the IDT descriptors that are computed over the whole video. The model reported so far, however, only takes 16-frame clips as input, which makes the objective highly ambiguous. In Table 3 we evaluate networks trained using 32-and 64-frame long clips instead, reporting results on UCF101 and HMDB51.

We observe that, as expected, performance of our approach ('Video LA + IDT' in the table) increases with more temporal information, but the improvement is non-linear, and our model is indeed able to better capture long-term motion cues when trained using longer clips. Similar improvements are observed for the plain Video LA objective, but our approach still shows top performance. Supervised model is also able to capitalize on longer videos, but on UCF101 the improvements are lower than seen by our approach (6.8% for the supervised model, compared to 8.7% for ours). Interestingly, the model trained from scratch does not benefit from longer videos as much as self-supervised or supervised variants. In particular, on HMDB51 its performance improves by about 1-2% with 32 frames, but actually decreases with 64. We attribute this to the fact that using longer clips lowers the diversity of the training set, which is crucial for optimizing an untrained representation. These results further demonstrate the importance of model pretraining for video understanding.

Effect of the number of videos

So far, we have reported all the results using 235 000 videos in the training set of Kinetics-400 [START_REF] Carreira | Quo vadis, action recognition? a new model and the kinetics dataset[END_REF]. We now train the model with our final objective (Video LA with IDT prior) using a varying number of videos to study the effect of the dataset size on the quality of the learned representations. In particular, we subsample the training set to 185 000 and 135 000 examples at random to see whether smaller datasets can be used for representation learning. We also add the videos from the larger Kinetics-600 dataset to see if our method scales to larger video collections. We use the 3D ResNet18 architecture with 16-frames long clips and input resolution of 112 × 112 in all experiments, and report results on the first split of UCF101 and HMDB51 in Figure 4. Firstly, we observe that useful representations can be learned with as few 135 000 videos. However, using more data results in improved performance on both datasets. On UCF101 the improvements are mostly linear, but accuracy drops somewhat for the largest training set (370 000 videos). We attribute this to the randomness in training and hypothesize that further improvements can be achieved with more data. On HMDB51 accuracy seems to plateau after 235 000 videos, but improves with 370 000. We will use the model trained on the largest available dataset for comparison to the state-of-the-art in the next section.

Comparison to the state-of-the-art

Finally, we compare our approach (Video LA with IDT prior) to the state-of-theart unsupervised video representations in Table 4. As noted in Section 4.2, to Table 4. Comparison to the state-of-the-art using accuracy and fraction of the fullysupervised performance on UCF101 and HMDB51, averaged over 3 splits. 'Ours': Video LA with IDT prior. DPC uses a non-standard version of 3D ResNet, and does not report fully-supervised performance for it. Our method shows top accuracy among the models using the same network architecture. When normalized for the architecture differences, it outperforms all the approaches.

Method

Network Frame size #Frames UCF101 HMDB51 Acc. % sup. Acc. % sup. PMAS [START_REF] Wang | Self-supervised spatio-temporal representation learning for videos by predicting motion and appearance statistics[END_REF] C3D fairly compare results achieved by methods with different network architectures, we use the fraction of fully supervised performance as an additional metric, whenever this information is available. To make the table size manageable, we only report approaches that use 3D ConvNets pretrained on Kinetics. These, however, cover all the top performing methods in the literature. Firstly, we observe that our principled approach is indeed a lot more effective that manually designed objectives used in PMAS [START_REF] Wang | Self-supervised spatio-temporal representation learning for videos by predicting motion and appearance statistics[END_REF], or 3D-Puzzle [START_REF] Jing | Self-supervised spatiotemporal feature learning via video rotation prediction[END_REF], confirming the effectiveness of clustering-based training. The improvements are especially large on HMDB, which is, as we have shown previously, can be attributed to the IDT prior helping to better model the temporal information. Our approach also outperforms DPC [START_REF] Han | Video representation learning by dense predictive coding[END_REF], when the network depth is the same for both methods, even though DPC uses much longer sequences (40 frames with a stride 2, so the effective length is 120). Notably, on HMDB our approach even outperforms a variant of DPC with a deeper network, and bigger frame size by a large margin. When trained with longer temporal sequences, our method also outperforms the deeper variant of DPC on UCF by 7.3%. On HMDB we are 14.7% ahead.

The very recent approach of Sun et al. [START_REF] Sun | Contrastive bidirectional transformer for temporal representation learning[END_REF] ('CBT' in the table), reports high accuracy on both datasets. However, we show that this is due to the authors of [START_REF] Sun | Contrastive bidirectional transformer for temporal representation learning[END_REF] using a much deeper network than other methods in the literature. In terms of the fraction of fully-supervised performance, the 16-frame variant of our method outperforms CBT by 4.4% on UCF and by 15.0% on HMDB. Moreover, the 64-frame variant also outperforms CBT in raw accuracy on both datasets.

Finally, we report the performance of Fisher vector encoded IDT descriptors ('IDT' in the table, the numbers are taken from [START_REF] Simonyan | Two-stream convolutional networks for action recognition in videos[END_REF]). Please note that these descriptors are computed on the full length of the video, using the original resolution. Despite this, our 64 frame model comes close to the IDT performance on both datasets. Training a deeper variant of this model with a larger input resolution can close the remaining gap.

Conclusions

This paper introduced a novel approach for unsupervised video representation learning. Our method transfers the heuristic-based IDT descriptors, that are effective at capturing motion information, to 3D ConvNets via non-parametric clustering, using an unlabeled collection of videos. We quantitatively evaluated the learned representations on UCF101 and HMDB51 action recognition benchmarks, and demonstrated that they outperform prior work. We also qualitatively analyzed the discovered video clusters, showing that they successfully capture video dynamics, in addition to appearance. This analysis highlighted that the clusters do not perfectly match with the human-defined action classes, partially explaining the remaining gap to the fully-supervised performance. values of these hyper-parameters improves the performance on UCF101, but hurts on HMDB51. Decreasing these values, in contrast, hurts the performance on both datasets. Overall, the values we used in the main paper strike a good balance for the two benchmarks.

C Effect of the network depth and resolution

In this section, we evaluate how the performance of our approach changes with the network depth and input resolution. To this end, we first independently increase the resolution to 224 × 224, and network depth to 34, compared to 112 × 112 and 18 used in the rest of the paper, and then report a combined variant (3D ResNet34 with 224 × 224 inputs) in Table 7. All the models are learned on the training set of Kinetics-400 with 16-frame long clips, using our final objective (Video LA with IDT prior), and tuned on UCF101 and HMDB51. Firstly, we observe that increasing the input resolution indeed results in a significant performance improvement on UCF101, whereas on HMDB51 accuracy remains almost unchanged. This is in line with to our intuition that appearance information is more important for UCF101. Curiously, increasing the network depth while keeping the original input resolution decreases the performance on HMDB51, while providing a modest improvement on UCF101. We hypothesize that the model capacity is limited by the small resolution. Indeed, the final variant, which combines larger inputs with a deeper network, shows significant improvements over the baseline on both UCF101 and HMDB51. Even higher accuracy could be obtained by training this configuration with longer clips.

Fig. 2 .

 2 Fig.2. Visualization of the clusters discovered by the Video LA objective without IDT prior. This variant groups videos in the space of a 3D ConvNet. As a results, the clusters are primarily defined by the appearance, grouping swimming pools, grass fields, vehicles, and backyards. The activity happening in the videos does not seem to play a significant role.

Fig. 3 .

 3 Fig. 3. Visualization of the clusters discovered by variant of Video LA objective that uses IDT prior. In contrast to the examples above, the videos are mainly grouped by motion properties, such as forward-backward hand motion, person rotation, fast person motion, and 'riding' action.

Fig. 4 .

 4 Fig. 4. Varying the number of Kinetics videos when training a 3D ConvNet with the 'Video LA with IDT prior' objective. Using more data for unsupervised pretraining results in better representations, as evident form transfer learning results on the first split of UCF101 and HMDB51 (reported using classification accuracy).

Table 1 .

 1 Comparison between variants of unsupervised learning objective using classification accuracy and fraction of fully supervised performance on the fist split of UCF101 and HMDB51. All models use a 3D ResNet18 backbone, and take 16 frames with resolution of 112 × 112 as input. Video LA with IDT prior consistently outperforms other objectives, with improvements on HMDB51 being especially significant.

	Method	UCF101	HMDB51
		Accuracy % sup. Accuracy % sup.
	Scratch [14]	42.4	50.2	17.1	30.3
	Video IR	70.0	82.9	39.9	70.7
	Video LA	71.4	84.6	41.7	73.9
	Video LA + IDT prior 72.8	86.3	44.0	78.0
	Supervised [14]	84.4	100	56.4	100

Table 2 .

 2 Comparison between variants of unsupervised learning objective on the first split of UCF101 in a few-shot regime, using classification accuracy. The networks are fully frozen, and a linear classifier is learned, gradually decreasing the amount of training data. The gap between unsupervised and supervised representations increases, but our full method ('Video LA + IDT') still outperforms other variants across the board.

	Method	1-shot 5-shot 10-shot 20-shot All
	Scratch	1.7	7.5	10.6	17.2	38.2
	Video IR	13.4	27.7	35.2	42.4	56.5
	Video LA	15.6	30.6	36.4	44.2	58.6
	Video LA + IDT prior 17.8	31.5	38.4	45.5	58.8
	Supervised	46.4	62.0	67.7	73.3	81.8

Table 7 .

 7 Evaluation of the effect of the network depth and input resolution on the first split of UCF101 and HMDB51 using classification accuracy.

	Network Frame resolution UCF101 HMDB51
	3D ResNet18	112 × 112	72.8	44.0
	3D ResNet18	224 × 224	74.6	43.8
	3D ResNet34	112 × 112	73.6	42.2
	3D ResNet34	224 × 224	77.1	45.8

Acknowledgements: We thank Piotr Koniusz and Lei Wang for sharing their implementation of Fisher vector encoding. This work was supported in part by the Inria associate team GAYA, and by the Intelligence Advanced Research Projects Activity (IARPA) via Department of Interior/Interior Business Center (DOI/IBC) contract number D17PC00345. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes not withstanding any copyright annotation theron. Disclaimer: The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied of IARPA, DOI/IBC or the U.S. Government.

Appendix

In this appendix, we provided additional analysis of our method, that was not included in the main manuscript due to space limitations. We begin with providing some additional ablations of our approach in Section A. We then study the effect of the hyper-parameters of LA objective function on the final performance in Section B. Finally, we evaluate the impact of the network depth and input resolution in Section C.

A Additional ablations

In the main paper, we capitalized on the version of IDT which uses human detections to suppress optical flow in background regions. To validate the importance of this component, we have recomputed IDTs without human detections, and report the results in Table 5 (denoted as Ours\Det). Removing this step from the IDT pipeline indeed decreases the perforamcne of our approach both on UCF101 and HMDB51, confirming the observations in [START_REF] Wang | Action recognition with improved trajectories[END_REF] that suppressing background motion improves the descriptors' quality. Next, we evaluate the final tuning step in our approach. Recall that after training the network with the clusters obtained in the IDT space, we construct a joint space of IDT and 3D ConvNet representations, and further tune the network in this space using the iterative Local Aggregation objective. A variant without this tuning step, in reported in Table 5 as Ours \Tune, indeed achieves lower performance (coincidentally, it is exactly the same as the performance of the variant without person detections). This demonstrates that, although IDT descriptors already capture appearance information, using the more expressive 3D ConvNet representation provides further benefits.

B Effect of the objective parameters


Hugo Kuentz
email: hugo.kuentz@cea.fr

Baba Wagué

Nicolas Vaxelaire

Valérie Demange

Christophe Poulain

Maryline Guilloux-Viry

Gwenael Le Rhun

Effect of electrically induced cracks on the properties of PZT thin film capacitors

We present a study of the effect of electrically induced cracks on both the ferroelectric and piezoelectric properties of Pt/PbZr0.52Ti0.48O3 (PZT)/Pt capacitors, and correlations with the domain structures of the PZT films. Above a threshold bipolar electric field, cracks appear inside the PZT layer thickness leading to an increase of the ferroelectric polarization (+ 50% for the remnant polarization, from 16 µC/cm 2 to 25 µC/cm 2 ) and the longitudinal piezoelectric coefficient d33,f (from ~150 pm/V to ~220 pm/V). The use of X-ray diffraction during in situ biasing provides direct evidence for a modification of the PZT crystalline structure as well as the a/c domain configuration. After cracking, the fraction of c-domains is strongly increased, thus contributing to higher polarization and larger strain in the out-of-plane direction.

The lead zirconate titanate perovskite, Pb(Zr0.52Ti0.48)O3 (PZT) is a ferroelectric material showing piezoelectric effects among the largest ones. This composition, which is in the vicinity of the morphotropic phase boundaries (MPB) region 1 , is used in many fields of microelectromechanical system (MEMS) actuator devices: optics (varifocal lens, micromirror), acoustics (micro-speaker), ultrasonics (Piezoelectric Micromachined Ultrasound Transducers: PMUT), fluidics (inkjet head, micro-pump), haptics (tactile feedback) and many others 2,3 . In actuator devices, high electrical fields are applied on the piezoelectric film capacitor to generate large mechanical displacements, which may lead to different failure mechanisms. The reliability of the integrated materials is of major importance for the devices. Several studies have been conducted to investigate reliability concerns in case of PZT films. Electrical failures such as electrical breakdown or fatigue in PZT thin films have already been well explored [4][5][6] . Mechanical failures through formation of cracks under either applied mechanical or electrical stress on the film, sometimes associated with fatigue, have been also reported [7][8][9][10][11] . A few papers report electrically induced cracks, but without detailing the properties of PZT after cracking 12- 15 . Understanding and mastering of the reliability of the material is key for its viability in applications.

We have studied the effect of cracks induced by the application of electric field on the structural and electrical properties of PZT capacitors. Highly (100)-oriented 2 µm thick PZT films were grown by chemical solution deposition on Pt (111) 100 nm/TiO2 20 nm/SiO2 500 nm/Si 725 µm substrate (200 mm wafer). PZT sol-gel solution with Zr/Ti ratio of 52/48 was provided by ACCEPTED MANUSCRIPT / CLEAN COPY Mitsubishi Materials Corporation. More details about the deposition of PZT thin films can be found elsewhere [START_REF] Kovacova | [END_REF] . Platinum layer with 100 nm thickness was sputter deposited and patterned as top electrodes to fabricate capacitors.

The cracking of the 2 µm-thick PZT capacitors was observed after applying a threshold electrical voltage of 70 V (bipolar signal) or 150 V (unipolar signal) on several similar samples. The density of cracks expands with the amplitude and the iteration of the signal. Despite the cracks, capacitors remain functional until electrical breakdown (typically around 140 V for a bipolar signal). It is worth mentioning that the observed phenomenon is irreversible. Here we present the properties of capacitors subjected to 4 periods of the applied bipolar signals (±80 V, 200 Hz, triangular waveform). In order to evaluate the impact of the cracks on the properties, various measurements have been performed before and after the cracking of the capacitors. A scanning electron microscope (SEM) from HITACHI (S-5000) was used to observe the crosssections of PZT capacitor stacks. Symmetric coplanar diffraction in Bragg-Brentano geometry (using an Empyrean from PANalytical® 2 circle goniometer) was employed to evaluate structural and crystallographic properties. In situ biasing XRD characterization was performed in the same geometry by applying ±20 V (step of 0.5 V) during the measurement using similar protocol as described in reference [ 17 ]. The X-ray energy was 8.048 keV (i.e. X-ray radiation with a copper cathode tube) and the beam footprint was chosen to be 1x5 mm² in our diffraction geometry. Two micromanipulators adapted to our X-ray goniometer equipped with Tungsten needles (radius of 25 µm) were used for the electrical biasing. Due to the beam footprint, all the XRD measurements were performed on bigger capacitors (2x8 mm²) after checking that we obtain similar results and observations between big (16 mm²) and small (1 mm²) capacitors. According to this diffraction geometry (Bragg-Brentano), peak evolution in the direction of the applied electric field (out-of-plane) is evaluated, which also corresponds to the major (100) texture direction. Ferroelectric and dielectric measurements were performed at room temperature with a TF Analyzer 2000 Measurement System from aixACCT. Typical polarization-voltage (P-V) hysteresis loops were recorded using a bipolar 200 Hz triangular waveform signal at a maximum voltage amplitude of 30V, which correspond to an electrical field of ±150 kV/cm. Butterfly curves of the capacitance and the associated dielectric losses were obtained by sweeping DC voltage (-20 V to +20 V) while applying an AC exciting signal (1 kHz, 150 mV). The longitudinal piezoelectric coefficient d33,f was measured with a double beam laser interferometer (DBLI) from aixACCT 18 with a 300 Hz triangular waveform signal at a maximum voltage amplitude of 25 V.

Optical microscope pictures of PZT capacitor surfaces before and after the appearance of cracks are shown on Fig. 1a and1b, respectively. Cracks can be easily observed through the platinum top electrode on Fig. 1b, while they are absent on Fig. 1a. It is noteworthy that the cracking appears homogenous throughout the entire capacitor surface, meaning that Fig. 1b is representative of the observed phenomenon. More interesting, SEM observations demonstrate that the cracks are located exclusively inside the PZT film and do not propagate through the electrodes (Fig. 1c). A collapse of the upper platinum layer at the location of the cracks allows observing the phenomenon from above (Fig. 1d). This plastic deformation of the top electrode has already been reported in the literature 8 . As a result, it is important to notice that the top and bottom electrodes are continuous despite the collapse. That is why the capacitors remain functional despite the cracking of the PZT film. The ferroelectric, dielectric and piezoelectric properties were measured before and after the apparition of cracks on 1mm² capacitors (Fig. 2). Once cracks appeared, positive maximum and remnant polarization increase up to 39 µC/cm 2 and 25 µC/cm 2 compared to the initial values of 33 µC/cm 2 and 16 µC/cm 2 , respectively, as shown in Fig. 2a. It corresponds to an increase of approximately 15% for the maximum polarization and more than 50% for the remnant polarization. The associated polarization current is plotted in Fig. 2b. The switching current corresponds to the visible peak under ±10 V, the latter has more than doubled for both positive and negative voltage, indicating greater switching after the apparition of cracks. This large change of polarization values was measured on several capacitors for a given wafer, meaning that this phenomenon is reproducible. As an example, for 5 capacitors investigated on a given wafer under exact same experimental conditions, we measured for the positive remnant polarisation on average an increase of 54% with a standard error of 2%. In the same time, the relative permittivity (r) decreases by more than 50 on average in the ±20 V range, while dielectric losses tan(δ) increases from ~6% to ~9% (Fig. 2c). The dielectric losses could be affected by the formation of new leakage paths through the cracks. Indeed, we observe an increase of the leakage current density from 16 nA/cm² to 165 nA/cm² after cracking (not ACCEPTED MANUSCRIPT / CLEAN COPY shown). As visible in Fig. 2d, the maximum displacement and thus the piezoelectric coefficient d33,f increases after cracking. The values of d33,f+ and d33,f-change from 146 pm/V to 213 pm/V and from -148 pm/V to -231 pm/V, respectively. A partial declamped of the PZT film on the substrate could occurred with the cracking and lead to an increase of the measured d33,f 19,20 . However, no apparent delamination was observed in our case. Such an important evolution of the electrical and piezoelectric behavior cannot be explained only by a wake-up of some dipoles, but implies significant modifications of the PZT properties such as a variation of the structure and/or a change in the extrinsic properties such as the stress in the thin film. In this study, cracks are only induced by the application of an electric field above a given threshold. By increasing significantly the electric field (E), the stress within piezoelectric film will increase as well. This is the electrical induced stress (σe) with a piezoelectric part (e31,f) and an electrostrictive part (M), which can be calculated as follows 8 :

𝜎 = 𝑒 , 𝐸 + 𝑀𝑐𝐸 (1)
where c is the stiffness of the material. This stress adds up with residual stress, which is mainly due to the mismatch of thermal expansion coefficient between PZT and the substrate. On silicon substrate, the residual stress (tensile) in sol-gel PZT can be estimated between 100 and 200 MPa [START_REF] Defay | IEEE 22nd Int. Conf. Micro Electro Mech. Syst[END_REF][START_REF] Zhao | [END_REF][START_REF] Mazzalai | Jt. IEEE Int. Symp. Appl. Ferroelectr. Int. Workshop Acoust. Transduct. Mater. Devices Workshop Piezoresponse Force Microsc[END_REF] . Upon a certain level of stress, cracks will be initiated within the thin film. We can estimate in our case a critical stress between 0.4 and 1 GPa [START_REF] Coleman | Jt. Conf. IEEE Int. Freq. Control Symp. Int. Symp. Appl. Ferroelectr. IFCS-ISAF[END_REF][START_REF] Mehta | [END_REF] . By combining this residual ACCEPTED MANUSCRIPT / CLEAN COPY stress and the induced electric stress (~350-400 MPa), we understand that we reach the critical stress, which explains the occurrence of cracks.

The residual tensile stress in the film directly impacts the domain distribution, which has a direct effect on the electric and piezoelectric responses [25][26][27][28][29][30] . Tensile stress favors in-plane domains, which lead to a lower polarization and a greater permittivity 27 . After cracking, the residual stress at least partially released [START_REF] Zhao | [END_REF] . As a consequence, the domain patterns may be changed, and thus the electric and piezoelectric properties. This assumption is in good agreement with our observations.

In order to confirm this hypothesis and to explore the structural change within the PZT, in situ biasing X-ray diffraction measurements have been performed within a single measurement session on two different capacitors close to each other on the sample, one uncracked and the other cracked. In Fig. 3, the diffraction pattern around the pseudo-cubic 200 PZT peak is shown for uncracked and cracked capacitors, with and without an applied electrical voltage during the measurement. At 0 V, we can see that after cracking, the peak shifts towards smaller angles from 44.575° to 44.373° and gets wider from 0.708° to 0.883° Full Width at Half Maximum (FWHM). At 20 V, the change is increased for the cracked capacitor. The peak position is 44.191° and the peak is a little bit sharper with a FWHM of 0.817°. Such changes may result from different reasons. A modification of the domain structure by ferroelastic switching can occur, as well as a phase transition since we know that there is a mix of tetragonal and rhombohedral phases for PZT at MPB. An accumulation of defects could also participate partially to a widening of the peaks. This is coherent to assume here since the cracks could lead to new defects, as can be seen with the increase of both the dielectric losses and the leakage current as mentioned above. However, we think that the main effects are the change in the phase and/or domain structure in relation with the variation of the properties shown in Fig. 2.

. In order to investigate this, profile refinement for each peak has been realized using pseudo-Voigt symmetrical functions (Fig. 4). For all samples, best refinement is obtained by ACCEPTED MANUSCRIPT / CLEAN COPY considering presence in the film of both rhombohedral and tetragonal phases that are located on either side of the morphotropic boundary at room temperature. The peaks are refined with three simulated sub-peaks, one is from the rhombohedral phase (024) noted "r-domain" and the other two are from the tetragonal phase ( 200) and (002) noted "a-domain" and "c-domain", corresponding respectively to the in-plane and out-of-plane polarization domain. In our configuration, increasing the c-domain will lead to an increase of polarization and a decrease of permittivity. We can see with the uncracked and cracked capacitors at 0 V, Fig. 4a and Fig. 4c respectively, that even with no applied voltage, there is an important difference in the proportion of the "c-domain". By taking the integrated intensities of the three sub-peaks, we can estimate the contribution of the out-of-plane polarization.

For the uncracked capacitor the contribution of the out-of-plane polarization domain is approximately 20%, whereas it is almost 70% for the cracked one. This indicates that there is indeed a relaxation of the film. It explains also that the remnant polarization is greater for the latter. As expected, applying a voltage raises the fraction of c-domain, up to more than 30% for the uncracked capacitor at 20 V (Fig. 4b). After cracking, we can observe an even greater amount of c-domain fraction, up to more than 75% (Fig. 4d). The domains are less constrained and can switch more easily, which also going along for a bigger saturated polarization for cracked capacitors as well as a higher strain in the out-of-plane direction as shown with the DBLI measurement (Fig. 2c). This increase of the (002) peak intensity is correlated with the diminution of the intensity of both the (200) and the (024) r-phase peaks. This means that, according to this profile refinement, an important increase of the out-of-plane domain occurred as a result of both ferroelastic domain switching from a-domain to c-domain and phase transition from rhombohedral to tetragonal phase. In this paper, we have found a correlation between the increase of both ferroelectric and piezoelectric properties resulting from electrically induced cracks with a clear modification of the PZT domain structure. Above a threshold bipolar electric field, an irreversible change of the crystalline structure occurs together with cracking of the film. A phase transition from rhombohedral towards tetragonal phase is observed. The volume fraction of c domain of the tetragonal phase is also strongly increased. The PZT capacitors are still functional with major change on the ferroelectric and piezoelectric properties. The remnant polarization and the longitudinal piezoelectric coefficient d33,f are increased, up to more than 50%. The evolution of the electrical characteristics is consistent with the structural change observed by XRD measurements. In a future work, similar experiments will be conducted on cantilever and membrane type actuators.

ACCEPTED MANUSCRIPT / CLEAN COPY

ACCEPTED MANUSCRIPT / CLEAN COPY

Fig. 1 :

 1 Fig. 1: Optical microscope pictures of PZT film covered by platinum (top view) a) before cracking and b) after cracking. The inset shows the observed area of the 1 mm² capacitors. SEM images of a crack in the PZT thin film through c) cross-sectional view and d) tilted view (40°).

Fig. 2 :

 2 Fig. 2: a) Polarization versus applied voltage loops, b) Polarization current versus applied voltage, c) permittivity and dielectric losses versus applied voltage curves and d) piezoelectric displacement versus applied voltage curves, all measured before and after cracking.

Fig. 3 :

 3 Fig. 3: XRD diagram θ-2θ profile evolution of PZT around peak (200) for cracked and uncracked capacitors with and without electrical tension applied (20V).

Fig. 4 :

 4 Fig. 4: XRD diagram with fitting of the PZT peak for the uncracked capacitor at a) 0 V and b) 20 V. and for the cracked capacitor at c) 0 V and d) 20 V.