Zafar Shahid
email: zafar.shahid@lirmm.fr

Marc Chaumont
email: marc.chaumont@lirmm.fr

Member, IEEE William Puech
email: william.puech@lirmm.fr

Editor M Barni

Fast Protection of H.264/AVC by Selective Encryption of CAVLC and CABAC for I and P Frames

Keywords: AES algorithm, CABAC, CAVLC, selective en-27 cryption, stream cipher, video security. 28 Encryption Algorithm Compliant Transcoding Increase Independent Scrambling for privacy protection [9] Yes No Transform Yes Yes Pseudo random sign inversion NAL unit encryption [14] No No Bitstream Pseudo random pixel permutations I frame encryption [2]

This paper presents a novel method for the protec-1 tion of bitstreams of state-of-the-art video codec H.264/AVC. The 2 problem of selective encryption (SE) is addressed along with the 3 compression in the entropy coding modules. H.264/AVC supports 4 two types of entropy coding modules. Context-adaptive variable 5 length coding (CAVLC) is supported in H.264/AVC baseline 6 profile and context-adaptive binary arithmetic coding (CABAC) 7 is supported in H.264/AVC main profile. SE is performed in both 8 types of entropy coding modules of this video codec. For this 9 purpose, in this paper the encryption step is done simultaneously 10 with the entropy coding CAVLC or CABAC. SE is performed by 11 using the advanced encryption standard (AES) algorithm with 12 the cipher feedback mode on a subset of codewords/binstrings. 13 For CAVLC, SE is performed on equal length codewords from 14 a specific variable length coding table. In case of CABAC, it is 15 done on equal length binstrings. In our scheme, entropy coding 16 module serves the purpose of encryption cipher without affecting 17 the coding efficiency of H.264/AVC by keeping exactly the same 18 bitrate, generating completely compliant bitstream and utilizing 19 negligible computational power. Owing to no escalation in bitrate, 20 our encryption algorithm is better suited for real-time multimedia 21 streaming over heterogeneous networks. It is perfect for playback 22 on handheld devices because of negligible increase in processing 23 power. Nine different benchmark video sequences containing 24 different combinations of motion, texture, and objects are used 25 for experimental evaluation of the proposed algorithm.

I. Introduction

Multimedia data requires either full encryption or selective encryption (SE) depending on the application requirements. For example, military and law enforcement applications require full encryption. Nevertheless, there is a large spectrum of applications that demands security on a lower level, as, e.g., that ensured by SE. SE encrypts part of the plaintext and has two main advantages. First, it reduces the computational requirements, since only a part of plaintext is encrypted [6]. Second, encrypted bitstream maintains the essential properties of the original bitstream [3]. SE just prevents abuse of the data. In the context of video, it refers to destroying the commercial value of video to a degree which prevents a pleasant viewing experience. SE schemes based on H.264/AVC have been already presented on context-adaptive variable length coding (CAVLC) [29] and context-adaptive binary arithmetic coding (CABAC) [30]. These two previous methods fulfill real-time constraints by keeping the same bitrate and by generating completely compliant bitstream. In this paper, we have enhanced the previous proposed approaches by encryption of more syntax elements for CAVLC and extending it for P frames. Here, we have also used advanced encryption standard (AES) [7] in the cipher feedback (CFB) mode which is a stream cipher algorithm. Security of the proposed schemes has also been analyzed in detail.

The rest of this paper is organized as follows. In Section II, overview of H.264/AVC and AES algorithm is presented. We explain the whole system architecture of the proposed methods in Section III. Section IV contains experimental evaluation and security analysis. In Section V, we present the concluding remarks about the proposed schemes.

II. Description of the H.264/AVC-Based Video Encryption System

A. Overview of H.264/AVC H.264/AVC (also known as MPEG4 Part 10) [1] is stateof-the-art video coding standard of ITU-T and ISO/IEC. H.264/AVC has some additional features and outperforms previous video coding standards including MPEG2 and MPEG4 Part II [35]. We review the basic working of CAVLC in Section II-A1 and of CABAC in Section II-A2. To adapt to the local statistical features of discrete cosine transform (DCT) coefficients, CAVLC uses seven fixed variable length coding (VLC) tables. For example, "2" will be coded as "010" using VLC1 table, while it will be coded as "1010" using VLC3 2) CABAC: CABAC is designed to better exploit the characteristics of NZs as compared to CAVLC, consumes more processing, and offers about 10% better compression than CAVLC on average [22]. Run-length coding has been replaced by significant map coding which specifies the position of NZs in the 4 × 4 block. Binary arithmetic coding (BAC) module of CABAC uses many context models to encode NZs and context model for a specific NZ depends on recently coded NZs.

CABAC consists of multiple stages as shown in Fig. 2(a).

First of all, binarization is done in which non-binary syntax elements are converted to binary form called binstrings which are more amenable to compression by BAC. Binary representation for a non-binary syntax element is done in such a way that it is close to minimum redundancy code. In CABAC, there are four basic code trees for binarization step, namely, the unary code, the truncated unary code, the kth order Exp-Golomb code (EGk), and the fixed length code as shown in Fig. 2(b).

For an unsigned integer value x ≥ 0, the unary code consists of x 1s plus a terminating 0 bit. The truncated unary code is only defined for x with 0 ≤ x ≤ s. For x < s, the code is given by the unary code, whereas for x = s the terminating "0" bit is neglected. EGk is constructed by a concatenation of a prefix and a suffix parts and is suitable for binarization of syntax elements that represent prediction residuals. For a given unsigned integer value x > 0, the prefix part of the EGk binstring consists of a unary code corresponding to the length l (x ) = log 2 ( x 2k + 1) . The EGk suffix part is computed as the binary representation of x + 2 k (1 -2 l(x) ) using k + l(x) significant bits. Consequently for EGk binarization, the code length is 2l(x) + k + 1. When k = 0, 2l(x) + k + 1 = 2l(x) + 1. The AES algorithm consists of a set of processing steps 143 repeated for a number of iterations called rounds [7]. The 144 number of rounds depends on the size of the key and the 145 size of the data block. The number of rounds is nine, e.g., 146 if both the block and the key are 128 bits long. Given a 147 sequence {X 1 , X 2 , ..., X n } of bit plaintext blocks, each X i is 148 encrypted with the same secret key k producing the ciphertext 149 blocks {Y 1 , Y 2 , ..., Y n }. To encipher a data block X i in AES, 150 you first perform an AddRoundKey step by XORing a subkey 151 with the block. The incoming data and the key are added 152 together in the first AddRoundKey step. Afterward, it follows 153 the round operation. Each regular round operation involves 154 four steps which are SubBytes, ShiftRows, MixColumns, and 155 AddRoundKey. Before producing the final ciphered data Y i , 156 the AES performs an extra final routine that is composed of 157 SubBytes, ShiftRows, and AddRoundKey steps.

158

The AES algorithm can support several cipher modes: 159 electronic code book (ECB), cipher block chaining, output 160 feedback (OFB), CFB, and counter (CTR) [31]. The ECB 161 mode is actually the basic AES algorithm. In CFB mode, as 162 shown in Fig. 3, the keystream element Z i is generated and 163 the ciphertext block Y i is produced as follows:

I E E E P r o o f

consequently the computational complexity is increased.

192

The AES [7] has been used for SE of image and video in 193 the literature. The AES was applied on the Haar discrete 194 wavelet transform compressed images in [23]. The encryption 195 of color images in the wavelet transform has been addressed 196 in [21]. In this approach, the encryption is performed on the 197 resulting wavelet code bits. In [25], SE was performed on color

202

AES encryption standard has been addressed in [37].

203

Combining PE and image/video compression using the set 204 partitioning in hierarchical trees was used in [6]. Nevertheless, 205 this approach requires a significant computational complexity.

206

A method that does not require significant processing time and which operates directly on the bit planes of the image was proposed in [19]. The robustness of partially encrypted videos to attacks which exploit the information from non-encrypted bits together with the availability of side information was studied in [27]. Fisch et al. [10] proposed a scalable encryption method for a DCT-coded visual data wherein the data are organized in a scalable bitstream form. These bitstreams are constructed with the DC and some AC coefficients of each block which are then arranged in layers according to their visual importance, and PE process is applied over these layers.

For video, there are several SE techniques for different video codecs presented in the literature. SE of MPEG4 video standard was studied in [34] wherein data encryption standard was used to encrypt fixed length and variable length codes. In this approach, the encrypted bitstream is completely compliant with MPEG4 bitstream format but it increases the bitrate. A tradeoff has to be made among complexity, security, and the bit overhead. In [38], SE of MPEG4 video standard is proposed by doing frequency domain selective scrambling, DCT block shuffling, and rotation. This scheme is very easy to perform but its limitation is its bitrate overhead. SE of region of interest (ROI) of MPEG4 video has been presented in [9]. It performs SE by pseudo randomly inverting sign of DCT coefficients in ROI. SE of H.264/AVC has been studied in [15] wherein encryption has been carried out in some fields like intra-prediction mode, residual data, inter-prediction mode, and motion vectors. A scheme for commutative encryption and watermarking of H.264/AVC is presented in [16]. Here, SE of some macroblock (MB) header fields is combined with watermarking of magnitude of DCT coefficients. This scheme presents a watermarking solution in encrypted domain without exposing video content. The limitation of techniques proposed in [15] and [16] is that they are not format compliant. Encryption for H.264/AVC has been discussed in [5] wherein they do permutations of the pixels of MBs which are in ROI. The drawback of this scheme is that bitrate increases as the size of the ROI increases. This is due to change in the statistics of ROI as it is no more a slow varying region which is the basic assumption for video signals. SE of H.264/AVC at network abstraction layer (NAL) has been proposed in [14]. Important NAL units, namely, instantaneous decoding refresh picture, sequence parameter set, and picture parameter set are encrypted with a stream cipher. The limitation of this scheme is that it is not format compliant and cannot be parsed even at frame level. SE of H.264/AVC using AES has been proposed in [2]. In this scheme, encryption of I frame is performed, since P and B frame are not significant without I frames. This scheme is not format compliant.

The use of general entropy coder as encryption cipher using statistical models has been studied in the literature in [36]. It encrypts by using different Huffman tables for different input symbols. The tables, as well as the order in which they are used, are kept secret. This technique is vulnerable to known plaintext attacks as explained in [12]. Key-based interval splitting of arithmetic coding (KSAC) has used an approach [13] wherein intervals are partitioned in each iteration of arithmetic coding. Secret key is used to decide how the interval will be partitioned. Number of subintervals in which an interval is divided should be kept small as it increases the bitrate of bitstream. Randomized arithmetic coding [11] is aimed at arithmetic coding but instead of partitioning of intervals like in KSAC, secret key is used to scramble the order of intervals. The limitation of these entropy coding-based techniques is that encrypted bitstream is not format compliant.

I E E E P r o o f

Moreover, these techniques require lot of processing power.

In 2) The encrypted codewords/binstrings must be valid so that they may be decoded by entropy decoder.

3) The decoded value of syntax element from encrypted codewords/binstrings must stay in the valid range for that syntax element. Any syntax element which is used for prediction of neighboring MBs should not be encrypted. Otherwise, the drift in the value of syntax element will keep on increasing and after a few iterations, value of syntax element will fall outside the valid range and bitstream will be no more decodable.

In each MB, header information is encoded first, which is followed by the encoding of MB data.

328

A. ES for SE-CAVLC

329

In CAVLC, five syntax elements are used to code levels 330 and runs as shown in Fig. 4. NZs are coded by three syntax 331 elements, namely, coeff token, signs of trailing ones, and 332 remaining nonzero levels. Zeros are coded by two syntax 333 elements, namely, total number of zeros and runs of zeros. 334 A single syntax element, namely, coeff token is used to code 335 total NZs and number of trailing ones. It is followed by coding 336 of signs of trailing ones (T1s). Remaining NZs are then coded 337 using seven VLC look-up tables either by regular mode or by 338 escape mode as explained in Section II-A1. They are mapped 339 to some code from a specific VLC look-up table.

340

To keep the bitstream compliant, we cannot encrypt co-341 eff token, total number of zeros, and runs of zeros. Two 342 syntax elements fulfill the above-mentioned conditions for 343 encryptions. First is signs of trailing ones. Second is sign and 344 magnitude of remaining NZs, both in regular and escape mode.

I E E E P r o o f

B. ES for SE-CABAC

The main difference between SE-CAVLC and SE-CABAC is that in SE-CABAC, SE is not performed on CABAC bitstream. Rather it is performed on binstrings which are input to BAC as shown in Fig. 5. Among all the four binarization techniques, the unary and truncated unary codes n+1) where n is the maximum possible value by suffix bits of EG0, i.e., when all the bits in suffix are 1.

2 log 2 (

C. SE of NZs in the Entropy Coding Stage of H.264/AVC

Let us consider

Y i = X i ⊕ E k (Y i-1
) as the notation for the encryption of a n bit block X i , using the secret key k with the AES cipher in CFB mode as given by (1), and performed as described in the scheme from Fig. 3. We have chosen to use this mode in order to keep the original compression rate. Indeed, with the CFB mode for each block, the size of the encrypted data Y i can be exactly the same one as the size of the plaintext X i . In this mode, the code from 397 the previously encrypted block is used to encrypt the current 398 one as shown in Fig. 3. The three stages of the proposed 399 algorithm are the construction of the plaintext X i , described 400 in Section III-C1, the encryption of X i to create Y i which is 401 provided in Section III-C2, and the substitution of the original 402 codeword/binstring with the encrypted information, which is 403 explained in Section III-C3. The overview of the proposed SE 404 method is provided in Fig. 7.

405 1) Construction of Plaintext: As slices are independent 406 coding units, SE should be performed on them independently. 407 In case of SE-CAVLC, the plaintext is created by copying 408 the encrypt-able bits from CAVLC bitstream to the vector X i 409 until either X i is completely filled or slice-boundary comes 410 as shown in Fig. 7(a). Let C, the length of the vector X i , is 411 128. In case of SE-CABAC, we perform SE before BAC as 412 shown in Fig. 7(b). In that case, we transform the non-binary 413 syntax elements to binstrings through process of binarization 414 and at the same time we fill the X i with encrypted bits until 415 either the vector X i is completely filled or the slice boundary 416 comes. The binarization of many syntax elements at the same 417 time also makes the CABAC coding faster and increases its 418 throughput [39]. Let L(X i ) be the length up to which vector 419 X i is filled. In case of slice boundary, if L(X i ) < C, we apply 420 a padding function p(j) = 0, where j ∈ {L(X i ) + 1, . . . , C}, 421 to fill in the vector X i with zeros up to C bits. Historically, 422 padding was used to increase the security of the encryption, 423 but in here it is used for rather technical reasons [28]. In the encryption step with AES in the CFB mode, the previous 426 encrypted block Y i-1 is used as the input of the AES algorithm 427 in order to create Z i . Then, the current plaintext X i is XORed 428 with Z i in order to generate the encrypted text Y i as given 429 by (1). For the initialization, the initialization vector (IV) 430 is created from the secret key k according to the following the experimental results, nine benchmark video sequences have been used for the analysis in QCIF format. Each of them represents different combinations of motion (fast/slow, pan/zoom/rotation), color (bright/dull), contrast (high/low), and objects (vehicle, buildings, people). The video sequences Bus, City, and Foreman contain camera motion while Football and Soccer contain camera panning and zooming along with object motion and texture in background. The video sequences Harbour and Ice contain high luminance images with smooth motion. Mobile sequence contains a complex still background and foreground motion.

In Section IV-A, we present an analysis of joint SE and H.264/AVC compression while in Section IV-B, we compare PSNR and quality when applying SE only on I frames and on I+P frames. In Section IV-C, security analysis, showing the efficiency of the proposed method, is developed.

A. Analysis of Joint SE and H.264/AVC Compression

We have applied simultaneously our SE and H.264/AVC compression as described in Section III, on all the benchmark video sequences. SE-CAVLC and SE-CABAC impart some characteristics to the bitstream. In spatial domain, SE video gets flat regions and change in pixel values mostly occur on MB boundaries. In temporal domain, luma and chroma values rise up to maximum limit and then come back to minimum values. This cycle keeps on repeating. Owing to this phenomenon, the pixel values change drastically in temporal domain. Lot of transitions are observed in values of color and brightness.

In a first set of experiments, we have analyzed the available ES in H.264/AVC bitstreams for both of SE-CAVLC and SE-CABAC. ES is defined as percentage of total bitstream size. MBs that contain many details and texture will have lot of NZs and, consequently, will be strongly encrypted. On the contrary, the homogeneous MBs, i.e., blocks that contain series of identical pixels, are less ciphered because they contain a lot of null coefficients which are represented by runs in CAVLC and by significant map in CABAC. In Table I, we provide ES for SE-CAVLC and SE-CABAC for different benchmark video sequences for quantization parameter (QP) value 18. While in Table II, ES for various QP values is shown for Foreman video sequence. Here the average number of bits available for SE per MB are also provided. One can note that ES is inversely proportional to QP value. When QP value is higher and implicitly the video compression is higher, we are able to encrypt fewer bits in the compressed frame. This is due to the fact that H.264/AVC has lesser number of NZs at higher QP values. From both these tables, it is evident that more ES is available for SE-CAVLC as compared to SE-CABAC. But ES is more affected by change in QP values for SE-CAVLC as compared to SE-CABAC. For example, for Foreman video sequence, ES varies from 28.55% to 6.70% for SE-CAVLC when QP varies from 12 to 42. For the same QP range, the change in ES for SE-CABAC is from 19.97% to 9.46% as shown in Table II. From Tables I and II, since PSNR of original H.264/AVC are very similar for both CAVLC and CABAC, in the rest of this section for the sake of comparison, we list only PSNR of CAVLC bitstreams. CABAC is less than SE-CAVLC owing to two reasons. First, ES of SE-CABAC is lesser than that of SE-CAVLC as shown in Tables I andII. Second, CABAC takes lot more processing power than CAVLC. So increase in processing power because of encryption will be lower in terms of percentage. Thus, SE-CAVLC and SE-CABAC is possible in real-time along with compression.

I E E E P r o o f

B. PSNR and Quality of SE-CAVLC and SE-CABAC for I Frames and I+P Frames

Peak signal to noise ratio (PSNR) is widely used objective video quality metric. However, it does not perfectly correlate with a perceived visual quality due to nonlinear behavior of human visual system. Structural similarity index (SSIM) [33] takes into account the structural distortion measurement, since human vision system is highly specialized in extracting structural information from the viewing field. SSIM has a better correlation to the subjective impression. SSIM ranges from -1 to 1. SSIM is 1 when both the images are the same. To present the visual protection of encrypted video sequences, PSNR and SSIM of I and I+P frames are presented.

1) I Frames:

To demonstrate the efficiency of our proposed scheme, we have compressed 100 I frames of each sequence at 30 f/s. Figs. 9 and 10 show the encrypted first frame of Foreman video sequence at different QP values for SE-CAVLC and SE-CABAC, respectively. In H.264/AVC, blocks on the top array are predicted only from left while blocks on left are always predicted from top. Owing to this prediction, a band having width of 8 pixels at top of video frames can be observed for both of SE-CAVLC and SE-CABAC while this band has width of 4 pixels on left of video frames as shown in Figs. 9 and 10. The average PSNR values of Foreman is given in Table IV over whole QP range. It is also compared with the PSNR obtained for the same video sequence without encryption. In Table IV, we present PSNR of original video only for CAVLC. PSNR for CABAC is very much similar as presented in Table I. One can note that whatever is the QP value, the quality of the encrypted video remains in the same lower range. If the probability of each gray level in the image is P(α i ) = 1 2 k , then the encryption of such image is robust against statistical attacks of first order, and thus H(X) = log 2 (2 k ) = k bits/pixel. In the image, the information redundancy r is defined as follows:

Table V compares

H(X) = - 2 k -1 i=0 P(α i )log 2 (P(α i )). (4)
r = k -H(X).
(5)

Similarly, the local standard deviation σ(j) for each pixel p(j) taking account of its neighbors to calculate the local mean p(j), is given as follows:

σ(j) = 1 m m i=1 (p(i) -p(j)) ( 6
)
where m is the size of the pixel block to calculate the local mean and standard deviation, and 0 ≤ j < M, if M is the image size. In case of full encryption, entropy H(X) is maximized with high values of local standard deviation. But in case of SE-CAVLC and SE-CABAC, the video frame is transformed to flat regions with blocking artifacts as depicted in Figs. 9 and 10. It is generally owing to variation in pixel values at MB boundaries. For all the benchmark sequences, the average information redundancy r for SE-CAVLC and SE-CABAC sequences is 0.94 and 0.55, respectively, while it is 1.11 for all the original sequences. Despite the fact that SE-CAVLC and SE-CABAC transform the video frames into flat region, the entropy of the encrypted video sequences from ( 4) is higher as compared to the original sequences. These flat regions are because of two reasons. First, flat regions are due to the fact that prediction is performed from edge pixels of neighboring MBs. Second, pixels have either very high value (bright video frame) or very low value (dark video frame) in SE video frame. This is owing to the fact that during reconstruction pixel value are clipped to 255 if they are greater than it and to 0 if they are below this lower range. So if many pixels have value beyond the upper or lower range, all of them will be clipped to the same value, thus creating a flat region which is either dark or bright. Based on this analysis, the statistical characteristics of SE-CAVLC and SE-CABAC bitstreams vary from full encryption systems.

From (6), we also analyzed the local standard deviation σ for each pixel while taking into account its neighbors. where y i is the adjacent pixel of x i . Since there is always three directions in images, i.e., horizontal, vertical, and diagonal, so we can define correlation direction between any two adjacent pixels as follows:

I E E E P r o o f

corr (x,y) = 1 n -1 n 0 ( x i -x i σ x )( y i -y i σ y ) ( 7
)
where n represents the total number of tuples (x i , y i ), x i and y i represent the local mean, and σ x and σ y represent the local standard deviation, respectively.

Owing to the flat regions in SE-CAVLC and SE-CABAC video sequences, the correlation values in these sequences will be higher as compared to original image which contain texture and edges. For all the benchmark sequences, the average horizontal correlation coefficient is 0.88 and 0.87 for the SE-CAVLC and SE-CABAC, respectively, while it is 0.80 for the original sequences. (1-bit different key) For the sake of comparative evaluation of our scheme, we 740 have compared it with six other recent techniques, which 741 include scrambling [9], NAL unit encryption [14], MB header 742 encryption [16], reversible ROI encryption [5], I frame en-743 cryption [2], and multiple Huffman table permutation [36]. 744 These techniques are different from each other in several 745 allow increase in bitrate [5], [9], [36] or use stream cipher 761 for the sake of same bitrate [14], [16], thus compromising on 762 the security of the system.

I E E E P r o o f

763

Format compliance is another important aspect for en- domain [5] and transform domain [9]. The proposed method has the advantage of being suitable for streaming over heterogeneous networks because of no change in bitrate. The experiments have shown that we can achieve the desired level of encryption, while maintaining the full bitstream compliance, under a minimal set of computational requirements. The presented security analysis confirmed a sufficient security level for multimedia applications in the context of SE. The proposed system can be extended for ROIspecific video protection [26] for video surveillance and can be applied to medical video transmission [24]. AQ:4= Please verify the volume no. in Ref. [5].

AQ:5= Please provide the issue no. or month in Ref. [5].

AQ:6= Please provide the technical report no. in Ref. [7].

AQ:7= Please provide the membership year of Puech.

AQ:8= Please verify the sense of the sentence "...he has been in the Area Chair...."

END OF ALL QUERIES

I E E E P r o o f

Fast The authors are with the Laboratory of Informatics, Robotics, and Microelectronics, University of Montpellier II, Montpellier 34392, France (e-mail: zafar.shahid@lirmm.fr; marc.chaumont@lirmm.fr; william.puech@lirmm.fr).

Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org.

Digital Object Identifier 10.1109/TCSVT.2011.2129090 of the multimedia property owners while ensuring the efficient rightful usage of such property.

Multimedia data requires either full encryption or selective encryption (SE) depending on the application requirements. For example, military and law enforcement applications require full encryption. Nevertheless, there is a large spectrum of applications that demands security on a lower level, as, e.g., that ensured by SE. SE encrypts part of the plaintext and has two main advantages. First, it reduces the computational requirements, since only a part of plaintext is encrypted [6]. Second, encrypted bitstream maintains the essential properties of the original bitstream [3]. SE just prevents abuse of the data. In the context of video, it refers to destroying the commercial value of video to a degree which prevents a pleasant viewing experience. SE schemes based on H.264/AVC have been already presented on context-adaptive variable length coding (CAVLC) [29] and context-adaptive binary arithmetic coding (CABAC) [30]. These two previous methods fulfill real-time constraints by keeping the same bitrate and by generating completely compliant bitstream. In this paper, we have enhanced the previous proposed approaches by encryption of more syntax elements for CAVLC and extending it for P frames. Here, we have also used advanced encryption standard (AES) [7] in the cipher feedback (CFB) mode which is a stream cipher algorithm. Security of the proposed schemes has also been analyzed in detail.

The rest of this paper is organized as follows. In Section II, overview of H.264/AVC and AES algorithm is presented. We explain the whole system architecture of the proposed methods in Section III. Section IV contains experimental evaluation and security analysis. In Section V, we present the concluding remarks about the proposed schemes.

II. Description of the H.264/AVC-Based Video Encryption System

A. Overview of H.264/AVC

H.264/AVC (also known as MPEG4 Part 10) [1] is stateof-the-art video coding standard of ITU-T and ISO/IEC. H.264/AVC has some additional features and outperforms previous video coding standards including MPEG2 and MPEG4 Part II [35]. We review the basic working of CAVLC in Section II-A1 and of CABAC in Section II-A2.

1051-8215/$26.00 c 2011 IEEE To adapt to the local statistical features of discrete cosine transform (DCT) coefficients, CAVLC uses seven fixed variable length coding (VLC) tables. For example, "2" will be coded as "010" using VLC1 table, while it will be coded as "1010" using VLC3 table. If magnitude of NZ lies within the range of that VLC table, it is coded by regular mode, otherwise escape mode is used. Adaptive nature is introduced by changing the table for the next NZ based on the magnitude of the current NZ as shown in Fig. 1. For the first NZ, VLC0 table is used unless there are more than ten NZs and less than three trailing ones, in which case it is coded with VLC1 table.

I E E E P r o o f

2) CABAC: CABAC is designed to better exploit the characteristics of NZs as compared to CAVLC, consumes more processing, and offers about 10% better compression than CAVLC on average [22]. Run-length coding has been replaced by significant map coding which specifies the position of NZs in the 4 × 4 block. Binary arithmetic coding (BAC) module of CABAC uses many context models to encode NZs and context model for a specific NZ depends on recently coded NZs.

CABAC consists of multiple stages as shown in Fig. 2(a). First of all, binarization is done in which non-binary syntax elements are converted to binary form called binstrings which are more amenable to compression by BAC. Binary representation for a non-binary syntax element is done in such a way that it is close to minimum redundancy code. In CABAC, there are four basic code trees for binarization step, namely, the unary code, the truncated unary code, the kth order Exp-Golomb code (EGk), and the fixed length code as shown in Fig. 2(b).

For an unsigned integer value x ≥ 0, the unary code consists of x 1s plus a terminating 0 bit. The truncated unary code is only defined for x with 0 ≤ x ≤ s. For x < s, the code is given by the unary code, whereas for x = s the terminating "0" bit is neglected. EGk is constructed by a concatenation of a prefix and a suffix parts and is suitable for binarization of syntax elements that represent prediction residuals. For a given unsigned integer value x > 0, the prefix part of the EGk binstring consists of a unary code corresponding to the length l (x ) = log 2 ( x 2k + 1) . The EGk suffix part is computed as the binary representation of x + 2 k (1 -2 l(x) ) using k + l(x) significant bits. Consequently for EGk binarization, the code The AES algorithm consists of a set of processing steps 143 repeated for a number of iterations called rounds [7]. The 144 number of rounds depends on the size of the key and the 145 size of the data block. The number of rounds is nine, e.g., 146 if both the block and the key are 128 bits long. Given a 147 sequence {X 1 , X 2 , ..., X n } of bit plaintext blocks, each X i is 148 encrypted with the same secret key k producing the ciphertext 149 blocks {Y 1 , Y 2 , ..., Y n }. To encipher a data block X i in AES, 150 you first perform an AddRoundKey step by XORing a subkey 151 with the block. The incoming data and the key are added 152 together in the first AddRoundKey step. Afterward, it follows 153 the round operation. Each regular round operation involves 154 four steps which are SubBytes, ShiftRows, MixColumns, and 155 AddRoundKey. Before producing the final ciphered data Y i , 156 the AES performs an extra final routine that is composed of 157 SubBytes, ShiftRows, and AddRoundKey steps.

length is 2l(x) + k + 1. When k = 0, 2l(x) + k + 1 = 2l(x) + 1.

158

The AES algorithm can support several cipher modes: 159 electronic code book (ECB), cipher block chaining, output 160 feedback (OFB), CFB, and counter (CTR) [31]. The ECB 161 mode is actually the basic AES algorithm. In CFB mode, as 162 shown in Fig. 3, the keystream element Z i is generated and 163 the ciphertext block Y i is produced as follows:

I E E E P r o o f

Z i = E k (Y i-1 ), for i ≥ 1 Y i = X i ⊕ Z i ( 1
)
where ⊕ is the XOR operator.

by such systems are high and previous approaches have not 178 entirely succeeded in tackling them [17].

179

In [32], Tang proposed a technique called zigzag permutation consequently the computational complexity is increased.

192

The AES [7] has been used for SE of image and video in 196 in [21]. In this approach, the encryption is performed on the 197 resulting wavelet code bits. In [25], SE was performed on color AES encryption standard has been addressed in [37].

203

Combining PE and image/video compression using the set 204 partitioning in hierarchical trees was used in [6]. Nevertheless,

205
this approach requires a significant computational complexity.

206

A method that does not require significant processing time and which operates directly on the bit planes of the image was proposed in [19]. The robustness of partially encrypted videos to attacks which exploit the information from non-encrypted bits together with the availability of side information was studied in [27]. Fisch et al. [10] proposed a scalable encryption method for a DCT-coded visual data wherein the data are organized in a scalable bitstream form. These bitstreams are constructed with the DC and some AC coefficients of each block which are then arranged in layers according to their visual importance, and PE process is applied over these layers.

For video, there are several SE techniques for different video codecs presented in the literature. SE of MPEG4 video standard was studied in [34] wherein data encryption standard was used to encrypt fixed length and variable length codes. In this approach, the encrypted bitstream is completely compliant with MPEG4 bitstream format but it increases the bitrate. A tradeoff has to be made among complexity, security, and the bit overhead. In [38], SE of MPEG4 video standard is proposed by doing frequency domain selective scrambling, DCT block shuffling, and rotation. This scheme is very easy to perform but its limitation is its bitrate overhead. SE of region of interest (ROI) of MPEG4 video has been presented in [9]. It performs SE by pseudo randomly inverting sign of DCT coefficients in ROI. SE of H.264/AVC has been studied in [15] wherein encryption has been carried out in some fields like intra-prediction mode, residual data, inter-prediction mode, and motion vectors. A scheme for commutative encryption and watermarking of H.264/AVC is presented in [16]. Here, SE of some macroblock (MB) header fields is combined with watermarking of magnitude of DCT coefficients. This scheme presents a watermarking solution in encrypted domain without exposing video content. The limitation of techniques proposed in [15] and [16] is that they are not format compliant. Encryption for H.264/AVC has been discussed in [5] wherein they do permutations of the pixels of MBs which are in ROI. The drawback of this scheme is that bitrate increases as the size of the ROI increases. This is due to change in the statistics of ROI as it is no more a slow varying region which is the basic assumption for video signals. SE of H.264/AVC at network abstraction layer (NAL) has been proposed in [14]. Important NAL units, namely, instantaneous decoding refresh picture, sequence parameter set, and picture parameter set are encrypted with a stream cipher. The limitation of this scheme is that it is not format compliant and cannot be parsed even at frame level. SE of H.264/AVC using AES has been proposed in [2]. In this scheme, encryption of I frame is performed, since P and B frame are not significant without I frames. This scheme is not format compliant.

The use of general entropy coder as encryption cipher using statistical models has been studied in the literature in [36]. It encrypts by using different Huffman tables for different input symbols. The tables, as well as the order in which they are used, are kept secret. This technique is vulnerable to known plaintext attacks as explained in [12]. Key-based interval splitting of arithmetic coding (KSAC) has used an approach [13] wherein intervals are partitioned in each iteration of arithmetic coding. Secret key is used to decide how the interval will be partitioned. Number of subintervals in which an interval is divided should be kept small as it increases the bitrate of bitstream. Randomized arithmetic coding [11] is aimed at arithmetic coding but instead of partitioning of intervals like in KSAC, secret key is used to scramble the order of intervals. The limitation of these entropy coding-based techniques is that encrypted bitstream is not format compliant.

I E E E P r o o f

Moreover, these techniques require lot of processing power.

In the context of DRM systems, our paper addresses the simultaneous SE and compression for state-of-the-art H.264/AVC. The encrypted bitstream is format compliant with absolutely no escalation in bitrate. Furthermore, it does not require lot of processing power for encryption and decryption. In Section III, we describe our proposed approaches to apply SE and H.264/AVC compression in video sequences, simultaneously.

III. Proposed SE Schemes

Our approach consists of SE during the entropy coding stage of H.264/AVC. In baseline profile, SE is performed in CAVLC entropy coding stage (SE-CAVLC). While in main profile, it is performed in CABAC entropy coding stage (SE-CABAC). In SE of video, encrypted bitstream compliance is a required feature for some direct operations such as displaying, time seeking, and browsing. Encrypted bitstream will be compliant and fulfills real-time constraints if the following three conditions are fulfilled.

1) To keep the bitrate of encrypted bitstream same as the original bitstream, encrypted codewords/binstrings must have the same size as the original codewords/binstrings.

2) The encrypted codewords/binstrings must be valid so that they may be decoded by entropy decoder.

3) The decoded value of syntax element from encrypted codewords/binstrings must stay in the valid range for that syntax element. Any syntax element which is used for prediction of neighboring MBs should not be encrypted. Otherwise, the drift in the value of syntax element will keep on increasing and after a few iterations, value of syntax element will fall outside the valid range and bitstream will be no more decodable.

In each MB, header information is encoded first, which is followed by the encoding of MB data. its sign are encoded. Similar to MB header, header of 4 × 4 321 block which includes coded block flag and significant map, 322 should not be encrypted for the sake of bitstream compliance. 323 Available encryption space (ES) which fulfills the above-324 mentioned conditions for SE-CAVLC and SE-CABAC is pre-325 sented in Sections III-A and III-B, respectively. Encryption 326 and decryption of the protected bitstream are presented in 327 Sections III-C and III-D, respectively.

328

A. ES for SE-CAVLC 329

In CAVLC, five syntax elements are used to code levels 330 and runs as shown in Fig. 4. NZs are coded by three syntax 331 elements, namely, coeff token, signs of trailing ones, and 332 remaining nonzero levels. Zeros are coded by two syntax 333 elements, namely, total number of zeros and runs of zeros. 334 A single syntax element, namely, coeff token is used to code 335 total NZs and number of trailing ones. It is followed by coding 336 of signs of trailing ones (T1s). Remaining NZs are then coded 337 using seven VLC look-up tables either by regular mode or by 338 escape mode as explained in Section II-A1. They are mapped 339 to some code from a specific VLC look-up table.

340

To keep the bitstream compliant, we cannot encrypt co-341 eff token, total number of zeros, and runs of zeros. Two 342 syntax elements fulfill the above-mentioned conditions for 343 encryptions. First is signs of trailing ones. Second is sign and 344 magnitude of remaining NZs, both in regular and escape mode. 345 For the sake of same bitrate, ES of SE-CAVLC consists of 346 only those NZs whose VLC codewords have the same length. 347 CAVLC uses multiple VLC tables with some threshold for 348 incrementing the table as given in (2). Since the threshold for 349 a specific table is highest possible value possible with that 350 codeword length (this is the case when all the suffix bits of 351 the codeword are 1), magnitude of encrypted NZ is such that 352 VLC table transition

I E E E P r o o f

B. ES for SE-CABAC

The main difference between SE-CAVLC and SE-CABAC is that in SE-CABAC, SE is not performed on CABAC n+1) where n is the maximum possible value by suffix bits of EG0, i.e., when all the bits in suffix are 1.

2 log 2 (

C. SE of NZs in the Entropy Coding Stage of H.264/AVC

Let us consider

Y i = X i ⊕ E k (Y i-1
) as the notation for the encryption of a n bit block X i , using the secret key k with the AES cipher in CFB mode as given by (1), and performed as described in the scheme from Fig. 3. We have chosen to use this mode in order to keep the original compression rate. Indeed, with the CFB mode for each block, the size of the encrypted data Y i can be exactly the same one as the size of the plaintext X i . In this mode, the code from 397 the previously encrypted block is used to encrypt the current 398 one as shown in Fig. 3. The three stages of the proposed 399 algorithm are the construction of the plaintext X i , described 400 in Section III-C1, the encryption of X i to create Y i which is 401 provided in Section III-C2, and the substitution of the original 402 codeword/binstring with the encrypted information, which is 403 explained in Section III-C3. The overview of the proposed SE 404 method is provided in Fig. 7.

405 1) Construction of Plaintext: As slices are independent 406 coding units, SE should be performed on them independently. 407 In case of SE-CAVLC, the plaintext is created by copying 408 the encrypt-able bits from CAVLC bitstream to the vector X i 409 until either X i is completely filled or slice-boundary comes 410 as shown in Fig. 7(a). Let C, the length of the vector X i , is 411 128. In case of SE-CABAC, we perform SE before BAC as 412 shown in Fig. 7(b). In that case, we transform the non-binary 413 syntax elements to binstrings through process of binarization 414 and at the same time we fill the X i with encrypted bits until 415 either the vector X i is completely filled or the slice boundary 416 comes. The binarization of many syntax elements at the same 417 time also makes the CABAC coding faster and increases its 418 throughput [39]. Let L(X i ) be the length up to which vector 419 X i is filled. In case of slice boundary, if L(X i ) < C, we apply 420 a padding function p(j) = 0, where j ∈ {L(X i ) + 1, . . . , C}, 421 to fill in the vector X i with zeros up to C bits. Historically, 422 padding was used to increase the security of the encryption, 423 but in here it is used for rather technical reasons [28]. In the encryption step with AES in the CFB mode, the previous 426 encrypted block Y i-1 is used as the input of the AES algorithm 427 in order to create Z i . Then, the current plaintext X i is XORed 428 with Z i in order to generate the encrypted text Y i as given 429 by (1). For the initialization, the initialization vector (IV) 430 In Section IV-A, we present an analysis of joint SE and H.264/AVC compression while in Section IV-B, we compare PSNR and quality when applying SE only on I frames and on I+P frames. In Section IV-C, security analysis, showing the efficiency of the proposed method, is developed.

A. Analysis of Joint SE and H.264/AVC Compression

We have applied simultaneously our SE and H.264/AVC compression as described in Section III, on all the benchmark video sequences. SE-CAVLC and SE-CABAC impart some characteristics to the bitstream. In spatial domain, SE video gets flat regions and change in pixel values mostly occur on MB boundaries. In temporal domain, luma and chroma values rise up to maximum limit and then come back to minimum values. This cycle keeps on repeating. Owing to this phenomenon, the pixel values change drastically in temporal domain. Lot of transitions are observed in values of color and brightness.

In a first set of experiments, we have analyzed the available ES in H.264/AVC bitstreams for both of SE-CAVLC and SE-CABAC. ES is defined as percentage of total bitstream size. MBs that contain many details and texture will have lot of NZs and, consequently, will be strongly encrypted. On the contrary, the homogeneous MBs, i.e., blocks that contain series of identical pixels, are less ciphered because they contain a lot of null coefficients which are represented by runs in CAVLC and by significant map in CABAC. In Table I, we provide ES for SE-CAVLC and SE-CABAC for different benchmark video sequences for quantization parameter (QP) value 18. While in Table II, ES for various QP values is shown for Foreman video sequence. Here the average number of bits available for SE per MB are also provided. One can note that ES is inversely proportional to QP value. When QP value is higher and implicitly the video compression is higher, we are able to encrypt fewer bits in the compressed frame. This is due to the fact that H.264/AVC has lesser number of NZs at higher QP values. From both these tables, it is evident that more ES is available for SE-CAVLC as compared to SE-CABAC. But ES is more affected by change in QP values for SE-CAVLC as compared to SE-CABAC. For example, for Foreman video sequence, ES varies from 28.55% to 6.70% for SE-CAVLC when QP varies from 12 to 42. For the same QP range, the change in ES for SE-CABAC is from 19.97% to 9.46% as shown in Table II. From Tables I and II, since PSNR of original H.264/AVC are very similar for both CAVLC and CABAC, in the rest of this section for the sake of comparison, we list only PSNR of CAVLC bitstreams. CABAC is less than SE-CAVLC owing to two reasons. First, ES of SE-CABAC is lesser than that of SE-CAVLC as shown in Tables I andII. Second, CABAC takes lot more processing power than CAVLC. So increase in processing power because of encryption will be lower in terms of percentage. Thus, SE-CAVLC and SE-CABAC is possible in real-time along with compression.

I E E E P r o o f

B. PSNR and Quality of SE-CAVLC and SE-CABAC for I Frames and I+P Frames

Peak signal to noise ratio (PSNR) is widely used objective video quality metric. However, it does not perfectly correlate with a perceived visual quality due to nonlinear behavior of human visual system. Structural similarity index (SSIM) [33] takes into account the structural distortion measurement, since human vision system is highly specialized in extracting structural information from the viewing field. SSIM has a better correlation to the subjective impression. SSIM ranges from -1 to 1. SSIM is 1 when both the images are the same. To present the visual protection of encrypted video sequences, PSNR and SSIM of I and I+P frames are presented.

1) I Frames:

To demonstrate the efficiency of our proposed scheme, we have compressed 100 I frames of each sequence at 30 f/s. Figs. 9 and 10 show the encrypted first frame of Foreman video sequence at different QP values for SE-CAVLC and SE-CABAC, respectively. In H.264/AVC, blocks on the top array are predicted only from left while blocks on left are always predicted from top. Owing to this prediction, a band having width of 8 pixels at top of video frames can be observed for both of SE-CAVLC and SE-CABAC while this band has width of 4 pixels on left of video frames as shown in Figs. 9 and 10. The average PSNR values of Foreman is given in Table IV over whole QP range. It is also compared with the PSNR obtained for the same video sequence without encryption. In Table IV, we present PSNR of original video only for CAVLC. PSNR for CABAC is very much similar as presented in Table I. One can note that whatever is the QP value, the quality of the encrypted video remains in the same lower range. If the probability of each gray level in the image is P(α i ) = 1 2 k , then the encryption of such image is robust against statistical attacks of first order, and thus H(X) = log 2 (2 k ) = k bits/pixel. In the image, the information redundancy r is defined as follows: r = k -H(X).

H(X) = - 2 k -1 i=0 P(α i )log 2 (P(α i )). (4)
(5)

Similarly, the local standard deviation σ(j) for each pixel p(j) taking account of its neighbors to calculate the local mean p(j), is given as follows:

σ(j) = 1 m m i=1 (p(i) -p(j)) ( 6
)
where m is the size of the pixel block to calculate the local mean and standard deviation, and 0 ≤ j < M, if M is the image size. In case of full encryption, entropy H(X) is maximized with high values of local standard deviation. But in case of SE-CAVLC and SE-CABAC, the video frame is transformed to flat regions with blocking artifacts as depicted in Figs. 9 and 10. It is generally owing to variation in pixel values at MB boundaries. For all the benchmark sequences, the average information redundancy r for SE-CAVLC and SE-CABAC sequences is 0.94 and 0.55, respectively, while it is 1.11 for all the original sequences. Despite the fact that SE-CAVLC and SE-CABAC transform the video frames into flat region, the entropy of the encrypted video sequences from (4) is higher as compared to the original sequences. These flat regions are because of two reasons. First, flat regions are due to the fact that prediction is performed from edge pixels of neighboring MBs. Second, pixels have either very high value (bright video frame) or very low value (dark video frame) in SE video frame. This is owing to the fact that during reconstruction pixel value are clipped to 255 if they are greater than it and to 0 if they are below this lower range. So if many pixels have value beyond the upper or lower range, all of them will be clipped to the same value, thus creating a flat region which is either dark or bright. Based on this analysis, the statistical characteristics of SE-CAVLC and SE-CABAC bitstreams vary from full encryption systems.

From (6), we also analyzed the local standard deviation σ for each pixel while taking into account its neighbors. 2) Correlation of Adjacent Pixels: Visual data is highly correlated, i.e., pixels values are highly probable to repeat in horizontal, vertical, and diagonal directions. A correlation of a pixel with its neighboring pixel is then given by a tuple (x i , y i )

I E E E P r o o f

where y i is the adjacent pixel of x i . Since there is always three directions in images, i.e., horizontal, vertical, and diagonal, so we can define correlation direction between any two adjacent pixels as follows:

corr (x,y) = 1 n -1 n 0 ( x i -x i σ x )( y i -y i σ y ) ( 7
)
where n represents the total number of tuples (x i , y i ), x i and y i represent the local mean, and σ x and σ y represent the local standard deviation, respectively.

Owing to the flat regions in SE-CAVLC and SE-CABAC video sequences, the correlation values in these sequences will be higher as compared to original image which contain texture and edges. For all the benchmark sequences, the average horizontal correlation coefficient is 0.88 and 0.87 for the SE-CAVLC and SE-CABAC, respectively, while it is 0.80 for the original sequences. (1-bit different key) For the sake of comparative evaluation of our scheme, we 740 have compared it with six other recent techniques, which 741 include scrambling [9], NAL unit encryption [14], MB header 742 encryption [16], reversible ROI encryption [5], I frame en-743 cryption [2], and multiple Huffman table permutation [36]. 744 These techniques are different from each other in several 745

I E E E P r o o f

and has the security of AES cipher. The proposed method has the advantage of being suitable for streaming over heterogeneous networks because of no change in bitrate. The experiments have shown that we can achieve the desired level of encryption, while maintaining the full bitstream compliance, under a minimal set of computational requirements. The presented security analysis confirmed a sufficient security level for multimedia applications in the context of SE. The proposed system can be extended for ROIspecific video protection [26] for video surveillance and can be applied to medical video transmission [24].

29 W

 29 ITH THE RAPID growth of processing power and net- 30 work bandwidth, many multimedia applications have 31 emerged in the recent past. As digital data can easily be 32 copied and modified, the concern about its protection and au-33 thentication have surfaced. Digital rights management (DRM) 34 has emerged as an important research field to protect the 35 copyrighted multimedia data. DRM systems enforce the rights of the multimedia property owners while ensuring the efficient rightful usage of such property.

  Fig. 1. Block diagram of level coding in CAVLC of H.264/AVC.

Fig. 2 .

 2 Fig. 2. (a) Block diagram of CABAC of H.264/AVC. (b) Binarization stage.

Fig. 3 .

 3 Fig. 3. CFB stream cipher. (a) Encryption. (b) Decryption.

198

  JPEG images by selectively encrypting only luma component199 using AES cipher. The protection rights of individuals and the 200 privacy of certain moving objects in the context of security 201 surveillance systems using viewer generated masking and the

4

  IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY

  the context of DRM systems, our paper addresses the simultaneous SE and compression for state-of-the-art H.264/AVC. The encrypted bitstream is format compliant with absolutely no escalation in bitrate. Furthermore, it does not require lot of processing power for encryption and decryption. In Section III, we describe our proposed approaches to apply SE and H.264/AVC compression in video sequences, simultaneously. III. Proposed SE Schemes Our approach consists of SE during the entropy coding stage of H.264/AVC. In baseline profile, SE is performed in CAVLC entropy coding stage (SE-CAVLC). While in main profile, it is performed in CABAC entropy coding stage (SE-CABAC). In SE of video, encrypted bitstream compliance is a required feature for some direct operations such as displaying, time seeking, and browsing. Encrypted bitstream will be compliant and fulfills real-time constraints if the following three conditions are fulfilled. 1) To keep the bitrate of encrypted bitstream same as the original bitstream, encrypted codewords/binstrings must have the same size as the original codewords/binstrings.

Fig. 4 .

 4 Fig. 4. Block diagram of CAVLC of H.264/AVC. Encircled syntax elements are used for SE-CAVLC.

Fig. 5 .Fig. 6 .

 56 Fig. 5. SE of binstrings in SE-CABAC.

  have different code lengths for each input value as explained in Section II-A2. They do not fulfill the first condition and their encryption will change the bitrate of bitstream. Suffix of EGk and the fixed length code can be encrypted while keeping the bitrate unchanged. EGk is used for binarization of absolute value of levels and MVDs. Number of MVD binstrings have the same length and hence, first and second conditions are fulfilled. But owing to the fact that MVDs are part of MB header and are used for prediction of future motion vectors, their encryption does not fulfill third condition and their encryption makes the bitstream non-compliant. To conclude, the syntax elements which fulfill the criteria for encryption of H.264/AVC compliant bitstream are suffix of EG0 and sign bits of levels. Hence, for each NZ with |NZ| > 14, encryption is performed on l(x) of EG0. It is followed by encryption of syntax element coeff sign flag which represents sign of levels of all nonzero levels. The fixed length code is used for binarization of syntax elements which belong to MB header and cannot be encrypted. To keep the bitrate intact, ES for SE-CABAC consists of only those NZs whose EG0 binstrings have the same length as shown in Fig. 6. EG0 codes, having same code length, constitute the ES and it depends upon NZ . The ES is

Fig. 7 .

 7 Fig. 7. (a) CAVLC plaintext. (b) CABAC plaintext. (c) Proposed SE scheme.

424 2 )

 2 Encryption of the Plaintext with AES in the CFB Mode: 425

431strategy. 3 )

 3 The secret key k is used as the seed of the pseudo 432 random number generator (PRNG). First, the secret key k 433 is divided into 8 bits (byte) sequences. The PRNG produces a random number for each byte component of the key that 435 defines the order of IV formation. Then, we substitute Y 0 436 with the IV, and Y 0 is used in AES to produce Z 1 . As 437 illustrated in Fig. 7(c), with the CFB mode of the AES 438 algorithm, the generation of the keystream Z i depends on the 439 previous encrypted block Y i-1 . Consequently, if two plaintexts 440 are identical X i = X j in the CFB mode, then always the two 441 corresponding encrypted blocks are different, Y i = Y j . 442 Substitution of the Original Bitstream: The third step 443 is the substitution of the original Y i by the encrypted Y i . 444 For SE-CAVLC, CAVLC bitstream is accessed in sequential common intermediate format (QCIF) and SD resolution. For AQ:3 484

Fig. 8 .

 8 Fig. 8. Framewise time taken by SE-CABAC of Foreman video sequence for I+P frames at QP value 18 with intra period 10 during (a) encoding and (b) decoding.

Fig. 9 .

 9 Fig. 9. Decoding of SE-CAVLC frame #1 of Foreman sequence with QP value equal to (a) 18, (b) 30, and (c) 42.

Fig. 10 .

 10 Fig. 10. Decoding of SE-CABAC frame #1 of Foreman sequence with QP value equal to (a) 18, (b) 30, and (c) 42.

Fig. 11 .

 11 Fig. 11. Framewise PSNR of I and I+P frames for Foreman for SE-CAVLC and SE-CABAC at QP value 18.

  Fig. 11. Table VI contains the experimental results of SE of 100 I frames for SD resolution. Here, average PSNR value of luma is 9.82 dB for SE-CAVLC and 9.83 dB for SE-CABAC,which is almost the same as that of QCIF resolution. It is evident that this algorithm is capable to encrypt high-quality information at all resolutions. For the rest of the section, we present analysis for QCIF resolution only, since more benchmark video sequences are available in this resolution.

3 ) 4 )

 34 Key Sensitivity Test: Robustness against cryptanalyst can be improved if the cryptosystem is highly sensitive toward the key. The more the visual data is sensitive toward the key, the more we would have data randomness. For this purpose, a key sensitivity test is assumed where we pick one key and then apply the proposed technique for encryption and then make a 1 bit change in the key and decode the bitstream. Numerical results show that the proposed technique is highly sensitive toward the key change, i.e., a different version of encrypted video sequence is produced when the keys are changed, as shown in Fig. 13. PSNR of luma of decrypted frames with 1bit different key is 10.39 dB and 8.31 dB for SE-CAVLC and SE-CABAC as shown in Table XI. It lies in the same lower range as decoded frames without decryption. Removal of Encrypted Data Attack: In another experiment, we have replaced the encrypted bits with constant values in order to measure the strength of SE-CAVLC and SE-

Fig. 13 .

 13 Fig. 13. Key sensitivity test for encrypted frame #1 of Foreman video sequence for QP value 18. Encrypted frames are decrypted and decoded with (a) original key, (b) 1-bit different key (SE-CAVLC), and (c) 1-bit different key (SE-CABAC).

Fig. 14 .

 14 Fig. 14. Attack in the selectively encrypted image by removing the encrypted data. (a) SE-CAVLC encrypted image {Y, U, V} = {10.01, 26.86, 25.24} dB. (b) SE-CAVLC attacked image {Y, U, V} = {8.87, 27.3, 26.3} dB. (c) SE-CABAC encrypted image {Y, U, V} = {8.20, 17.95, 24.53} dB. (d) SE-CABAC attacked image {Y, U, V} = {7.72, 28.6, 24.6} dB.

764

  crypted video data. Most of the schemes are not format 765 complaint and their encrypted bitstreams cannot be decoded 766 by reference decoder except SE schemes which work in pixel 767

Zafar 8 978"

 8 Shahid received the B.S. degree in electrical 917 engineering from the University of Engineering and 918 Technology, Lahore, Pakistan, and the M.S. degree 919 in image processing from the National Institute 920 of Applied Sciences, Lyon, France, in 2001 and 921 2007, respectively. Currently, he is a Ph.D. student 922 at the Laboratory of Computer Science, Robotics, 923 and Microelectronics, University of Montpellier II, 924 Montpellier, France. 925 Before his M.S. degree, he was a Senior Embedded 926 System Engineer with Streaming Networks, Santa 927 Clara, CA, where he was involved in the research and development in the do-928 main of video processing. His current research interests include compression, 929 watermarking, and encryption of scalable video. 930 Marc Chaumont was born in November 1976, in 931 France. He received the Engineering Diploma degree 932 from the National Institute of Applied Sciences 933 (INSA), Rennes, France, in 1999, and the Ph.D. 934 degree from the IRISA Rennes (INRIA, CNRS, 935 University of Rennes 2, and INSA) in 2003, both 936 in computer sciences. 937 His Ph.D. thesis was about video objects rep-938 resentations, with dynamic coding and scalability 939 functionalities, in the video compression area. He 940 has carried on research activities for one year at the 941 INRIA Rennes and for another year at the University Technological Institute, 942 Bayonne, France, as a Visiting Assistant Professor. During this last year, he 943 focused on face tracking using a deformable 3-D face model. Since September 944 2005, he is an Assistant Professor with the Laboratory of Computer Science, 945 Robotics, and Microelectronics, Montpellier, France, and the University of 946 Nîmes, Nîmes, France. His current research interests include watermarking, 947 steganography, video compression, and to a lesser extent segmentation and 948 tracking in videos. 949 William Puech (M'XX) received the Diploma de-AQ:7 950 gree in electrical engineering from the University 951 of Montpellier, Montpellier, France, in 1991, and 952 the Ph.D. degree in signal-image-speech from the 953 Polytechnic National Institute, Grenoble, France, in 954 1997. 955 He started his research activities in image pro-956 cessing and computer vision. He was a Visiting 957 Research Associate with the University of Thessa-958 loniki, Thessaloniki, Greece. From 1997 to 2000, he 959 was an Assistant Professor with the University of 960 Toulon, Toulon, France, with research interests including methods of active 961 contours applied to medical images sequences. Between 2000 and 2008, he 962 was an Associate Professor and since 2009, he has been a Full Professor of 963 image processing with the Laboratory of Computer Science, Robotics, and 964 Microelectronics, University of Montpellier. He has developed applications 965 on medical images, cultural heritage, and video surveillance. He is the Head 966 of the ICAR Team (Image and Interaction), University of Montpellier. He 967 has published more than 12 journal papers, 4 book chapters, and more than 968 65 conference papers. His current research interests include the areas of 969 protection of visual data (image, video, and 3-D object) for safe transfer by 970 combining watermarking, data hiding, compression, and cryptography.971 Prof. Puech is a reviewer for more than 15 journals (IEEE Transactions 972 on Image Processing, IEEE Transactions on Multimedia, Signal 973 Processing: Image Communication, Journal of Applied Signal Processing, 974 Journal of Electronic Imaging, and others) and more than ten conference 975 proceedings (IEEE ICIP, EUSIPCO, WIAMIS, IWDW, and others). He is 976 currently a member of SPIE. Since 2005, he has been in the Technical Program 977 Committee of EUSIPCO, and since 2009, he has been in the Area Chair AQ:Image and Multidimensional Signal Processing" of EUSIPCO. Please provide the expanded form of NZs. AQ:2= Please provide the expanded form of AC and DC.AQ:3= Please provide the expanded form of SD.

20 our encryption algorithm is better suited for real-time multimedia 21 streaming over heterogeneous networks. It is perfect for playback 22 on handheld devices because of negligible increase in processing 23 power. Nine different benchmark video sequences containing 24 different combinations of motion, texture, and objects are used 25 for experimental evaluation of the proposed algorithm. 26 Index 29 W

 2021222324252629 Terms-AES algorithm, CABAC, CAVLC, selective encryption, stream cipher, video security. ITH THE RAPID growth of processing power and net- 30 work bandwidth, many multimedia applications have 31 emerged in the recent past. As digital data can easily be 32 copied and modified, the concern about its protection and au-33 thentication have surfaced. Digital rights management (DRM) 34 has emerged as an important research field to protect the 35 copyrighted multimedia data. DRM systems enforce the rights Manuscript received December 9, 2009; revised May 18, 2010; accepted July 6, 2010. This work is supported in part by the VOODDO Project (2008-2011), which is a French national project of Agence Nationale de la Recherche (ANR), and the region of Languedoc Roussillon, France. This paper was recommended by Associate Editor M. Barni.

Fig. 1 .

 1 Fig. 1. Block diagram of level coding in CAVLC of H.264/AVC.

Fig. 2 .

 2 Fig. 2. (a) Block diagram of CABAC of H.264/AVC. (b) Binarization stage.

Fig. 3 .

 3 Fig. 3. CFB stream cipher. (a) Encryption. (b) Decryption.

193

  the literature. The AES was applied on the Haar discrete 194 wavelet transform compressed images in[23]. The encryption 195 of color images in the wavelet transform has been addressed

198

  JPEG images by selectively encrypting only luma component199 using AES cipher. The protection rights of individuals and the 200 privacy of certain moving objects in the context of security 201 surveillance systems using viewer generated masking and the 202

4

  IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY

Fig. 4 .

 4 Fig. 4. Block diagram of CAVLC of H.264/AVC. Encircled syntax elements are used for SE-CAVLC.

Fig. 5 .Fig. 6 .

 56 Fig. 5. SE of binstrings in SE-CABAC.

  bitstream. Rather it is performed on binstrings which are input to BAC as shown in Fig. 5. Among all the four binarization techniques, the unary and truncated unary codes have different code lengths for each input value as explained in Section II-A2. They do not fulfill the first condition and their encryption will change the bitrate of bitstream. Suffix of EGk and the fixed length code can be encrypted while keeping the bitrate unchanged. EGk is used for binarization of absolute value of levels and MVDs. Number of MVD binstrings have the same length and hence, first and second conditions are fulfilled. But owing to the fact that MVDs are part of MB header and are used for prediction of future motion vectors, their encryption does not fulfill third condition and their encryption makes the bitstream non-compliant. To conclude, the syntax elements which fulfill the criteria for encryption of H.264/AVC compliant bitstream are suffix of EG0 and sign bits of levels. Hence, for each NZ with |NZ| > 14, encryption is performed on l(x) of EG0. It is followed by encryption of syntax element coeff sign flag which represents sign of levels of all nonzero levels. The fixed length code is used for binarization of syntax elements which belong to MB header and cannot be encrypted. To keep the bitrate intact, ES for SE-CABAC consists of only those NZs whose EG0 binstrings have the same length as shown in Fig. 6. EG0 codes, having same code length, constitute the ES and it depends upon NZ . The ES is

Fig. 7 .

 7 Fig. 7. (a) CAVLC plaintext. (b) CABAC plaintext. (c) Proposed SE scheme.

424 2 )

 2 Encryption of the Plaintext with AES in the CFB Mode: 425

  is divided into 8 bits (byte) sequences. The PRNG produces and to generate the original CAVLC bitstream. Afterward, we 469 apply the entropy decoding and retrieve the quantized DCT 470 coefficients. After the inverse quantization and the inverse 471 DCT we get the decrypted and decoded video frame. In 472 case of SE-CABAC, the difference is that binary arithmetic 473 decoder is used to transform the SE-CABAC bitstream to 474 encrypted binstrings which are then accessed to make the 475 plaintext Y i-1 . The plaintext is decrypted and substituted back 476 to generate original binstrings. They are then passed through 477 inverse binarization, inverse quantization, and inverse DCT 478 steps to get the decrypted and decoded video frame. 479 IV. Experimental Results 480 In this section, we analyze the results for SE-CAVLC and 481 SE-CABAC. We have used the reference implementation of 482 H.264 JSVM 10.2 in AVC mode for video sequences in quarter 483 common intermediate format (QCIF) and SD resolution. For AQ:3 484the experimental results, nine benchmark video sequences have been used for the analysis in QCIF format. Each of them represents different combinations of motion (fast/slow, pan/zoom/rotation), color (bright/dull), contrast (high/low), and objects (vehicle, buildings, people). The video sequences Bus, City, and Foreman contain camera motion while Football and Soccer contain camera panning and zooming along with object motion and texture in background. The video sequences Harbour and Ice contain high luminance images with smooth motion. Mobile sequence contains a complex still background and foreground motion.

Fig. 8 .

 8 Fig. 8. Framewise time taken by SE-CABAC of Foreman video sequence for I+P frames at QP value 18 with intra period 10 during (a) encoding and (b) decoding.

Fig. 9 .

 9 Fig. 9. Decoding of SE-CAVLC frame #1 of Foreman sequence with QP value equal to (a) 18, (b) 30, and (c) 42.

Fig. 10 .

 10 Fig. 10. Decoding of SE-CABAC frame #1 of Foreman sequence with QP value equal to (a) 18, (b) 30, and (c) 42.

Fig. 11 .

 11 Fig. 11. Framewise PSNR of I and I+P frames for Foreman for SE-CAVLC and SE-CABAC at QP value 18.

  10 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY

In

  Table X, the mean local standard deviation for Foremansequence at different QP values is given. For all benchmark video sequences, the mean local standard deviation of luma equals to 69.15 and 61.48 for the SE-CAVLC and SE-CABAC bitstreams, respectively, where the mean local standard deviation is less than ten gray levels for the original benchmark sequences. One can note that local standard deviation of encrypted sequences is higher than original sequences.

3 ) 4 )

 34 Key Sensitivity Test: Robustness against cryptanalyst can be improved if the cryptosystem is highly sensitive toward the key. The more the visual data is sensitive toward the key, the more we would have data randomness. For this purpose, a key sensitivity test is assumed where we pick one key and then apply the proposed technique for encryption and then make a 1 bit change in the key and decode the bitstream. Numerical results show that the proposed technique is highly sensitive toward the key change, i.e., a different version of encrypted video sequence is produced when the keys are changed, as shown in Fig. 13. PSNR of luma of decrypted frames with 1bit different key is 10.39 dB and 8.31 dB for SE-CAVLC and SE-CABAC as shown in Table XI. It lies in the same lower range as decoded frames without decryption. Removal of Encrypted Data Attack: In another experiment, we have replaced the encrypted bits with constant values in order to measure the strength of SE-CAVLC and SE-

Fig. 13 .

 13 Fig. 13. Key sensitivity test for encrypted frame #1 of Foreman video sequence for QP value 18. Encrypted frames are decrypted and decoded with (a) original key, (b) 1-bit different key (SE-CAVLC), and (c) 1-bit different key (SE-CABAC).

Fig. 14 .

 14 Fig. 14. Attack in the selectively encrypted image by removing the encrypted data. (a) SE-CAVLC encrypted image {Y, U, V} = {10.01, 26.86, 25.24} dB. (b) SE-CAVLC attacked image {Y, U, V} = {8.87, 27.3, 26.3} dB. (c) SE-CABAC encrypted image {Y, U, V} = {8.20, 17.95, 24.53} dB. (d) SE-CABAC attacked image {Y, U, V} = {7.72, 28.6, 24.6} dB.

779

  In this paper, an efficient SE system has been proposed for 780 H.264/AVC video codec for CAVLC and CABAC. The SE 781 is performed in the entropy coding stage of the H.264/AVC 782 using the AES encryption algorithm in the CFB mode. In 783 this way, the proposed encryption method does not affect 784 the bitrate and the H.264/AVC bitstream compliance. The SE 785 is performed in CAVLC codewords and CABAC binstrings 786 such that they remain a valid codewords/binstrings thereafter 787 having exactly the same length. Experimental analysis has 788 been presented for I and P frames. The proposed scheme can be used for B frames without any modification, since B frames are also inter-frames but have bidirectional prediction.

  345 For the sake of same bitrate, ES of SE-CAVLC consists of 346 only those NZs whose VLC codewords have the same length. 347 CAVLC uses multiple VLC tables with some threshold for 348 incrementing the table as given in (2). Since the threshold for 349 a specific table is highest possible value possible with that 350 codeword length (this is the case when all the suffix bits of 351 the codeword are 1), magnitude of encrypted NZ is such that 352 VLC table transition is not affected. VLC codes, having same 353 code length, constitute the ES. For VLC n table, ES is 2 n as 354 given in (3). For table VLC0, every NZ has different codeword 355 length, consequently we cannot encrypt the NZs in this table 356 as follows:

		357
	TH[0 . . . 6] = (0, 2, 3, 6, 12, 24, 48, ∞).	(2)
	ES[0 . . . 6] = (1, 2, 4, 8, 16, 32, 64, ∞).

TABLE I

 I Analysis of ES for SE for Different Benchmark VideoSequences at QPValue 18

		SE-CAVLC	SE-CABAC
	Sequence	PSNR	ES	PSNR	ES
		(dB)	(%)	(dB)	(%)
	Bus	44.25	31.05	44.24	19.93
	City	44.29	26.41	44.27	19.79
	Crew	44.82	20.66	44.81	18.97
	Football	44.61	25.33	44.59	19.45
	Foreman	44.38	22.76	44.36	18.72
	Harbor	44.10	30.49	44.09	20.01
	Ice	46.47	24.64	46.46	17.72
	Mobile	44.44	36.17	44.43	19.80
	Soccer	44.27	23.42	44.21	19.94
		SE-CAVLC	SE-CABAC
	QP	PSNR	ES	PSNR	ES
		(dB)	(%)	(dB)	(%)
	12	50.07	28.55	50.05	19.97
	18	44.38	22.76	44.36	18.72
	24	39.43	17.13	39.42	17.61
	30	35.08	13.24	35.08	15.65
	36	31.04	9.88	31.06	12.22
	42	27.23	6.70	27.35	9.46

TABLE II Analysis of ES for SE Over Whole Range of QP Values for Foreman Video Sequence

TABLE III Analysis

 III

			SE-CAVLC			SE-CABAC
		Encoder	Decoder	Encoder	Decoder
	Sequence	I	I+P	I	I+P	I	I+P	I	I+P
		(%)	(%)	(%)	(%)	(%)	(%)	(%)	(%)
	Bus	0.69 0.31	3.77	2.7	0.57 0.25	3.37	2.3
	City	0.5	0.26	3.36	2.4	0.44 0.23	3.06	2.1
	Crew	0.31 0.15	2.52	1.5	0.29 0.14	2.22	1.2
	Football	0.41 0.23	3.46	2.4	0.31 0.18	3.26	2.2
	Foreman	0.47 0.23	3.19	2.2	0.41 0.20	2.99	2.0
	Harbor	0.55 0.30	3.65	2.7	0.47 0.26	3.25	2.3
	Ice	0.41 0.21	3.16	2.1	0.33 0.17	2.96	1.9
	Mobile	0.76 0.35	4.33	3.3	0.72 0.33	4.03	3.0
	Soccer	0.44 0.21	3.17	2.2	0.38 0.18	2.87	1.9
	557

of Increase in Processing Power for SE-CAVLC and SE-CABAC at QP Value 18

Table III gives a detailed overview of the required process-542 ing power for I and I+P video sequences at QP value 18. 543 Intra period has been set 10 for I+P video sequences. One 544 can observe that increase in computation time for encoder is 545 less than 0.4% for both of SE-CAVLC and SE-CABAC while 546 it is below than 3% for decoder for I+P sequence.

547

Fig. 8(a) and (b) shows the framewise analysis of increase 548 in processing power for SE-CABAC at QP value 18 for Fore-549 man. For experimentation, 2.1 GHz Intel Core 2 Duo T8100 550 machine with 3072 MB random access memory has been used. 551 For I+P sequence encoding of 100 frames with intra period 552 10, it took 4372.5 s and 4381.3 s for CABAC and SE-CABAC, 553 respectively. While it took 2.005 s and 2.045 s for CABAC and 554 SE-CABAC decoding. It is a negligible increase in processing 555 power and can be managed well even by handheld devices. It 556 is important to note that increase in processing power of SE-

Table VII

 VII

shows the SSIM values of luma of benchmark video sequences without encryption and with SE. Results

TABLE IV PSNR

 IV Comparison for I Frames Without Encryption and with SE for Foreman at Different QP Values

	PSNR (Y) (dB)	PSNR (U) (dB)	PSNR (V) (dB)
	QP ORIG	SE	SE	ORIG	SE	SE	ORIG	SE	SE
		CAVLC	CABAC			CAVLC	CABAC		CAVLC	CABAC
	12 50.1	8.6	8.4	50.0 19.8 24.1 50.8	9.6	22.6
	18 44.4	8.7	8.6	45.7 24.1 24.4 47.6 10.2 22.1
	24 39.4	8.7	8.7	41.9 26.4 24.4 44.2 24.9 22.8
	30 35.1	9.4	8.7	39.8 27.4 24.6 41.4 25.4 23.6
	36 31.0	9.4	8.5	37.7 28.1 24.9 38.6 24.8 23.2
	42 27.2	9.4	8.7	36.2 25.5 24.9 36.9 24.6 24.0
				TABLE V
	PSNR Comparison for I Frames Without Encryption and with SE
				at QP Value 18
		PSNR (Y) (dB)	PSNR (U) (dB)	PSNR (V) (dB)
	Sequence ORIG SE	SE ORIG SE	SE ORIG SE	SE
		CAVLC	CABAC		CAVLC	CABAC	CAVLC	CABAC

Bus 44.2 7.9 8.2 45.2 26.8 25.0 46.6 26.6 27.2 City 44.3 10.9 11.2 45.8 31.9 30.3 46.8 33.5 31.8 Crew 44.8 9.0 9.9 45.8 24.0 23.4 45.7 19.7 19.8 Football 44.6 11.5 11.5 45.8 14.9 14.4 46.0 24.3 23.6 Foreman 44.4 8.7 8.6 45.7 24.1 24.4 47.6 10.2 22.1 Harbor 44.1 9.2 9.5 45.6 27.1 24.6 46.7 33.2 31.3 Ice 46.5 10.6 10.4 48.8 24.3 25.6 49.3 16.9 20.4 Mobile 44.4 8.3 8.3 44.1 10.4 13.1 44.1 9.6 11.0 Soccer 44.3 9.3 10.6 46.6 22.1 19.7 47.9 28.2 24.4 Average 44.6 9.5 9.8 46.0 22.8 22.3 46.7 22.5 23.5

TABLE VI PSNR

 VI 614 formation present in the original video. Average SSIM value 615 of video sequences without encryption is 0.993, while it is 616 0.164 and 0.180 for SE-CAVLC and SE-CABAC, respectively. 617 Fig.12shows the framewise SSIM of luma of Foreman video 618 sequence for I frames. It is important to note SSIM value of 619 complex video sequences is less than that of simple video 620 sequences. Video data normally consists of an I frame 622 and a trail of P frames. I frames are inserted periodically to 623 restrict the drift because of lossy compression and rounding 624 errors. In these experiments, intra period is set at 10 in a 625 sequence of 100 frames. Results shown in Table VIII verify the 626 effectiveness of our scheme over the whole range of QP values 627 for Foreman video sequence. Table IX verifies the performance 628 of our algorithm for all video sequences for I+P frames at 629 QP value 18. Average PSNR of luma for all the sequences 630 is 9.75 dB and 10.02 dB for SE-CAVLC and SE-CABAC, 631

	P r o o f
	I E E E

Comparison for I Frames Without Encryption and with SE at QP Value 18 (SD Resolution) PSNR (Y) (dB) PSNR (U) (dB) PSNR (V) (dB) Sequence ORIG SE SE ORIG SE SE ORIG SE SE CAVLC CABAC CAVLC CABAC CAVLC CABAC City 44.6 9.9 10.1 47.8 27.3 26.2 49.1 31.4 29.9 Crew 45.2 9.1 9.1 46.6 24.5 22.8 47.7 20.1 20.0 Harbor 44.5 9.4 9.4 47.5 22.9 22.9 48.7 28.8 26.8 Ice 46.2 10.7 10.4 51.5 27.8 27.8 52.0 25.0 26.0 Soccer 45.1 10.0 10.2 47.7 18.4 18.0 49.2 26.7 24.1 Average 45.1 9.8 9.8 48.2 24.2 23.5 49.4 26.4 25.4 verify the proposed scheme has distorted the structural in-621 2) I+P Frames:

TABLE VII SSIM

 VII Comparison of luma of I Frames Without Encryption and with SE at QP Value 18

	Sequence	CAVLC	SE-CAVLC	CABAC	SE-CABAC
	Bus	0.995	0.069	0.994	0.064
	City	0.994	0.115	0.994	0.093
	Crew	0.991	0.184	0.991	0.153
	Football	0.991	0.219	0.991	0.184
	Foreman	0.990	0.198	0.990	0.165
	Harbor	0.998	0.047	0.998	0.038
	Ice	0.990	0.419	0.990	0.398
	Mobile	0.998	0.040	0.998	0.356
	Soccer	0.988	0.185	0.988	0.171
	Average	0.993	0.164	0.993	0.180

Fig. 12. Framewise SSIM of I frames for Foreman for SE-CABAC at QP value 18.

TABLE VIII PSNR

 VIII

		Sequence ORIG SE	SE ORIG SE	SE ORIG SE	SE
			CAVLC	CABAC	CAVLC	CABAC	CAVLC	CABAC
		12	49.6 8.7	8.1 49.9 18.4	23.0 50.6 10.4	21.6
		18	43.9 9.1	10.4 45.5 23.6	23.9 47.6 8.0	23.2
		24	38.9 9.6	9.7 42.0 26.9	24.9 44.3 25.8	25.0
		30	34.6 9.2	9.2 39.8 28.6	24.9 41.5 26.6	24.0
		36	30.7 10.1	8.2 37.9 28.4	24.3 38.8 22.8	23.3
		42	27.0 9.4	8.6 36.3 26.5	26.8 36.9 25.6	24.6
	640
	641	considering the variations (local or global) in the protected
	642	image. Entropy is a statistical measure of randomness or
	643	disorder of a system which is mostly used to characterize the
	644	texture in the input images. Considering this, the information
	645	content of image can be measured with the entropy H(X) and
		local standard deviation σ(j). If an image has 2 k gray levels
	649

Comparison for I+P Frames Without Encryption and with SE for Foreman at Different QP Values PSNR (Y) (dB) PSNR (U) (dB) PSNR (V) (dB) respectively. Fig. 11 shows the framewise PSNR of luma of 632 Foreman video sequence for I+P. Here, PSNR of SE-CAVLC 633 and SE-CABAC remains almost the same for sequence of P 634 frames and changes at every I frame, thus producing a staircase 635 graph. SSIM quality metric has very low values and is not 636 given here for the sake of brevity. 637 C. Security Analysis 638 1) Analysis of Entropy and Local Standard Deviation: 639 The security of the encrypted image can be measured by 646 α i with 0 ≤ i ≤ 2 k and the probability of gray level α i is 647 P(α i ), and without considering the correlation of gray levels, 648 the first order entropy H(X) is defined as follows:

TABLE X

 X Standard Deviation for SE of Foreman Video Sequence at

			Different QP Values
			CAVLC	CABAC
	QP	ORIG SE-CAVLC	ORIG SE-CABAC
	12	6.75	71.49	7.02	69.69
	18	7.21	73.23	7.53	59.97
	24	8.57	91.98	8.63	84.55
	30	6.35	35.99	6.71	57.87
	36	6.90	47.42	6.93	68.04
	42	7.91	75.26	8.11	71.17

In Table

X

, the mean local standard deviation for Foreman sequence at different QP values is given. For all benchmark video sequences, the mean local standard deviation of luma equals to 69.15 and 61.48 for the SE-CAVLC and SE-CABAC bitstreams, respectively, where the mean local standard deviation is less than ten gray levels for the original benchmark sequences. One can note that local standard deviation of encrypted sequences is higher than original sequences.

2) Correlation of Adjacent Pixels: Visual data is highly correlated, i.e., pixels values are highly probable to repeat in horizontal, vertical, and diagonal directions. A correlation of a pixel with its neighboring pixel is then given by a tuple (x i , y i )

TABLE XI Key

 XI Sensitivity Test of SE-CAVLC and SE-CABAC Encrypted Video for Frame #1 Foreman Video Sequence for QP Value 18

		PSNR (Y)	PSNR (U)	PSNR (V)
		(dB)	(dB)	(dB)
	Original key	44.60	45.73	47.35
	SE-CAVLC	10.39	24.46	14.02
	(1-bit different key)
	SE-CABAC	8.31	25.13	24.82

TABLE XII

 XII Comparison of Proposed Scheme with Other Recent Methods

	747	Video SE Scheme Bitrate and encryption algorithm (pseudo random permutation, stream Format Robust to Domain
	748	cipher, or AES). The comparison has been made based on
	749	several important characteristics of SE systems and is summa-
		rized in Table XII. Encryption algorithm used in SE scheme

750

is of vital importance for the security level. AES has the 751 highest security among all the known ciphers and our proposed 752 scheme utilizes AES. Among the recent techniques, AES has 753 been used only in

[2]

but their SE scheme is very naive and 754 encrypts only I frames. 755 SE should not result in increase of bitrate. For example, if 756 a video for 3G wireless connection has bitrate of 384 kb/s, its 757 encrypted version should have the same bitrate. Otherwise, it 758 cannot be played back on 3G connection. Our scheme keeps 759 the bitrate intact. It is in contrast to other schemes which either 760

  for I and P frames. The proposed scheme can be used for B frames without any modification, since B frames are also inter-frames but have bidirectional prediction.

	778
	779	V. Conclusion
	780	In this paper, an efficient SE system has been proposed for
		H.264/AVC video codec for CAVLC and CABAC. The SE

768

Our SE-CABAC scheme is the first format compliant tech-769 nique which is for arithmetic coding-based entropy coding 770 module, while keeping the bitrate unchanged. Recent encryp-771 tion techniques for arithmetic coding

[11]

,

[13]

are not format 772 complaint and require lot of processing power. 773 To summarize, our proposed schemes (SE-CAVLC and 774 SE-CABAC) meet all the requirements of an integrated 775 compression-encryption system. Our proposed system is fully 776 compliant to H.264/AVC decoder, with no change in bitrate 777 and has the security of AES cipher. 781 is performed in the entropy coding stage of the H.264/AVC 782 using the AES encryption algorithm in the CFB mode. In 783 this way, the proposed encryption method does not affect 784 the bitrate and the H.264/AVC bitstream compliance. The SE 785 is performed in CAVLC codewords and CABAC binstrings 786 such that they remain a valid codewords/binstrings thereafter 787 having exactly the same length. Experimental analysis has 788 been presented

This paper presents a novel method for the protec- 1 tion of bitstreams of state-of-the-art video codec H.264/AVC. The 2 problem of selective encryption (SE) is addressed along with the 3 compression in the entropy coding modules. H.264/AVC supports 4 two types of entropy coding modules. Context-adaptive variable 5 length coding (CAVLC) is supported in H.264/AVC baseline 6 profile and context-adaptive binary arithmetic coding (CABAC) purpose, in this paper the encryption step is done simultaneously 10 with the entropy coding CAVLC or CABAC. SE is performed by 11 using the advanced encryption standard (AES) algorithm with 12 the cipher feedback mode on a subset of codewords/binstrings. 13 For CAVLC, SE is performed on equal length codewords from 14 a specific variable length coding table. In case of CABAC, it is done on equal length binstrings. In our scheme, entropy coding 16 module serves the purpose of encryption cipher without affecting 17 the coding efficiency of H.264/AVC by keeping exactly the same 18 bitrate, generating completely compliant bitstream and utilizing negligible computational power. Owing to no escalation in bitrate,

	Protection of H.264/AVC by Selective
	Encryption of CAVLC and CABAC for
	I and P Frames
	Zafar Shahid, Marc Chaumont, and William Puech, Member, IEEE
	Abstract-

  is not affected. VLC codes, having same 353 code length, constitute the ES. For VLC n table, ES is 2 n as 354 given in (3). For table VLC0, every NZ has different codeword 355 length, consequently we cannot encrypt the NZs in this table 356 as follows:

		357
	TH[0 . . . 6] = (0, 2, 3, 6, 12, 24, 48, ∞).	(2)

ES[0 . . . 6] = (1, 2, 4, 8, 16, 32, 64, ∞).

TABLE I

 I Analysis of ES for SE for Different Benchmark Video

	Sequences at QP Value 18
		SE-CAVLC	SE-CABAC
	Sequence	PSNR	ES	PSNR	ES
		(dB)	(%)	(dB)	(%)
	Bus	44.25	31.05	44.24	19.93
	City	44.29	26.41	44.27	19.79
	Crew	44.82	20.66	44.81	18.97
	Football	44.61	25.33	44.59	19.45
	Foreman	44.38	22.76	44.36	18.72
	Harbor	44.10	30.49	44.09	20.01
	Ice	46.47	24.64	46.46	17.72
	Mobile	44.44	36.17	44.43	19.80
	Soccer	44.27	23.42	44.21	19.94
		TABLE II
	Analysis of ES for SE Over Whole Range of QP Values for
		Foreman Video Sequence
		SE-CAVLC	SE-CABAC
	QP	PSNR	ES	PSNR	ES
		(dB)	(%)	(dB)	(%)
	12	50.07	28.55	50.05	19.97
	18	44.38	22.76	44.36	18.72
	24	39.43	17.13	39.42	17.61
	30	35.08	13.24	35.08	15.65
	36	31.04	9.88	31.06	12.22
	42	27.23	6.70	27.35	9.46

TABLE III

 III Analysis of Increase inProcessing Power for SE-CAVLC and SE-CABAC at QP Value 18

				SE-CAVLC			SE-CABAC
			Encoder	Decoder	Encoder	Decoder
		Sequence	I	I+P	I	I+P	I	I+P	I	I+P
			(%)	(%)	(%)	(%)	(%)	(%)	(%)	(%)
		Bus	0.69 0.31	3.77	2.7	0.57 0.25	3.37	2.3
		City	0.5	0.26	3.36	2.4	0.44 0.23	3.06	2.1
		Crew	0.31 0.15	2.52	1.5	0.29 0.14	2.22	1.2
		Football	0.41 0.23	3.46	2.4	0.31 0.18	3.26	2.2
		Foreman	0.47 0.23	3.19	2.2	0.41 0.20	2.99	2.0
		Harbor	0.55 0.30	3.65	2.7	0.47 0.26	3.25	2.3
		Ice	0.41 0.21	3.16	2.1	0.33 0.17	2.96	1.9
		Mobile	0.76 0.35	4.33	3.3	0.72 0.33	4.03	3.0
		Soccer	0.44 0.21	3.17	2.2	0.38 0.18	2.87	1.9
	542	Table III gives a detailed overview of the required process-
	543	ing power for I and I+P video sequences at QP value 18.
	544	Intra period has been set 10 for I+P video sequences. One
	545	can observe that increase in computation time for encoder is
	546	less than 0.4% for both of SE-CAVLC and SE-CABAC while
	547	it is below than 3% for decoder for I+P sequence.
	548	Fig. 8(a) and (b) shows the framewise analysis of increase
	549	in processing power for SE-CABAC at QP value 18 for Fore-
	550	man. For experimentation, 2.1 GHz Intel Core 2 Duo T8100
	551	machine with 3072 MB random access memory has been used.
	552	For I+P sequence encoding of 100 frames with intra period
	553	10, it took 4372.5 s and 4381.3 s for CABAC and SE-CABAC,
	554	respectively. While it took 2.005 s and 2.045 s for CABAC and
	555	SE-CABAC decoding. It is a negligible increase in processing
		power and can be managed well even by handheld devices. It
	557

556

is important to note that increase in processing power of SE-

Table V

 V compares the average PSNR of 100 I frames of all benchmark video sequences at QP value 18 without encryption and with SE. Average PSNR value of luma for all the sequences at QP value 18 is 9.49 dB for SE-CAVLC and 9.80 dB for SE-CABAC. It confirms that this algorithm works well for various combinations of motion, texture, and objects for I frames. It is also evident in framewise PSNR of luma of I frames of Foreman video sequence as shown in Fig. 11. Table VI contains the experimental results of SE of 100 I frames for SD resolution. Here, average PSNR value of luma is 9.82 dB for SE-CAVLC and 9.83 dB for SE-CABAC, which is almost the same as that of QCIF resolution. It is evident that this algorithm is capable to encrypt high-quality information at all resolutions. For the rest of the section, we present analysis for QCIF resolution only, since more benchmark video sequences are available in this resolution.Table VII shows the SSIM values of luma of benchmark video sequences without encryption and with SE. Results

TABLE IV PSNR

 IV Comparison for I Frames Without Encryption and with SE for Foreman at Different QP Values

	PSNR (Y) (dB)	PSNR (U) (dB)	PSNR (V) (dB)
	QP ORIG	SE	SE	ORIG	SE	SE	ORIG	SE	SE
		CAVLC	CABAC			CAVLC	CABAC		CAVLC	CABAC
	12 50.1	8.6	8.4	50.0 19.8 24.1 50.8	9.6	22.6
	18 44.4	8.7	8.6	45.7 24.1 24.4 47.6 10.2 22.1
	24 39.4	8.7	8.7	41.9 26.4 24.4 44.2 24.9 22.8
	30 35.1	9.4	8.7	39.8 27.4 24.6 41.4 25.4 23.6
	36 31.0	9.4	8.5	37.7 28.1 24.9 38.6 24.8 23.2
	42 27.2	9.4	8.7	36.2 25.5 24.9 36.9 24.6 24.0
				TABLE V
	PSNR Comparison for I Frames Without Encryption and with SE
				at QP Value 18
		PSNR (Y) (dB)	PSNR (U) (dB)	PSNR (V) (dB)
	Sequence ORIG SE	SE ORIG SE	SE ORIG SE	SE
		CAVLC	CABAC		CAVLC	CABAC	CAVLC	CABAC
	Bus	44.2 7.9	8.2 45.2 26.8	25.0 46.6 26.6	27.2
	City	44.3 10.9	11.2 45.8 31.9	30.3 46.8 33.5	31.8
	Crew	44.8 9.0	9.9 45.8 24.0	23.4 45.7 19.7	19.8
	Football 44.6 11.5	11.5 45.8 14.9	14.4 46.0 24.3	23.6
	Foreman 44.4 8.7	8.6 45.7 24.1	24.4 47.6 10.2	22.1
	Harbor 44.1 9.2	9.5 45.6 27.1	24.6 46.7 33.2	31.3
	Ice	46.5 10.6	10.4 48.8 24.3	25.6 49.3 16.9	20.4
	Mobile	44.4 8.3	8.3 44.1 10.4	13.1 44.1 9.6	11.0
	Soccer	44.3 9.3	10.6 46.6 22.1	19.7 47.9 28.2	24.4
	Average 44.6 9.5	9.8 46.0 22.8	22.3 46.7 22.5	23.5

TABLE VI PSNR

 VI verify the proposed scheme has distorted the structural in-614 formation present in the original video. Average SSIM value 615 of video sequences without encryption is 0.993, while it is 616 0.164 and 0.180 for SE-CAVLC and SE-CABAC, respectively. 617 Fig.12shows the framewise SSIM of luma of Foreman video 618 sequence for I frames. It is important to note SSIM value of 619 complex video sequences is less than that of simple video 620 sequences. Video data normally consists of an I frame 622 and a trail of P frames. I frames are inserted periodically to 623 restrict the drift because of lossy compression and rounding 624 errors. In these experiments, intra period is set at 10 in a 625 sequence of 100 frames. Results shown in Table VIII verify the 626 effectiveness of our scheme over the whole range of QP values 627 for Foreman video sequence. Table IX verifies the performance 628 of our algorithm for all video sequences for I+P frames at 629 QP value 18. Average PSNR of luma for all the sequences 630 is 9.75 dB and 10.02 dB for SE-CAVLC and SE-CABAC, 631

	P r o o f
		Sequence ORIG SE	SE ORIG SE	SE ORIG SE	SE
			CAVLC	CABAC	CAVLC	CABAC	CAVLC	CABAC
		City	44.6 9.9	10.1 47.8 27.3	26.2 49.1 31.4	29.9
	I E E E	Crew Harbor 44.5 9.4 45.2 9.1 Ice 46.2 10.7 Soccer 45.1 10.0 Average 45.1 9.8	9.1 46.6 24.5 9.4 47.5 22.9 10.4 51.5 27.8 10.2 47.7 18.4 9.8 48.2 24.2	22.8 47.7 20.1 22.9 48.7 28.8 27.8 52.0 25.0 18.0 49.2 26.7 23.5 49.4 26.4	20.0 26.8 26.0 24.1 25.4

Comparison for I Frames Without Encryption and with SE at QP Value 18 (SD Resolution) PSNR (Y) (dB) PSNR (U) (dB) PSNR (V) (dB) 621 2) I+P Frames:

TABLE VII SSIM

 VII Comparison of luma of I Frames Without Encryption and with SE at QP Value 18

	Sequence	CAVLC	SE-CAVLC	CABAC	SE-CABAC
	Bus	0.995	0.069	0.994	0.064
	City	0.994	0.115	0.994	0.093
	Crew	0.991	0.184	0.991	0.153
	Football	0.991	0.219	0.991	0.184
	Foreman	0.990	0.198	0.990	0.165
	Harbor	0.998	0.047	0.998	0.038
	Ice	0.990	0.419	0.990	0.398
	Mobile	0.998	0.040	0.998	0.356
	Soccer	0.988	0.185	0.988	0.171
	Average	0.993	0.164	0.993	0.180

Fig. 12. Framewise SSIM of I frames for Foreman for SE-CABAC at QP value 18.

TABLE VIII PSNR

 VIII

		Sequence ORIG SE	SE ORIG SE	SE ORIG SE	SE
			CAVLC	CABAC	CAVLC	CABAC	CAVLC	CABAC
		12	49.6 8.7	8.1 49.9 18.4	23.0 50.6 10.4	21.6
		18	43.9 9.1	10.4 45.5 23.6	23.9 47.6 8.0	23.2
		24	38.9 9.6	9.7 42.0 26.9	24.9 44.3 25.8	25.0
		30	34.6 9.2	9.2 39.8 28.6	24.9 41.5 26.6	24.0
		36	30.7 10.1	8.2 37.9 28.4	24.3 38.8 22.8	23.3
		42	27.0 9.4	8.6 36.3 26.5	26.8 36.9 25.6	24.6
		respectively. Fig. 11 shows the framewise PSNR of luma of
	637
	638	C. Security Analysis
	639	1) Analysis of Entropy and Local Standard Deviation:
	640	The security of the encrypted image can be measured by
	641	considering the variations (local or global) in the protected
	642	image. Entropy is a statistical measure of randomness or
	643	disorder of a system which is mostly used to characterize the
	644	texture in the input images. Considering this, the information
	645	content of image can be measured with the entropy H(X) and
		local standard deviation σ(j). If an image has 2 k gray levels
	649

Comparison for I+P Frames Without Encryption and with SE for Foreman at Different QP Values PSNR (Y) (dB) PSNR (U) (dB) PSNR (V) (dB) 632 Foreman video sequence for I+P. Here, PSNR of SE-CAVLC 633 and SE-CABAC remains almost the same for sequence of P 634 frames and changes at every I frame, thus producing a staircase 635 graph. SSIM quality metric has very low values and is not 636 given here for the sake of brevity. 646 α i with 0 ≤ i ≤ 2 k and the probability of gray level α i is 647 P(α i ), and without considering the correlation of gray levels, 648 the first order entropy H(X) is defined as follows:

TABLE X

 X Standard Deviation for SE of Foreman Video Sequence at

			Different QP Values
			CAVLC	CABAC
	QP	ORIG SE-CAVLC	ORIG SE-CABAC
	12	6.75	71.49	7.02	69.69
	18	7.21	73.23	7.53	59.97
	24	8.57	91.98	8.63	84.55
	30	6.35	35.99	6.71	57.87
	36	6.90	47.42	6.93	68.04
	42	7.91	75.26	8.11	71.17

TABLE XI Key

 XI Sensitivity Test of SE-CAVLC and SE-CABAC Encrypted Video for Frame #1 Foreman Video Sequence for QP Value 18

		PSNR (Y)	PSNR (U)	PSNR (V)
		(dB)	(dB)	(dB)
	Original key	44.60	45.73	47.35
	SE-CAVLC	10.39	24.46	14.02
	(1-bit different key)
	SE-CABAC	8.31	25.13	24.82

This work is supported in part by the VOODDO Project (2008-2011), which is a French national project of Agence Nationale de la Recherche (ANR), and the region of Languedoc Roussillon, France.

The decryption process in the CFB mode works as follows.

456

The previous block Y i-1 is used as the input to the AES allow increase in bitrate [5], [9], [36] or use stream cipher 761 for the sake of same bitrate [14], [16], thus compromising on 762 the security of the system.

763

Format compliance is another important aspect for en-