Himanshu Aggarwal

Member, IEEE Dinesh K Vishwakarma

Covariate conscious approach for Gait recognition based upon Zernike moment invariants

Keywords: Human gait recognition, Gait biometrics, Average energy silhouette image, Zernike moment invariants, Spatial distribution of oriented gradients, Mean of directional pixels

Gait recognition i.e. identification of an individual from his/her walking pattern is an emerging field. While existing gait recognition techniques perform satisfactorily in normal walking conditions, their performance tend to suffer drastically with variations in clothing and carrying conditions. In this work, we propose a novel covariate cognizant framework to deal with the presence of such covariates. We describe gait motion by forming a single 2D spatio-temporal template from video sequence, called Average Energy Silhouette image (AESI). Zernike moment invariants (ZMIs) are then computed to screen the parts of AESI infected with covariates. Following this, features are extracted from Spatial Distribution of Oriented Gradients (SDOGs) and novel Mean of Directional Pixels (MDPs) methods. The obtained features are fused together to form the final well-endowed feature set. Experimental evaluation of the proposed framework on three publicly available datasets i.e. CASIA dataset B, OU-ISIR Treadmill dataset B and USF Human-ID challenge dataset with recently published gait recognition approaches, prove its superior performance.

I. INTRODUCTION

B IOMETRICS have become a common sight these days.

While fingerprint scanner, iris scanner, palm scanner etc. are being extensively used in offices, banks, legal organizations; there are still some biometric methods whose usage in daily lives has been much constricted.

Individual recognition on the basis of gait is one such domain. The domain exploits the notion that every individual has its own idiosyncratic way of walking. Also, gait data collection being non-intrusive, contact-free and taking into account its easy availability, owing to the widespread usage of surveillance cameras, make gait biometrics a very promising field. However the efficiency of gait recognition algorithms do suffer from presence of some external factors such as covariates like clothing, footwear, bags, etc. This makes gait biometrics a challenging task but nevertheless useful. Gait recognition approaches can be classified into broadly two categories: Model based approach and Model-free (Appearance based) approach. Model based approach [START_REF] Johansson | Visual perception of biological motion and a model for its analysis[END_REF] [2] [3] [4] [START_REF] Rida | Human body part selection by group lasso of motion for model-free gait recognition[END_REF] employ modelling of human body structure and local movement patterns of different body parts. The parameters of the Himanshu Aggarwal is working as an Associate Software Engineer in the Division of Computer Vision at Qualcomm Incorporated, Hyderabad, India e-mail: (hiaggarw@qti.qualcomm.com, himanshub43@gmail.com) D.K. Vishwakarma is working as an Assistant Professor in the Department of Electronics and Communication Engineering, Delhi Technological University, Delhi, India-11042 e-mail: (dinesh@dtu.ac.in, dvishwakarma@gmail.com) models, like knee position, pelvis to knee distance, hip position etc. are mostly learnt by processing the gait sequence. Model based approach is generally view and scale invariant. However, the performance of this approach is highly dependent on the quality of extracted human silhouettes, low quality of which can lead to inaccurate and incorrect parameter estimation resulting in plummeting of the performance. On the other hand, Model free approach [START_REF] Man | Individual recognition using gait energy image[END_REF] [7] [START_REF] Bashir | Gait recognition without subject cooperation[END_REF] [9] [START_REF] Xu | Marginal fisher analysis and its variants for human gait recognition and content-based image retrieval[END_REF] is not explicitly based on structural information. This approach works on temporal and shape information. Compared to model-based approach, model-free approach is computationally less expensive and use of temporal information results in much better recognition performance. Furthermore, model-free approach is more robust to noise and has thus become the preferred approach in research.

One of the primary challenge for vision based gait recognition techniques is to disentangle the identity-unrelated factors which yet alter gait appearances drastically. Though existing works [START_REF] Man | Individual recognition using gait energy image[END_REF] [START_REF] Wang | Human identification using temporal information preserving gait template[END_REF] [10] [START_REF] Tao | General tensor discriminant analysis and gabor features for gait recognition[END_REF] [12] [START_REF] Li | Discriminant locally linear embedding with high-order tensor data[END_REF], give satisfactory performance in normal walking conditions taken under controlled environment, these methods are sensitive to silhouette distortion which occur due to the presence of covariates. A representative gallery set is required for such metrics to perform effectively. In real-world scenarios such an assumption does not hold true. Since the conditions in which a query gait sequence is collected is mostly unknown, a gallery set encapsulating entire population cannot be obtained.

To address these problems, a unified covariate cognizant approach to gait recognition has been proposed in this paper. The method works on spatio-temporal image representation of gait named AESI [START_REF] Vishwakarma | Human activity recognition based on spatial distribution of gradients at sub-levels of average energy silhouette images[END_REF]. AESI is a compact representation of gait which preserves the necessary shape and temporal information of the gait sequence. Furthermore, the presented framework employ two stages. The first stage involves detection of anomalies due to presence of covariates. Zernike invariant moments [15] are used for this purpose. The moments are computed for the AESI and then the distance between the moments is used as a comparison metric. Thereafter, second stage involve feature extraction from parts of AESI devoid of any anomalies as decided by the comparison in the first stage. For this, Spatial distribution of gradients (SDOG) [START_REF] Vishwakarma | Human activity recognition based on spatial distribution of gradients at sub-levels of average energy silhouette images[END_REF] is employed. SDOG calculates local orientation of gradients over the silhouette image (AESI) and its sub-pyramids creating feature set that encapsulates both local and global information about the image. In addition to SDOG, MDPs are computed and integrated in the feature set. MDPs show distribution of the MDPs in direction, the direction in which walking motion is most prominent [START_REF] Foster | Automatic gait recognition using area-based metrics[END_REF]. MDPs are computed only in regions of high temporality, typically the limb and neck area, and preserve structural information in these regions. As a result the final formed feature set has extensive distinguishing capability, in addition to being low dimensional, resulting in optimal recognition performance. Moreover, the classical approach employed in existing works [START_REF] Man | Individual recognition using gait energy image[END_REF] [7] [START_REF] Lam | Gait flow image: A silhouettebased gait representation for human identification[END_REF] is to use Principal component analysis(PCA) and Multiple Discriminant Analysis(MDA) approach for feature set dimension reduction obtained from direct matching on various kinds of spatio temporal representations of gait. Our method intrinsically generate a reliable and compact feature set, thus making the use of these approaches superfluous.

Experimental results on three popular gait datasets demonstrate the efficacy of our approach. Our method achieved an overall recognition accuracy of 72.7% on OU-ISIR Treadmill B dataset [START_REF] Makihara | The ou-isir gait database comprising the treadmill dataset[END_REF], this is better than recent GII-SF approach [START_REF] Arora | Gait based authentication using gait information image features[END_REF] by 11.5%. For CASIA B dataset [START_REF] Yu | A framework for evaluating the effect of view angle, clothing and carrying condition on gait recognition[END_REF], the improvement is even more, where our method achieved overall accuracy of 91.47% which is nearly 19% better than the GII-SF. We also achieved accuracy of 72.53% on the USF Human-ID challenge dataset [START_REF] Sarkar | The humanid gait challenge problem: data sets, performance, and analysis[END_REF] which slightly outperforms the very recent clothing and carrying condition invariant rotation forest approach [START_REF] Choudhurya | Clothing and carrying condition invariant gait recognition based on rotation forest[END_REF]. The key contributions of this work are as follows:

1) A new consolidated approach is developed for the recognition of human gait that is capable of detecting covariates and minimize their repercussions by employing AESI, ZMIs, SDOGs and MDPs. 2) Enlistment of AESIs that effectively represents temporal and shape evidence by fusing gait silhouette images using reference point alignment and forming a single 2D templates. 3) Introduction of a novel method for determination of presence of covariates by segmenting AESI into multiple parts and using ZMIs as a comparison metric. 4) Adoption of SDOGs method for defining gait characteristics, that is based on local gradient orientation for feature extraction from gait template and provides low dimensional feature set that encompass both local and global viable information. 5) A novel MDPs method that extracts structural information from high temporal regions by studying variation of MDPs in horizontal direction, the direction in which gait motion is most prominent. The remainder of the paper is organized as follows: A brief survey of the existing popular gait recognition techniques is done in Section 2. Section 3 discusses the pipeline of the proposed approach and details the various modules of the pipeline. A detailed analysis of the performance of the proposed technique is performed in Section 4, alongside comparisons with existing similar state-of-the-art methods. Section 6 concludes the paper.

II. RELATED WORKS

In recent years, several techniques have been devised for the recognition of individuals through their gait. As already discussed two approaches to gait recognition exist -Model based approach and Appearance based approach. In model based approach human body structure is modelled through various geometrical shapes. Johansson [START_REF] Johansson | Visual perception of biological motion and a model for its analysis[END_REF] introduced Moving light display i.e., MLD, a unique technique for gait recognition. MLD involved fastening of bright markers to human clothing in order to capture gait motion of subject. Ben Abdelkader et al. [START_REF] Benabdelkader | Person identification using automatic height and stride estimation[END_REF] argued that gait stride and cadence can be used for identification of individuals. The method estimated stride length using a calibrated camera and then used Bayesian classifier for identification. Johnson and Bobick [START_REF] Bobick | Gait recognition using static, activityspecific parameters[END_REF] inspected a method to recognize people using static body parameters like height of individual, distance between head and pelvis, distance between pelvis and left foot, and distance between right and left foot. Liu and Sarkar [START_REF] Liu | Simplest representation yet for gait recognition: Averaged silhouette[END_REF] proposed averaging of silhouette frames for determination of gait characteristics of an individual. The method then used Euclidean distance between averaged silhouettes for classification. Lee et al [START_REF] Lee | Shape variation-based frieze pattern for robust gait recognition[END_REF] proposed Shape Variated-based frieze pattern (SVBfrieze pattern) representation for gait. SVB-frieze pattern was formed by projecting difference frames onto 1 dimension, the projected values of difference frames were then summed over a gait cycle to obtain the frieze pattern. Nixon et al [START_REF] Yam | Model-based gait recognition[END_REF] [START_REF] Bouchrika | Model-based feature extraction for gait analysis and recognition[END_REF] presented automatic gait recognition involving estimation of rotation pattern of hips and modelling of leg movement using a pendulum model. K-nearest neighbor classifier was then used for classification.

Model free approaches involve period based methods. These methods provide compact representation of gait by extracting features from a single image obtained from one gait cycle. Chai et al. [START_REF] Chai | A new automatic gait recognition method based on the perceptual curve[END_REF] introduced Perceptual Shape Descriptor (PSD) that encompassed shape information of walking silhouettes. The formed PSDs were then accumulated over time to form Perceptual curve, which was used for gait recognition. Tan et al. [START_REF] Tan | Uniprojective features for gait recognition[END_REF] formed frieze pattern by projecting normalized silhouettes along four directions-vertical, horizontal and two diagonals. The resultant feature set was dimensionally reduced using PCA. A Mahalanobis distance based nearest neighbour method was then used for classification. Han and Bhanu introduced concept of Gait energy image (GEI) [START_REF] Man | Individual recognition using gait energy image[END_REF], which involved averaging of binary silhouette images over a gait cycle to form single image. GEI still remains one of the most popular technique for gait recognition and numerous works use GEI as underpinning. Tao et al. [START_REF] Tao | General tensor discriminant analysis and gabor features for gait recognition[END_REF] used Gabor functions over GEI for image understanding and employed a General Tensor Discriminant Analysis for recognition. A matrix based Marginal Fisher analysis (MFA) was presented by Xu et al. [START_REF] Xu | Marginal fisher analysis and its variants for human gait recognition and content-based image retrieval[END_REF] ,that produced a compact feature set, to address problem of gait recognition. Liu et al. [START_REF] Liu | Improved gait recognition by gait dynamics normalization[END_REF] modelled human gaits using population Hidden Markov Model (pHMM) and calculated distance between sums of normalized gait stances to identify individual identity. Wang et al. [START_REF] Wang | Human identification using temporal information preserving gait template[END_REF] introduced a novel template named Chrono-Gait Image (CGI). CGI encodes variation in averaged images into multiple channels thereby effectively preserving the temporal information of gait motion. One of the principal challenge in gait recognition is dealing with intra-class variations spawning from clothing variations and carrying conditions. Several techniques have been studied to reduce adverse effects of these variations. Bashir et al. [START_REF] Bashir | Gait representation using flow fields[END_REF] proposed Gait Entropy image (GEnI). GEnI calculates entropy for each pixel in GEI and then segments high temporal regions from regions of low temporality. The regions of high temporality are argued to be devoid of covariate conditions and thus are more relevant to gait recognition. Bashir et al. [START_REF] Bashir | Gait recognition without subject cooperation[END_REF] used optical flow fields to capture motion intensity and motion direction. Dupuis et al. [START_REF] Dupuis | Feature subset selection applied to model-free gait recognition[END_REF] selected relevant features from GEI by using Random Forest feature rank algorithm. Jeevan et al. [START_REF] Jeevan | Gait recognition based on gait pal and pal entropy image[END_REF] presented novel temporal template named Gait Pal and Pal Entropy (GPPE) Image. Kusakunniran [START_REF] Kusakunniran | Recognizing gaits on spatio-temporal feature domain[END_REF], [START_REF]Attribute-based learning for gait recognition using spatiotemporal interest points[END_REF] proposed Space Time interest points (STIPs) descriptor to extract gait features directly from raw video. Rokanujjaman et al. [START_REF] Rokanujjaman | Effective part-based gait identification using frequency-domain gait entropy features[END_REF] introduced a novel discrete fourier transform based entropy representation of gait named EnDFT. Lam et al. [START_REF] Lam | Gait flow image: A silhouettebased gait representation for human identification[END_REF] proposed Gait flow image(GFI) which represented motion information in a gait cycle by optical flow. Recently, Parul et al. proposed Gait Information image (GII) [START_REF] Arora | Gait based authentication using gait information image features[END_REF], a technique based on information set theory which involve features derived from Hanman-Anirban entropy function. GII have two featuresenergy feature (GII-EF) and sigmoid feature (GII-SF). Guan et al. [START_REF] Guan | On reducing the effect of covariate factors in gait recognition: a classifier ensemble method[END_REF] proposed a classifier ensemble method based on the random subspace method (RSM) and majority voting (MV). Very recently, Choudhury and Tjahjadi [START_REF] Choudhury | Robust view-invariant multiscale gait recognition[END_REF] introduced averaged gait key-phase image (AGKI) which use rotation forest ensemble learning technique to distinguish individuals and and recognize intra-class diversity, leading to good results even in presence of covariates.

III. PROPOSED COVARIATE CONSCIOUS FRAMEWORK

The covariate conscious gait recognition approach can be understood in two phases: Enrollment phase and Identification phase as outlined in Fig. 1.

In Enrollment phase, AESI is formed for each of the gallery gait sequence. The formed AESI is segmented into 4 parts according to human body geometry namely neck region, chest region, pelvic region and limb region. Each region is termed as part-AESI. Zernike moments are then calculated for each of the part-AESI and added in 'Zernike moments for gallery sequences' database. In parallel, sub-AESIs are formed by arranging part-AESIs of an incoming gait sequence into all possible combinations resulting in a total of 2 4 -1 = 15 sub-AESIs. Features are then extracted from each of the sub-AESI using SDOGs and MDPs methods. Finally, a linear SVM classifier is trained on the obtained feature set.

In Identification phase, AESI is formed from the probe gait sequence. Similar to enrollment phase, part-AESIs are constructed and Zernike moments are computed for each of the part-AESI. The presence of covariates is then decided by comparison of the computed moments of a part-AESI with the moments stored in 'Zernike moments for gallery sequences' database for the corresponding part. As decided by the comparison, parts devoid of any covariates are used to form a sub-AESI which is later classified using the corresponding SVM classifier trained during the enrollment phase. The extraction of binary silhouettes is performed using background subtraction technique. Background subtraction is a challenging task owing to the highly dynamic nature of the background, at the same time, quality of the extracted silhouettes is critical for the performance of the whole gait recognition algorithm. This makes effective modelling of background imperative. This work employs textual feature based background subtraction approach proposed by Chua et al. [START_REF] Chua | Adaptive texture-color based background subtraction for video surveillance[END_REF] for foreground segmentation. The approach reckons entropy as an important texture information parameter, which is defined as:

ε = i j F (i, j) log F (i, j) (1)
where F (i, j) = M (i, j)/ i,j M (i, j) is the probability density function and, i and j are indices to co-occurrence matrix M . Once the binary images are formed, the silhouettes are obtained by scanning the image row-wise and columnwise. The extreme white pixels are then determined to get the desired bounding box.

One key step in formation of AESI for gait sequence is determination of gait period. Gait period is defined as the number of frames over which two consecutive walking cycles span. One walking cycle is completed when a person moves from mid-stance position (position where legs are closest) to double-support position (legs are farthest apart) and then back to mid-stance position. The gait period can be computed simply by counting the number of foreground pixels in the lower half [START_REF] Sarkar | The humanid gait challenge problem: data sets, performance, and analysis[END_REF]. At the mid-stance position the number of foreground pixels attains a local minima, while at doublesupport position the number attains a local maxima. Therefore, the analysis as shown in Fig. 2 of these local minima and maxima can give the estimate of the gait period.

Once the gait period is determined, the binary silhouettes over that period are aligned using reference point and then fused into a single template on incremental basis [START_REF] Vishwakarma | Integrated approach for human action recognition using edge spatial distribution, direction pixel, and r -transform[END_REF]. One of the key strengths of AESI is the degree of compression it offers, an entire gait sequence, typically 40-70 silhouettes, is compressed into a single template. AESI fails to maintain the chronology thus loosing on some of the temporal information, however, it has adequate shape information thus making it apposite for the task of gait recognition. AESI is defined as:

AESI (x, y) = 1/n n τ =1 |S(x, y) τ | 2 (2)
where 'n is the number of silhouettes in a gait period and 'S(x, y) τ is the binary silhouette at time instant 'τ . The salient characteristic of AESI is that the degree of temporality in a region is manifested as intensity variation in AESI. Examples of generated AESIs for normal walking and covariate conditions is shown in Fig. 3.

B. Covariate detection using ZMIs

The presence of covariates can significantly alter the shape of human silhouettes. While even though this is one of the prime reason for drop in performance of gait recognition systems, this information could actually be leveraged in order to filter out the regions of the silhouettes affected by such distortion. In this paper, we use Zernike moment based shape descriptors for the purpose. Zernike Polynomials were proposed in [15] to describe optical aberrations. Zernike polynomials are a set of complex polynomials forming an orthogonal basis, which are defined in the interior of unit circle x 2 + y 2 ≤ 1. Zernike basis function V nm (ρ, θ) of order n and repetition m is defined in polar coordinates as:

V nm (x, y) = V nm (ρ, θ) = R nm (ρ) e jmθ for ρ ≤ 1 (3)
where 'R nm (ρ) is a radial polynomial defined as:

R nm (ρ) = n-|m| 2 s=0 (-1) s (n -s)! s! n+|m| 2 -s ! n-|m| 2 -s ! ρ n-2s (4)
V * nm (ρ, θ) V pq (ρ, θ) ρdρdθ = π n + 1 δ np δ mq ( 5
)
where δ xy = 1, x = y 0, otherwise The 2-dimensional Zernike moments for continuous image function ψ (ρ, θ) are defined as:

Z nm = n + 1 π 2π 0 1 0 ψ (ρ, θ) V * nm (ρ, θ) ρdρdθ = n + 1 π 2π 0 e -jmθ 1 0 ψ (ρ, θ) R nm (ρ) ρdρdθ (6)
The 2-dimensional Zernike moments for discrete digital image function ψ (x, y) are defined as:

Z nm = n + 1 π x y ψ (x, y) V * nm (x, y) , x 2 + y 2 ≤ 1 (7)
The incentive to using Zernike moments is that they are invariant to position, size and orientation. Teh et al. [START_REF] Teh | On image analysis by the methods of moments[END_REF] demonstrated the efficacy of Zernike moments as shape descriptors among other many moment-based descriptors. As stated before, the AESIs of both the gallery and probe gait sequences are partitioned into 4 segments called part-AESIs. This is done in order to restrict the extent of discarded region of AESI to a minimum, and thereby preserving viable information as much as possible. Since the AESIs are size normalized, the same segmentation scheme is followed for all the images. An AESI of height H is segmented based on human body anatomy [START_REF] Drillis | Body segment parameters: A survey of measurement techniques[END_REF] and scheme is given in Table I. Once the partition is done, Zernike moment invariants are calculated for each part-AESI formed for the gallery sequences, and are stored in 'Zernike moments for gallery sequences' database. To detect the presence of covariate in incoming part-AESI P i (i = 1, 2 . . . 4) of probe gait sequence, we compute mean Euclidean distance D P i between its Zernike moments and the moments stored in 'Zernike moments for gallery sequences' database for the corresponding part. In our empirical testing we found Zernike moments of order n = 5 and repetition m = 1, 3 and 5 to be efficient for the comparison. Fig. 4 shows plots of real and imaginary part of these moments. D P i is defined as:

D P i = 1 N N k=1 m = 1, 3, 5 n = 5 |Z Gki nm -Z P i nm | 2 ( 8
)
where N is the total number of gallery sequences and G Ki is i th part AESI of K th gallery sequence. A part-AESI is declared to infected with covariate if:

D P i ≥ µ i + 3σ i (9)
where:

µ i = 1 N 2 N j=1 N k = 1 k = j m = 1, 3, 5 n = 5 Z Gji nm -Z Gki nm 2 (10)
σ i = 1 N N j=1        N k = 1 k = j m = 1, 3, 5 n = 5 Z Gji nm -Z Gki nm 2 -µ i        (11
) It should be noted that computation of µ i and σ i is an offline process, and needs to be executed only once, when the database is constructed. This work considers the prevalent scenario where gallery is formed of normal walking sequences and probe sequences involve presence of covariates. However, the same proposed algorithm can be used in the hypothetical opposite scenario as well, since covariate detection works only on relative distance between probe and gallery sequences.

Fig. 5 demonstrates variation of Euclidean distance for infected and non-infected part-AESIs for two instances. It can be discerned from the figure that the mean Euclidean distance does serve as a suitable metric for the detection of covariates.

The added advantage of using ZNIs on AESI rather than on binary silhouettes themselves is AESIs immunity to noise. Since, AESI is formed by fusing multiple silhouettes; the effect of noise in any one silhouette becomes negligible and hence does not perniciously influence the performance of Zernike shape descriptors.

C. Computation of SDOGs shape feature

Spatial Distribution of Oriented Gradients (SDOG) [START_REF] Vishwakarma | Human activity recognition based on spatial distribution of gradients at sub-levels of average energy silhouette images[END_REF] extracts textural information from images by binning the intensity of image gradients according to their orientation in the image. These characteristics are computed at various decomposition levels (sub-regions) of the AESI, and hence encapsulate both coarse and fine details. The SDOGs computation for K orientation bins at decomposition level v results in a feature vector of length K v v=0 4 v . In this work, we apportion gradient orientations into 9 bins each spanning 20 degrees and a total of 3 decomposition levels are employed. The process to compute SDOG is delineated in Algorithm1 and Fig. 6.

Algorithm 1: Computation of SDOGs

Input : AESI (x, y) of gait sequence computed using Eq.2. 1 Decompose AESI (x, y) into sub-regions. 2 Determine the magnitude and orientation at each pixel of gradient image Grad (x, y) at multiple decomposition levels (v) using

H (x, y) = Grad x (x, y) 2 + Grad y (x, y) 2 and
(x, y) = tan -1 Grady(x,y) Gradx(x,y) . 3 Apportion the computed magnitude of gradients at different levels (v) into Korientation bins. 4 Aggregate all the sub-regions at level (v)

f SDOGs = K v v=0 4 v
5 Concatenate all the levels (0, 1, 2) of SDOGs to form a final SDOGs feature vector defined as:

f SDOGs = K      0 v=0 4 v level v=0
;

1 v=0 4 v level v=1 ; 2 v=0 4 v level v=2     
Output: The histogram of SDOGs feature vector of dimension 1 × 189.

D. MDPs calculation

Human gait motion is attributed more to horizontal motion than vertical motion [START_REF] Foster | Automatic gait recognition using area-based metrics[END_REF]. To exploit this, we introduce Mean of Directional Pixels (MDPs) features, and append these with SDOGs to form the final feature set. MDPs treat each row of AESI as an individual feature element, and compute the mean of pixel intensity in individual rows. To keep the dimension of the resulting feature set low, only the rows of AESI which belong to high temporal regions, are included in the computation. High temporal rows in AESI can be identified from the variance of intensity values in that particular row.

For AESI of dimension m × n, M DP (i) for row vector i is defined as:

M DP (i) = 1 n n k=1 AESI (i, k) (12)
In this work, only m/2 MDPs corresponding to most intensity variable rows, are calculated for a m × n AESI.

It should be noted that since the number of rows vary for different part-AESIs, the length of resulting feature set varies as well. Fig. 7 demonstrates the variation of MDPs for two subjects with two sequences each. It can be studied from the figure that MDPs possess remarkable inter-class distinguishing ability while at the same time have the needed intra-class similarity.

IV. EXPERIMENTAL RESULTS

In order to test the performance of proposed framework, an experiment is conducted on three publicly available human gait dataset sets. These datasets are CASIA Dataset B [START_REF] Yu | A framework for evaluating the effect of view angle, clothing and carrying condition on gait recognition[END_REF], OU-ISIR Treadmill Dataset B [START_REF] Makihara | The ou-isir gait database comprising the treadmill dataset[END_REF] and the USF Human-ID gait challenge Dataset [START_REF] Sarkar | The humanid gait challenge problem: data sets, performance, and analysis[END_REF]. For classification, Linear kernel SVMs are used with penalty parameter c = 1 and tolerance for stopping criteria = 1×e -4 . The computational complexity of the proposed algorithm in Identification phase is Ω(mn + N ), while for Enrollment phase( i.e construction of Database), the computational complexity is Ω(mnN + N 2 ) where m × n is the size of AESI in pixels and N is the number of members enrolled\to be enrolled in the dataset. Performance of the algorithm is measured by calculating the correct class recognition rate (CCR) in percentage using support vector machine (SVM) in leave-one-out cross validation scheme. The CCR is determined using Eq. 14 and defined as:

CCR = T P + T N T P + T N + F P + F N
× 100 (In P ercentage) (13) where T P, T N, F P and F N are true positive, true negative, false positive, and false negative respectively. To evaluate the effectiveness of the proposed algorithm, the highest CCR achieved through the proposed algorithm is compared with the others similar state-of-the-art. The highest CCR achieved in each probe of the dataset is compared with the others similar state-of-the-art and as presented in Table II and Fig. 9. Table II, compares the performance of the proposed method with various existing algorithms in terms of average CCR for three different probe of CASIA dataset. From Table II, it can be observed that the proposed algorithm performs admirably well in presence of covariates (i.e. CAS-B and CAS-C), where it achieves 93.1% accuracy for sequences having carrying conditions (CAS-B). This is 9.5% better than the existing stateof-the-art, Flow Field which achieved recognition accuracy of 83.6%. For sequences having clothing variations (CAS-C), difference is even more drastic. Our method achieved accuracy of 81.3%. This is 17.6% better than the existing Random forest approach, which achieved recognition accuracy of 63.7%. Fig. 9 show Cumulative Match Characteristic (CMC) curve for all three probe sequences, and it can be considered an exhaustive comparison of accuracy because it is achieved through the variation of Rank. In all these cases the CMC curves shows better accuracy. Hence, it can be said the proposed algorithms perform well in comparison with many such existing approaches. Table III studies The Table IV compares the rank 1 and rank 5 performance of the proposed method with the similar state of the art. Fig. 11 shows the comparison of CMC on similar state-of-theart methods and higher accuracy can be comprehend by the proposed approach AESI+ZNK. The results are depicted using CMC curve in Fig. 12. shows the CMC curve for all the probe It is clear from Table IV that the proposed approach outperforms existing methods in robustness to distortion by presence of covariates. The recognition accuracy achieved by the proposed algorithm is 72.7%, which is 11.5% better than the existing state-of-the-art, GII-SF that achieved 61.2% recognition accuracy.

A. CASIA Dataset B

From Fig. 12 it can be seen that the recognition accuracy achieved is highest for probe sequences A, X and Y. This is understandable since these sequences have less drastic variations in clothing compared to gallery set. On the contrary, the proposed algorithm fared worst for sequences V and H. This can be attributed to minimal commonality between these sequences and the gallery set.

C. USF Human-ID Gait Challenge Dataset

Finally, the performance of the proposed work is evaluated on USF Human-ID dataset [START_REF] Yu | A framework for evaluating the effect of view angle, clothing and carrying condition on gait recognition[END_REF]. The dataset is shot in outdoor environment (Fig. 13) and comprises of five variations-concrete or grass surface, presence or absence of briefcase, shoe variation (Type A or Type B), right or left view and variation in elapse time. The USF Human-ID gait challenge dataset contains a total of 1870 gait sequences of 122 human subjects. The dataset provides a predetermined experimental setup and is segmented into 13 subsets out of which 1 forms the gallery set and the remaining 12 form the probe sets (A-L). As in the case of OU-ISIR dataset, the USF Human-ID dataset also provides size-normalized silhouettes of 128 × 88 pixels. Therefore, the formed AESIs are of the same size i.e. 128 × 88 pixels.

Table V informs about the covariate variations in all the probe sets (A-L) along with the number of sequences in each set. The table also demonstrates the achieved overall accuracy of the proposed algorithm and similar state-of-the-art for individual probe sets.

It can be studied that the proposed algorithm achieved an overall CCR of 72.53% for Rank 1 evaluation. This is highest among all the compared methods. For Rank 5 evaluation, the proposed method achieved CCR of 84.67%, which again is highest among all the existing methods. This proves the efficacy of our approach. The achieved CCR is particularly high for probe sets A, C, D, H and J. However, the approach doesnt offer improved performance for probe sets K and L over the existing state-of-the-art. This shows that while the use of Zernike moments can effectively handle and iron out covariates affecting the shape information, it understandably doesnt offer much immunity from temporal covariates.

V. CONCLUSION AND FUTURE WORK

In this paper, a covariate cognizant approach to gait recognition has been presented. A single spatio-temporal template called AESI is formed to represent the gait motion. After which, Zernike moment based shape descriptors are used to detect the presence of covariates. An effective segmentation technique is used which discards the parts of AESI deemed infected by covariates. For feature extraction, SDOGs and MDPs methods are employed. SDOGs method provides a reliable and compact feature set by exploiting gradient information in the AESI. MDPs is a new feature extraction method, adapted specifically for gait motion and extracts information about the horizontal movement, which characterize the gait sequence. The efficacy of the presented approach is evaluated on three publicly available datasets. On all the three datasets our approach showed promising results and outperformed the existing state-of-the-art.

For future studies, an extension of the proposed approach, which is multi-view invariant, can be developed. We will also explore how to improve the representation of temporal information for gait sequences. In addition, we will consider employing the presented approach to the domain of medical prognosis. Since many ailments causes substantial change in walking pattern of an individual, it would be interesting to explore if the technique could help in diagnosis of such ailments.

Fig. 1 :

 1 Fig. 1: Work flow diagram of the proposed framework

Fig. 2 :

 2 Fig. 2: Determination of gait period from variation of foreground pixels

Fig. 3 :

 3 Fig. 3: Formed AESIs (a) Normal Walking (b) Clothing and carrying conditions

Fig. 4 :

 4 Fig. 4: Plot of Zernike Basis function Vnm(p, θ) for order n = 5 (a) Re(V 5,1 ) (b) Im(V 5,1 ) (c) Re(V 5,3 ) (d) Im(V 5,3 ) (e) Re(V 5,5 ) (f) Im(V 5,5 )

Fig. 5 :

 5 Fig. 5: Variation of Euclidean distance between Zernike moments in presence and absence of covariates for two instances (a) and (b). From figure it can be seen that the mean of Euclidean distance does serve as a reliable metric to detect the presence of covariates

Fig. 6 :

 6 Fig. 6: Representation of decomposition levels and their spatial distribution, involved in calculation of final SDOG feature vector

Fig. 7 :

 7 Fig. 7: Variation of MDPs for two subjects. The figure illustrates the intra-class similarity and inter-class distinguishing ability possessed by MDPs.

Fig. 8 :

 8 Fig. 8: Probe sequences of CASIA B dataset

  the variation of recognition rate with different number of training data. K is the number of gait normal walking sequences involved to form the training data. The remaining sequences form the testing set. B. OU-ISIR Treadmill Dataset B OU-ISIR Treadmill B [18] dataset comprises of large clothing variations. The dataset consists of 68 subjects with up to 32 possible variations in clothing from 15 different kinds of clothes such as half shirt, full shirt, baggy pants, hat, cap etc. Fig. 10 shows different gait sequences in the dataset. Table III show clothing combinations for each of the gait sequence. The dataset is partitioned into 3 segments-Gallery set, Training set and Probe set. The gallery set comprises of 48 subjects with one standard combination of clothing (Sequence type 9). The training set consists of 446 sequences of 20 subjects with all possible clothing combinations. The set is provided for tuning of model parameters. Since, its unlikely in real world to know the type of covariate variations beforehand, the training set is not used in this evaluation in any manner. The probe set comprises of 856 sequences of the same 48 subjects as in gallery set with all possible clothing combinations, except type 9. The OU-ISIR dataset itself provide extracted silhouettes size normalized to 128 × 88 pixels, therefore the formed AESIs are of the same size (128 × 88).

Fig. 9 :

 9 Fig. 9: Depiction of accuracy vs Rank curve for different approaches on three probe sequences of dataset (a) CAS-A, (b) CAS-B (c) CAS-C

Fig. 10 :

 10 Fig. 10: Gait sequences in OU-ISIR Treadmill Dataset B

Fig. 11 :

 11 Fig. 11: Comparison using CMC on OU-ISIR Treadmill Dataset B

Fig. 12 :

 12 Fig. 12: Comparison of performance of proposed algorithm for individual sequences of OU-ISIR Treadmill B Dataset using CMC TABLE VI: COMPARISON OF CCR WITH THE SIMILAR STATE-OF THE-ART ON USF HUMAN-ID GAIT DATASET AT RANK 1 AND RANK 5. KEY FOR COVARIATES -V -VIEW; H -SHOE; S -SURFACE; B -BRIEFCASE; T -TIME; AND C -CLOTHES Probe Set A B C D E F G H I J K L Avg.

Fig. 13 :

 13 Fig. 13: Sample images of USF Human-ID gait challenge dataset: (a, b) left and right of camera view on concrete surface (c, d) left and right camera view on grass

TABLE I

 I

	:
	AESI SEGMENTATION
	Body Part	Height
	Neck region	1H -0.80H
	Chest region	0.80H -0.55H
	Pelvic region	0.55H -0.30H
	Limb region	0.30H -0H

TABLE II :

 II COMPARISON OF CCR WITH THE SIMILAR STATE-OF-THE-ART ON CASIA DATASET B

	Method	CAS-A CAS-B CAS-C Average
	GEI [6]	93.1	48.8	18.8	53.56
	GEnI [9]	92.6	57.3	22	57.3
	DFT [40]	95.1	58.8	21.5	58.46
	EnDFT [33]	96	60.4	39.2	65.2
	CGI [7]	100	64.2	43.6	69.26
	Flow Field [29]	97.5	83.6	48.8	76.63
	Random Forest [29]	98.8	73.8	63.7	78.76
	GPPE [30]	93.6	56.1	22.4	57.36
	STIP [32]	95.4	60.9	52	69.43
	GII-SF [19]	98	74.5	45	72.5
	AESI+ZNK	100	93.1	81.3	91.47

TABLE III :

 III PERFORMANCE ON CASIA B DATASET USING DIFFERENT NUMBER OF TRAINING DATA

		K=1	K=2	K=3	K=4	K=5
	CAS-A	89.04	96.22	99.17	100	100
	CAS-B	87.52	91.14	92.36	93.1	93.85
	CAS-C	75.81	79.67	81.3	81.3	81.3

TABLE IV :

 IV CLOTHING COMBINATIONS INCLUDED IN THE OU-ISIR B DATASET. KEYS FOR CLOTHES: RP-REGULAR PANTS; BP-BAGGY PANTS; SP-SHORT PANTS; SK-SKIRT; CP: CASUAL PANTS; HS-HALF SHIRT; FS-FULL SHIRT; LC-LONG COAT; PK-PARKER; DJ-DOWN JACKET; CW-C

	Type	S1	S2	S3	Type	S1	S2	Type	S1	S2
	3	RP HS	Ht	0	CP CW	F	CP	FS
	4	RP HS	Cs	2	RP	HS	G	CP	Pk
	6	RP LC Mf	5	RP	LC	H	CP	DJ
	7	RP LC	Ht	9	RP	FS	I	BP	HS
	8	RP LC	Cs	A	RP	Pk	J	BP	LC
	C	RP	DJ	Mf	B	RP	DJ	K	BP	FS
	X	RP	FS	Ht	D	CP	HS	L	BP	Pk
	Y	RP	FS	Cs	E	CP	LC	M	BP	DJ
	N	SP	HS	-	P	SP	Pk	R	RC	-
	S	Sk	HS	-	T	Sk	FS	U	Sk	PK
	V	Sk	DJ	-	Z	SP	FS	-	-	-
	TABLE V: COMPARISON OF CCR WITH THE SIMILAR STATE-OF
		THE-ART ON OU-ISIR TREADMILL DATASET B
		Method			Rank 1 CCR		Rank 5 CCR
		GEI [6]			44.2				71.2
		GEnI [9]			47.3				75.6
	DFT [37]			46.6				73.2
	EnDFT [32]		52				77.4
	GII-SF [18]		61.2				85.1
	AESI+ZNK		72.7				89.7