Frank Breitinger

Christian Winter

York Yannikos

Tobias Fink

Michael Seefried

USING APPROXIMATE MATCHING TO REDUCE THE VOLUME OF DIGITAL DATA

Keywords: File identification, approximate matching, ssdeep

Digital forensic investigators frequently have to search for relevant files in massive digital corpora -a task often compared to finding a needle in a haystack. To address this challenge, investigators typically apply cryptographic hash functions to identify known files. However, cryptographic hashing only allows the detection of files that exactly match the known file hash values or fingerprints. This paper demonstrates the benefits of using approximate matching to locate relevant files. The experiments described in this paper used three test images of Windows XP, Windows 7 and Ubuntu 12.04 systems to evaluate fingerprint-based comparisons. The results reveal that approximate matching can improve file identification -in one case, increasing the identification rate from 1.82% to 23.76%.

Introduction

Traditional physical media such as books, photos, letters and longplaying records (LPs) have been replaced by digital media in the form of e-books, digital photos, email and MP3 files. The result is that forensic investigators are faced with overwhelming amounts of digital data even in routine cases. Investigators need automated processes that can quickly classify and filter terabytes of data, yielding only the relevant files that can then be inspected manually.

A common approach is to employ cryptographic hash functions to automatically classify files: an investigator computes the hash values of all the files residing on digital media and compares these fingerprints against file hash values stored in a reference database. A matching hash confirms that the referenced file is present in the storage media. Depending on the hash values stored in the reference database, the files can be classified as relevant (blacklisting) and non-relevant (whitelisting). Files not found in the database are not classified.

White [START_REF] White | Hashing of file blocks: When exact matches are not useful[END_REF] and Baier and Dichtelmuller [START_REF] Baier | Datenreduktion mittels kryptographischer Hashfunktionen in der IT-Forensik: Nur ein Mythos? DACH Security[END_REF] have analyzed the impact of using cryptographic hash functions for file classification in digital forensic investigations. White obtained identification rates as high as 70%. However, Baier and Dichtelmuller measured identification rates between 15% and 52%; these lower identification rates were due to small changes in files that typically occur during computer use.

As a consequence, the digital forensic research community has proposed the use of approximate matching that maps similar files to similar hash values. This paper shows that approximate matching can increase the file identification rate. In one case, the identification rate increased from 1.82% with traditional cryptographic hash value comparisons to 23.76% with approximate matching.

The principal contribution of this paper is the quantitative evaluation of identification rates for approximate matching on complete disk images. While reference results are available for cryptographic hashing [START_REF] Baier | Datenreduktion mittels kryptographischer Hashfunktionen in der IT-Forensik: Nur ein Mythos? DACH Security[END_REF][START_REF] White | Hashing of file blocks: When exact matches are not useful[END_REF], approximate matching has not been evaluated on such a large scale. Also, the paper analyzes the error rates for approximate matching and establishes similarity score thresholds for use in real-world digital forensic investigations.

Background

Two strategies exist for filtering known content: (i) blacklisting, which compares a file to a reference database containing the fingerprints of illegal and suspicious files (e.g., child pornography) that should be retained for further analysis; and (ii) whitelisting, which compares a file to a reference database containing fingerprints of innocuous files (e.g., operating system files) that should be eliminated from consideration.

The most prominent reference database is the National Software Reference Library (NSRL) [11] maintained by the U.S. National Institute of Standards and Technology (NIST). NIST regularly publishes reference data sets containing the cryptographic hashes of software products. The current reference data set, RDS 2.42 of September 2013, covers approximately 115 million files.

Known file filtering predominantly utilizes cryptographic hash functions. However, the security properties of cryptographic hash functions imply that only identical files can be matched using these hash func-tions -a difference in just one bit produces a completely different hash value. Although this property is desired for cryptographic purposes, it complicates forensic investigations. For example, an investigator is often interested in locating similar files, file fragments and embedded objects. Therefore, it is helpful to have algorithms that provide approximate matches to correlate related versions of data objects.

Approximate matching techniques find matches based on bytewise similarity or semantic similarity. Bytewise matching operates at the byte level: two inputs are similar if they have similar byte structures. Semantic matching, also called perceptual hashing or robust hashing, attempts to understand the input format and is, therefore, bound to specific media types, images or movies. While a semantic hashing solution is highly domain specific, bytewise approximate matching has the advantage of generality.

This paper investigates bytewise approximate matching, in particular, using the ssdeep tool [START_REF] Kornblum | Identifying almost identical files using context triggered piecewise hashing[END_REF]10]. This was motivated by the prominence of ssdeep, the fact that NIST has published a reference set of ssdeep hashes, and the ability to process large file volumes in reasonable time using the F2S2 tool [START_REF] Winter | F2S2: Fast forensic similarity search through indexing piecewise hash signatures[END_REF].

It is important to note that the concept of approximate matching should not be confused with locality sensitive hashing. Approximate matching reduces large files to small digests such that similar files are mapped to similar digests. On the other hand, locality sensitive hashing is an indexing strategy, which places items into hash buckets such that similar items have a high probability of being in the same bucket. Hence, locality sensitive hashing could speed up approximate matching depending on the matching approach and the locality sensitive hashing method that are employed.

ssdeep

Context triggered piecewise hashing was proposed by Kornblum [START_REF] Kornblum | Identifying almost identical files using context triggered piecewise hashing[END_REF]10] and implemented in the ssdeep tool. It originates from the spam detection algorithm of Tridgell [START_REF] Tridgell | [END_REF] implemented in spamsum.

The ssdeep tool divides a byte sequence (file) into chunks and hashes each chunk separately using the Fowler-Noll-Vo (FNV) algorithm [START_REF] Noll | FNV hash[END_REF]. Context triggered piecewise hashing then encodes the six least significant bits of each FNV hash as a Base64 character. All the characters are concatenated to create the file fingerprint.

The trigger points for splitting a file into chunks are determined by a rolling hash function. This function, which is a variation of the Adler-32 algorithm [START_REF] Deutsch | ZLIB Compressed Data Format Specification Version 3.3[END_REF], is computed over a seven-byte sliding window to generate a sequence of pseudorandom numbers. A number r in the sequence triggers a chunk boundary if r ≡ -1 (mod b). The modulus b, called the block size, correlates with the file size.

Kornblum suggests dividing a file into approximately S = 64 chunks and using the same modulus b for similar-sized files. The modulus b is a saltus function:

b = b min • 2 ⌊log 2 (N/S/b min )⌋ (1)
where b min = 3 and N is the input length in bytes. Since two fingerprints can only be compared if they were generated using blocks of the same size, ssdeep calculates two fingerprints for each file using the block sizes b and 2b and stores both fingerprints in one ssdeep hash. The similarity of two fingerprints is calculated in two steps. First, the fingerprints are treated as text strings and compared with each other using an edit distance function. An edit distance of zero indicates matching strings and the distance increases with the dissimilarity of strings. In the second step, the computed distance of the two fingerprints is converted into a similarity score in the range 0 to 100, where a higher score indicates greater similarity.

The ssdeep similarity measure defines a score boundary for small fingerprints. Hence, small files that are identical often do not receive a score of 100, although this is expected. The score boundary is enforced in lines 600 through 604 of fuzzy.c (ssdeep version 2.10). Tridgell's comment in the source code says that this is done so as not to "exaggerate the match size" for a small block size. However, we eliminated this constraint in our research because the score boundary introduces uncertainty when interpreting the match results.

While ssdeep is a pioneer in the domain of approximate matching, several improvements to ssdeep hashing have been proposed in the literature [START_REF] Chen | An efficient piecewise hashing method for computer forensics[END_REF][START_REF] Seo | Detecting similar files based on hash and statistical analysis for digital forensic investigations[END_REF].

F2S2

The task of comparing files to a reference list can be very time consuming. In particular, the comparison of similarity digests cannot be performed as efficiently as the exact matching of cryptographic hashes. Approximate matching tools -including ssdeep -typically compare every hash computed for the investigation target to every hash in the reference list.

Unlike exact matching, approximate matching cannot be accelerated using classical indexing strategies developed for relational databases. However, it is possible to find suitable indexing strategies. The F2S2 tool [START_REF] Winter | F2S2: Fast forensic similarity search through indexing piecewise hash signatures[END_REF] implements one such solution that involves n-gram indexing.

F2S2 has been applied to ssdeep hashes and has achieved a speedup of 2,000 times in realistic test cases.

Experimental Methodology

The experiments were performed on an Ubuntu 12.04 host using ssdeep 2.7-1. The drive images came from computers running the Windows and Linux operating systems. Since Windows is the most commonly used operating system, the experiments analyzed two Windows versions: Windows XP (Professional, 32 bit) and Windows 7 (Professional, 32 bit). The experiments also analyzed the Linux Ubuntu operating system (version 12.4, 32 bit).

Overview

The experiments focused on determining the appropriateness of SHA-1 and ssdeep for known file identification with approximate matching. The detection rates with respect to true positives and false positives for cryptographic hashing (SHA-1) and approximate matching (ssdeep) were examined. Note that the assessment only considered positive errors. Some large files, such as hyberfil.sys (Windows XP), device files and named pipes (Ubuntu), did not generate ssdeep hashes. Since the algorithm was considered to be a blackbox, these problems were treated as bugs and were not investigated further.

Hash values could not be generated for a very small number of files. When a file hash value could not be generated, the file was treated as if it did not exist. In the worst case (Ubuntu U ), only 0.12% of the files had to be dropped.

System Description and Snapshots

The Windows XP samples came from two snapshots of an extensivelyused system, which were 14 and 27 months old, respectively, when the snapshots were taken (the age of a system is defined as the time elapsed between system installation and snapshot generation). The Windows XP system was used on a daily basis for office work; it contained software installations, system and software update artifacts, user created files and other software usage traces. The Ubuntu system was about six months old when the snapshot was taken. This snapshot included system updates, and system development and web browsing artifacts. The Windows 7 system was used for half a day; during that time, the latest updates were applied, some files were created and some web pages were visited. The trusted reference snapshots were created from default installations of the operating systems (i.e., without any updates, service packs or additional programs).

Table 1 provides information about the operating system snapshots. Note that WinXP U 1 used more disk space than WinXP U 2 because an 82 GB image file was deleted between the times that the two snapshots were taken.

Reference Databases

The following reference databases were used to determine the identification rates for each operating system image:

NIST Database: This NIST-published database (RDS 2.27 of December 2009) contains more than eight million entries. The hashes are provided in the form of a text file with one SHA-1 hash and one ssdeep hash per line. The SHA-1 hash is used to link each entry to the regular NIST RDS. The experiments used the SHA-1 and ssdeep hashes from the NIST database.

Custom Reference Databases: Because the NIST database above is an outdated subset of the current RDS 2.42 (September 2013) and no newer ssdeep dataset was available, we created our own reference sets based on the default installations of the operating systems. The procedure used to create the reference sets was essentially the same as that used to create the NIST database.

Quality of Matches

The matches reported by ssdeep correspond to the number of positive collisions. The collisions require truth data in order to be classified as true positives and false positives. Because there were too many files to perform a manual inspection, the true positives were considered to be the files that matched the SHA-1 hashes. To perform approximate matching, a threshold t was set to yield a match score M such that M ≥ t indicates a positive match (i.e., similar files). However, identifying an appropriate threshold t was challenging because a low t value increases the false positive rate while a high t value decreases the true positive rate.

Roussev [START_REF] Roussev | An evaluation of forensic similarity hashes[END_REF] has studied the ratio between true and false positives based on the comparison of 4,457 files (= 9,930,196 pairs). His results are presented in Figure 1. Accordingly, we set the thresholds to t = 60 and t = 40. Although the t = 60 threshold yielded a few false positives, we rated the false positive rate of the t = 40 threshold as acceptable.

Experimental Results

This section presents the experimental results. First, the difference between SHA-1 matches and ssdeep matches is examined. Next, identification rates are compared for the various images and databases. Following this, the identification rates are presented in correlation with file names and paths. Finally, the relationship between identification rate and file type is clarified.

Seemingly Identical Files

Minor differences exist between a SHA-1 match and an ssdeep score of 100. A SHA-1 match implies that two files are identical with extremely The score matching problem is due to the similarity digest comparison method used by ssdeep. The method requires that the two digests being compared have a common substring of length seven, otherwise the score is simply set to zero [START_REF] Breitinger | Security aspects of piecewise hashing in computer forensics[END_REF]. Note also that ssdeep assigns a score of zero to two empty files whereas SHA-1 matches the two files.

An ssdeep score of 100 can be obtained even when the SHA-1 hashes are different; this is due to fingerprint collisions and the comparison method. Note that each ssdeep digest consists of two fingerprints and it is sufficient to have one matching pair to obtain a score of 100. Thus, there are two types of collisions for ssdeep. In some cases, the ssdeep hashes match completely; in other cases, only one pair of ssdeep fingerprints matches. We consider a similarity score 100 for two non-identical files to be a false positive.

Detection Thresholds

We also analyzed the detection rates based on the thresholds t = 40 and t = 60. When presenting the results, note that D denotes the analyzed system and |D| denotes the number of files in the system. Also, I A is the identification rate using algorithm A ∈ {ssdeep, SHA-1}. For example, I ssdeep = 10 % means that 10 % of the files in system D were found in the database using ssdeep. Thus, the higher the value of I, the greater the number of files that are identified automatically.

Test Case 1 (TC1): This test case compared the files in the default operating system installations with the files represented in the NIST database. The results are shown in Table 2. The identification rates for the Windows XP system are significantly higher compared with those for the other operating systems; this because the underlying database was created in December 2009 when Windows XP was very popular. Still, the trend is obvious, regardless of the operating system, the identification rate is much higher for ssdeep (e.g., nearly ten times better for Win7 with the threshold t = 40).

Test Case 2 (TC2): This test case compared the files in the used operating system installations with the files represented in the NIST database. The results in Table 3 are comparable with those obtained in Test Case 1 (Figure 2). Once again, the Windows XP systems have the best identification rates due to the underlying database. However, the identification rates are smaller than in the previous test case because the systems contained large numbers of files. For example, WinXP U 2 had in excess of 50 times more files than WinXP D . Still, the identification rates are approximately two to ten times higher for ssdeep with threshold t = 60 compared with SHA-1. Test Case 3 (TC3): This test case compared the files in the used installations against those in the default installations. The main results are shown in Table 4. The low rates for the Windows XP systems are a consequence of the "small" default installation. Recall that the default installation had only about 9,000 files. How- ever, some files in the default installation are similar or identical to files in the used installation. Examples are the desktop.ini files, DLLs and file shortcuts (.lnk). Typical locations for such files in the default installation are WINDOWS/system32/config/ systemprofile, WINDOWS/pchealth/helpctr/System and WIND OWS/system32/dllcache. Hence, it was possible to identify more files in the used system than were present in the default system.

The high identification rates for the Windows 7 system reflect the fact that the system was not used very much. Also, the updates introduced many files that are similar to the files found in the default system.

Test Case 4 (TC4): This test case compared the files in the two Windows XP snapshots where WinXP U 1 emulates the database.

The main results are shown in Table 5. Despite the difference of 13 months between the two snapshots, identification rates are 24% for SHA-1 and 32% for ssdeep, a difference of 8%. The percentage may seem small, but 8% of the 466,266 files in the system corresponds to approximately 37,000 files.

The experiments confirmed that the detection rates for ssdeep are higher than for SHA-1 in all four test cases. This is especially relevant for blacklisting (files that are similar to suspicious files could constitute evidence). Also, as shown in the comprehensive study by Roussev [START_REF] Roussev | An evaluation of forensic similarity hashes[END_REF], the thresholds used in the experiments are reasonable and have acceptable false positive rates.

File Names and Paths

This section analyzes the reliability of positive matches obtained using the two thresholds. Recall that the number of approximate matches is the total number of ssdeep matches minus the number of exact SHA-1 matches. Table 6 shows the results. The first column specifies the comparisons, e.g., WinXP U1 versus WinXP U2. The second and third columns list the numbers of files in the systems (|D|) and the numbers of matches with scores greater than or equal to t (ssdeep Hits), respec- tively. The fourth column shows the benefit of ssdeep, i.e., matches that are not identified by SHA-1 (relative to |D|) -this is the "critical amount" of files. The last two columns list the file path and name matches relative to the numbers of files excluding the SHA-1 matches. Consider, for example, the last row of Figure 6, which examines the default Ubuntu installation against the used Ubuntu installation that contains 411,209 files. When t = 60, ssdeep detects 53.38% as similar files. Reducing this figure by the number of SHA-1 matches yields 5.71% (i.e., 23,480 files). 22.91% of these files have the same path and 85.89% have the same file name. Hence, these files are considered to be true positives. The remaining 100% -85.89% = 14.11% files are either false positives or files that have been moved or renamed. A total of 3,313 files remain and these files need to be analyzed manually.

File Types

While approximate matching at the syntactic level can be applied to any file, it is not useful for all file types. This depends on the file type and the type of modification -whether small modifications preserve most of the binary content of a file or lead to a completely different binary pattern. For example, text is favorable while compressed data causes problems. The reason is that small text modifications result in small changes to the binary data of a text file, but compression algorithms create very different byte sequences given similar inputs. Thus, approximate matching at the raw byte level cannot discern the real similarity when compression algorithms have been used.

The final experiment sought to identify the file types for which ssdeep is better than cryptographic hashing. The test considered only ssdeep scores between 60 and 99. Table 7 shows the ten most frequent file types with scores between 60 and 99 for each operating system based on Test Case 2. The numbers correspond to the file percentages compared with Table 7. File types with high identification rates for non-identical files.

WinXPU2

Win7U UbuntuU Type Amount Type Amount Type Amount all the files identified within the score range. The file types that include text are marked with an asterisk; they constitute the majority of the listed files. However, binary file types (e.g., .dll in Table 7) also have high identification rates. Other results (not shown in this paper) reveal that .exe, .pyc and .so are also among the top ten file types. Note that files without suffixes are typically text files or binary executables.

Related Work

duction leaves an average of 30% of disk space for human investigation and incorporating block hashes reduces the amount of data for human review to 15% of disk space. However, White focused on basic installations, which is not a realistic scenario. Baier and Dichtelmuller [START_REF] Baier | Datenreduktion mittels kryptographischer Hashfunktionen in der IT-Forensik: Nur ein Mythos? DACH Security[END_REF] showed that the reduction rates for used workstations were much worse than those obtained by White. In particular, Baier and Dichtelmuller obtained rates of approximately 50% for a basic Windows XP installation and rates as low as 15% for used Windows 7 and Ubuntu systems.

In both cases, Baier and Dichtelmuller used the latest reference data set that contained many more entries than the reduced NIST database employed in this research.

Conclusions

This paper describes the first experimental evaluation of approximate matching with large test cases. The results show that approximate matching provides substantial benefits compared with cryptographic hashing. Approximate matching significantly increases the number of files identified as known files. Also, it reduces the number of files that have to be inspected manually by digital forensic investigators. Text files and binary files can be filtered effectively using approximate matching at the syntactic level. However, it is important that an up-to-date reference database is used to obtain the best identification rates.

Better results than ssdeep can be obtained using a more accurate approximate matching algorithm such as sdhash. However, an efficient similarity comparison method must be devised before sdhash can be evaluated using large test cases.

Figure 1 .

 1 Figure 1. Distribution of ssdeep scores according to Roussev [14].

Table 1 .

 1 Operating system snapshots.

	Operating System	File Count Disk Usage
	WinXPD (Windows XP, Default installation)	8,946	1.9 GB
	WinXPU1 (Windows XP SP3, 14 months old)	195,186	128.4 GB
	WinXPU2 (Windows XP SP3, 27 months old)	466,266	109.5 GB
	Win7D (Windows 7, Default installation)	45,470	8.1 GB
	Win7U (Windows 7 SP1, Used installation)	66,312	9.4 GB
	UbuntuD (v12.04, Default installation)	185,468	3.3 GB
	UbuntuU (v12.04, Used installation)	411,209	25.2 GB

Table 2 .

 2 TC1: Comparing default installations against the NIST database.

	D	|D|	I ssdeep t = 40	I ssdeep t = 60 t = 100 I ssdeep	ISHA-1
	WinXPD	8,946	68.69 % 63.85 %	35.84 % 35.24 %
	Win7D	45,470	16.59 %	9.32 %	1.73 %	1.70 %
	UbuntuD 185,468 16.14 %	9.54 %	1.90 %	1.95 %
	high certainty. In contrast, an ssdeep score of 100 does not necessarily
	mean that the two files are identical. Additionally, identical ssdeep
	hashes may not receive a score of 100 despite the fact that the small file
	score boundary was removed.

Table 3 .

 3 TC2: Comparing used installations against the NIST database.

	D	|D|	I ssdeep t = 40	I ssdeep t = 60 t = 100 I ssdeep ISHA-1
	WinXPU1 195,186 17.79 % 14.70 %	7.69 % 8.03 %
	WinXPU2 466,266 23.02 % 17.39 %	7.05 % 7.30 %
	Win7U	66,312	17.88 % 10.21 %	1.45 % 1.44 %
	UbuntuU	411,209 23.76 % 17.11 %	1.79 % 1.82 %

Table 4 .

 4 TC3: Comparing used installations against default installations.

	D	|D|	I ssdeep t = 40	I ssdeep t = 60 t = 100 I ssdeep	ISHA-1
	WinXPU1 195,186	5.12 %	4.71 %	4.13 %	4.14 %
	WinXPU2 466,266	2.27 %	2.01 %	1.72 %	1.74 %
	Win7U	66,312 93.85 % 90.19 %	67.41 % 67.72 %
	UbuntuU	411,209 55.67 % 53.38 %	47.74 % 47.83 %

Table 5 .

 5 TC4: Comparing WinXPU2 against WinXPU1 as the reference database.

	D	|D|	I ssdeep t = 40	I ssdeep t = 60	I ssdeep t = 100	ISHA-1
	WinXPU2 466,266 31.98 % 28.87 % 24.04 % 24.37 %

Table 6 .

 6 Identification rates for file paths and names in different images.

	D	|D|	ssdeep Without Hits SHA-1 Matches Matches Path Name
	WinXP(U 1 v. U 2); t=40 466,266 31.98 %	7.97 %	10.57 %	37.09 %
	WinXP(U 1 v. U 2); t=60 466,266 28.87 %	4.86 %	15.58 %	50.68 %
	Win7(B v.U ); t=40	66,312 93.85 % 26.44 %	1.51 %	5.79 %
	Win7(B v. U ); t=60	66,312 90.19 % 22.78 %	1.51 %	5.91 %
	Ubuntu(B v. U ); t=40	411,209 55.67 %	7.99 %	16.99 %	71.37 %
	Ubuntu(B v. U ); t=60	411,209 53.38 %	5.71 %	22.91 %	85.89

Acknowledgement

This research was partially supported by the European Union under the Integrated Project FIDELITY (Grant No. 284862) and by the Center for Advanced Security Research Darmstadt (CASED).


Ariel Stolerman

Alex Fridman

Rachel Greenstadt

Patrick Brennan

Patrick Juola

Active Linguistic Authentication Using Real-Time Stylometric Evaluation for Multi-Modal Decision Fusion

Keywords: Active authentication, stylometry, authorship verification

des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

The challenge of identity verification for the purpose of access control is the trade-off between maximizing the probability of intruder detec-tion and minimizing the cost to the legitimate user in terms of time and distraction due to false alerts, along with the extra hardware requirements for physical biometric authentication. In recent years, behavioral biometric systems have been explored to address this challenge [START_REF] Ahmed | A new biometric technology based on mouse dynamics[END_REF]. These systems rely on inexpensive input devices such as a keyboard and mouse. However, their performance in terms of detecting intruders and maintaining a low-distraction human-computer interaction experience has been mixed [START_REF] Bergadano | User authentication through keystroke dynamics[END_REF]. In particular, they have error rates ranging from 0% [START_REF] Obaidat | Verification of computer users using keystroke dynamics[END_REF] to 30% [START_REF] Ord | User authentication for keypad-based devices using keystroke analysis[END_REF], depending on the context, variability in task selection and other dataset characteristics.

The bulk of biometric-based authentication research has focused on verifying a user based on a static set of data. This type of one-time authentication is not well suited to a live multi-user environment, where a person may leave the computer for an arbitrary period of time without logging off. Such an environment requires continuous authentication when the computer is in a non-idle state. The Active Linguistic Authentication Dataset [START_REF] Juola | Towards active linguistic authentication[END_REF] used in this work was created to represent this general real-world scenario. The dataset, which was generated in a simulated office environment, contains behavioral biometrics associated with typical human-computer interactions by office workers.

Stylometry is a form of authorship recognition that relies on the linguistic information found in a document. While stylometry existed before computers and artificial intelligence (AI), the field is currently dominated by techniques such as neural networks and statistical pattern recognition. State-of-the-art stylometry approaches can identify individuals in sets of 50 authors with more than 90% accuracy [START_REF] Abbasi | Writeprints: A stylometric approach to identity-level identification and similarity detection in cyberspace[END_REF] and even scaled to more than 100,000 authors [START_REF] Narayanan | On the feasibility of Internet-scale author identification[END_REF]. Stylometry is currently used in intelligence analysis and forensics, with increasing applications in digital communication analysis [START_REF] Wayman | Technology Assessment for the State of the Art Biometrics Excellence Roadmap[END_REF]; the results are accurate and reliable enough to be admissible as legal evidence [START_REF] Chaski | Who's at the keyboard: Authorship attribution in digital evidence investigations[END_REF][START_REF] Chaski | The keyboard dilemma and forensic authorship attribution[END_REF]. The application of stylometry as a high-level modality for authenticating users in a continuous user verification system is novel. Initial evaluations of authorship attribution technologies are promising, realizing in excess of 90% identification accuracy over fourteen users [START_REF] Juola | Towards active linguistic authentication[END_REF]. This paper considers a set of stylometric classifiers, also referred to as sensors, as a representative selection of high-level behavioral biometrics. The primary goal is to evaluate authorship attribution approaches in realistic settings for active authentication, which require constant monitoring and frequent decision making about the legitimacy of a user at a computer in a dynamic, time-constrained environment. This work is designed as a preliminary evaluation of one modality among many to consider for an active authentication system. In the future, the sty-lometric modalities discussed in this paper would be interleaved with other low-and high-level modalities, such as keyboard dynamics [START_REF] Shanmugapriya | A survey of biometric keystroke dynamics: Approaches, security and challenges[END_REF], mouse movements [START_REF] Ahmed | A new biometric technology based on mouse dynamics[END_REF] and web browsing behavior [START_REF] Yampolskiy | Behavioral modeling: An overview[END_REF], in a centralized decision fusion system. The use of such modalities, including stylometry, may provide a cost-effective alternative to sensors based on physiological biometrics [START_REF] Wayman | Fundamentals of biometric authentication technologies[END_REF].

Although this research focuses on active authentication, a live security application of stylometric analysis and its implications for the usability and configuration of stylometric sensors are relevant to forensic contexts. Consider, for example, a standard post-mortem forensic analysis of user input data aggregated over an entire day. This research seeks to identify the features to consider in such "noisy" settings, which include window sizes, effects of overlapping windows and how idle periods in data input should be considered.

Related Work

A defining problem of active authentication is that the verification of an identity must be carried out continuously on a sample of sensor data that varies drastically with time. Therefore, the classification has to be made using a "window" of recent data, dismissing or discounting the value of older data outside the window. Depending on the user task being performed, some biometric sensors may provide more data than others. For example, when a user browses the web, mouse and web activity sensors are flooded with data, while keystroke and stylometric sensors may only receive a few infrequent key presses.

This motivates recent work on multimodal authentication systems where the decisions of multiple classifiers are fused together [START_REF] Sim | Continuous verification using multimodal biometrics[END_REF]; the resulting verification process is more robust to the dynamic nature of real-time human-computer interactions. This paper examines only the effectiveness of stylometric sensors under active authentication settings, the goal being to eventually construct a multi-modal biometric system. The notion of decision fusion is motivated by the research of Ali and Pazzani [START_REF] Ali | On the Link Between Error Correlation and Error Reduction in Decision Tree Ensembles[END_REF], which achieved reduced error rates using distinctly different classifiers (i.e., using different behavioral biometrics) with several fusion options [START_REF] Chair | Optimal data fusion in multiple sensor detection systems[END_REF][START_REF] Hashem | Improving model accuracy using optimal linear combinations of trained neural networks[END_REF][START_REF] Kittler | On combining classifiers[END_REF].

Authorship attribution based on linguistic style, or stylometry, is a well-researched field [START_REF] Argamon | Automatically profiling the author of an anonymous text[END_REF][START_REF] Jockers | A comparative study of machine learning methods for authorship attribution[END_REF][START_REF] Juola | Authorship attribution[END_REF][START_REF] Koppel | Computational methods in authorship attribution[END_REF][START_REF] Rudman | The state of authorship attribution studies: Some problems and solutions[END_REF][START_REF] Stamatatos | A survey of modern authorship attribution methods[END_REF]. Its principal application domain is written language -identifying an anonymous author of a text by mining it for linguistic features. The theory behind stylometry is that every person has a unique linguistic style or "stylome" [START_REF] Van Halteren | New machine learning methods demonstrate the existence of a human stylome[END_REF] that can be quantified and measured in order to distinguish between different authors. The feature space is potentially endless, with frequency measurements and numeric evaluations based on features across different levels of the text, including function words [START_REF] Binongo | Who wrote the 15th Book of Oz? An application of multivariate analysis of authorship attribution[END_REF][START_REF] Mosteller | Inference and Disputed Authorship: The Federalist[END_REF], grammar [START_REF] Kukushkina | Using literal and grammatical statistics for authorship attribution[END_REF], character n-grams [START_REF] Stamatatos | On the robustness of authorship attribution based on character n-gram features[END_REF] and more. Although stylometry has not been used for active user authentication, its application to this task brings higher-level inspection into the process compared with lower-level biometrics such as mouse movements and keyboard dynamics [START_REF] Bakelman | Continual keystroke biometric authentication on short bursts of keyboard input[END_REF][START_REF] Zheng | An efficient user verification system via mouse movements[END_REF].

The most common practice of stylometry is in supervised learning, where a classifier is trained on texts of candidate authors and used to attribute the stylistically closest candidate author to unknown writings. In an unsupervised setting, a set of writings whose authorship is unknown are classified into style-based clusters, each representing texts of a unique author.

In an active authentication setting, authorship verification is applied where unknown text is to be classified by a unary author-specific classifier. The text is attributed to an author only if it is stylistically close enough to the author. Although pure verification is the ultimate goal, standard authorship attribution as a closed-world problem is an easier (and sometimes sufficient) goal. In either case, classifiers are trained in advance and used for real-time classification of processed sliding windows of input keystrokes. If enough windows are recognized as an author other than the real user, the presence of an intruder is indicated.

Another use of stylometry is in author profiling [START_REF] Argamon | Automatically profiling the author of an anonymous text[END_REF][START_REF] Juola | Geographically localizing tweets using stylometric analysis[END_REF][START_REF] Van Halteren | Author verification by linguistic profiling: An exploration of the parameter space[END_REF]. This application is quite different from author recognition because writings are mined for linguistic features to identify characteristics of their authors such as age and gender [START_REF] Argamon | Mining the blogosphere: Age, gender and the varieties of self-expression[END_REF], native language [START_REF] Koppel | Determining an author's native language by mining a text for errors[END_REF] and personality characteristics [START_REF] Gray | Personality identification through on-line text analysis[END_REF].

In a pure authorship attribution setting, where classification is done off-line on complete texts (rather than sequences of input keystrokes) and in a supervised setting where all candidate authors are known, state-ofthe-art stylometry techniques perform very well. For instance, at PAN-2012 (pan.webis.de), some methods achieved more than 80% accuracy on a set of 241 documents, sometimes with added distractor authors.

Two key challenges arise in an active authentication setting. First, open-world stylometry is a much harder problem, with a tendency to yield high false negative (false reject) rates. The unmasking technique [START_REF] Koppel | Authorship verification as a one-class classification problem[END_REF] has been shown to be effective -95.7% accuracy -on a dataset of 21 books by ten nineteenth-century authors. However, the amount of data collected by sliding windows of sufficiently small durations requires efficient authentication and the lack of quality coherent literary writings render this approach infeasible for active linguistic authentication. Second, the inconsistent frequency nature of keyboard input along with the relatively large amount of data required for good performance of stylometric techniques render a large portion of the input windows unusable for learning writing style.

On the other hand, an active authentication setting offers some advantages with regard to potential features and analysis method. Since the raw data consists of keystrokes, some linguistic and technical idiosyncratic features can be extracted; these include misspellings caught before they are auto-corrected and vanish from the dataset and patterns of deletions such as selecting a sentence and hitting delete as opposed to repeatedly hitting backspace to delete individual characters. Additionally, in an active authentication setting, it is intuitive to consider overlaps between consecutive windows, resulting in a large dataset and providing grounds for local voting based on a set of windows and controlling the frequency at which decisions are made.

Evaluation Dataset

The complete Active Linguistic Authentication Dataset [START_REF] Juola | Towards active linguistic authentication[END_REF] is used in the evaluation. The dataset, which contains data collected in a simulated work environment, is designed specifically for behavioral biometric evaluation. The data collection utilized an office space, which was allocated, organized and supervised by some of the authors of this paper. The office space contained five desks, each with a laptop, mouse and headphones. The equipment and supplies were chosen to be representative of a standard office environment. However, one of the important properties of the dataset is uniformity. Because the computers and input devices in the simulated office environment were identical, the variations in behavioral biometrics data can be attributed to variations in the characteristics of the users instead of the effects of variations in the physical environment.

The dataset contains data collected from 80 users. Due to equipment and software crashes and sick days taken by the subjects, a little more than 80 subjects were used for data collection to reach the 80 user goal. However, when examining the data obtained from the 80 users, it was observed that some users had produced significantly less data than the others. In order to eliminate user activity variance effects, a threshold of 60,000 seconds (16.67 hours) of minimum activity was set. This threshold left 67 qualifying users for the evaluation presented in this paper.

Five temporary employees (subjects) were hired during each week of the data collection. The subjects had to work for a total of 40 hours. Each subject was assigned two tasks each day. The first, for six hours of the eight-hour workday, was an open-ended task to write blog-style articles related to the city in which the testing was carried out. The second task, involving two hours of the workday, was less open-ended. Each subject was asked to write summaries from a list of topics or web articles. The articles were from various reputable news sources and were kept consistent between users.

Both tasks encouraged the subjects to conduct extensive online research using web browsers. They were allowed to copy and paste content, but they were told that the final work product had to be their own authorship. As expected, the subjects almost exclusively used two applications: Microsoft Word 2010 for word processing and Internet Explorer for browsing the web. Although the user-generated documents are available in the dataset, the evaluation in this paper is based on the stream of keystrokes recorded during the workday, with the purpose of simulating the settings with which a real-time authentication system would have to operate.

The 67-user dataset was further parsed in order to produce one large stream of mouse/keyboard events. For every user, the entire five days of data was concatenated into one stream (in JSON format), and marked to be divided into five equally-sized folds for later cross-validation evaluation. In addition, periods of inactivity lasting more than two minutes were marked as idle. For the purposes of this research, a subset of events that only included keyboard strokes was maintained (the mouse events will be used by other sensors in future work). The format of one continuous stream permitted the complete utilization of the data in a real-time, continuous active authentication system. Table 1 summarizes the keystroke event statistics for the parsed 67-user dataset. The keystroke events include the alphanumeric keys as well as special keys such as shift, backspace, ctrl and alt. The keystroke counts only include the down presses, the key releases were ignored.

Methodology

This section describes the methodology in detail, including the challenges and limitations.

Challenges and Limitations

An active authentication system presents a few concerns. First, a potential performance overhead is expected to accompany the deployment of such a system because it requires constant monitoring and logging of user input, and on-the-fly processing of all its sensor components. With stylometric sensors, large amounts of memory and computation power are often consumed by language processing tools (e.g., dictionary-based features and part-of-speech taggers). Therefore, the system should be carefully configured to balance accuracy and resource consumption. This issue becomes more acute in a multi-modal system that uses multiple sensors.

A second concern with an active authentication system is the user input requirements. In a non-active authentication scheme, a user is required to provide credentials only when logging in and perhaps when certain operations are to be executed. The credentials include some sort of personal key (e.g., password or private key) that identifies the user. However, in the case of an active authentication system based on stylometric modalities, the user keyboard input is required. In a multimodal system, all user interactions may be required, including mouse events and web browsing behavior. The precise sequence and timing of keyboard events are essential to enhance system performance. However, such input is not designed for stylometric analysis and authentication. Additionally, it often contains sensitive and private information, which is collected when the user types in passphrases to log into accounts, writes something personal or simply browses the web. Some actions may be taken during system design to address security and privacy concerns. For example, the collected data could be managed carefully, storage of raw collected data could be avoided (except for parsed feature vectors extracted from the data) and all stored data could be encrypted. The privacy issue specifically applies to stylometric modalities, where the content of user input is of importance. Other modalities may void these issues by not targeting content, such as mouse movement biometrics that focus on physical characteristics of the user instead of the possibly sensitive semantics of the generated input.

Previous Evaluation

In earlier work [START_REF] Juola | Towards active linguistic authentication[END_REF], we presented an initial evaluation of a portion of the Active Linguistic Authentication Dataset (i.e., data for fourteen users). Two methods of evaluation were applied.

First, each day's worth of work was analyzed as one unit or document, for a total of 69 documents (five days for fourteen users, minus a missing day by one user). One-vs.-all analysis was applied using a simple nearestneighbor classifier with the Manhattan or intersection distance metric and character n-grams as features (1 ≤ n ≤ 5). The best result achieved was 88.4% accuracy.

In the second analysis, a number-of-characters-based sliding window technique was applied to generate the input segments to be classified; this provides a better simulation of the performance of a realistic active stylometric authentication system. The generated windows were nonoverlapping, with the window sizes set to 100, 500 and 1,000 words (tokens separated by whitespaces). The minimum window size was used to allow sufficient data for the stylistic profiling of the window. An extensive linguistic feature set, inspired by the one used in the Writeprints [START_REF] Abbasi | Writeprints: A stylometric approach to identity-level identification and similarity detection in cyberspace[END_REF] stylometry method, was employed along with a linear SVM and a nearest-neighbor classifier. The best result achieved was 93.33% accuracy with 0.009 FAR and 0.067 FRR.

The results demonstrate that it is beneficial to use stylometric biometrics for active authentication. However, the analysis approach, although satisfactory as a preliminary study, omitted some key requirements for an active authentication system. First, only fourteen subjects were used; thus, the system performance for a large set of users remains unknown. Stylometry research has thus far provided solutions for large author sets, but no research to date has focused on a dataset with incoherent and noisy qualities. Indeed, the approaches discussed above may prove to be inefficient for larger author sets.

Perhaps the main issue with the method of analysis is the units determined for learning/classification. Day-based windows are certainly not useful for active authentication, which aims to provide intruder alerts as quickly as possible (in minutes, if not seconds). Even the second data-based-windows analysis is insufficient: each window may have an arbitrary length in terms of its time span and collecting the minimum amount of words may allow an intruder enough time to apply an attack. Moreover, due to the possibility of large windows that may cross idle periods, data associated with different users could be mixed. For example, the first half of a window could contain legitimate user input while the second half, an idle-period later, could be supplied by an intruder. This causes the contamination of "bad" windows with "good" data that could throw off the classifier and cause it to miss an alert.

This paper provides an analysis of the authentication system in a more realistic setting. The focus is on a time-wise (instead of a data-wise) sliding window, and overlapping windows are allowed so that the system can output decisions with increased frequency. With this approach, the system is compelled to decide whether to accept or reject the latest window in a timely manner based on the data it has acquired thus far, or to determine that it cannot make a decision. The issue of balancing the amount of collected data, the required time-wise size of windows and the desired decision frequency is examined in the following section.

Real-Time Approach

The stylometric classifiers, or sensors, presented in this section are based on the simplest settings of closed-world stylometry: the classifiers are trained on the closed set of 67 users, where each classification results with one of the users being identified as the author. A more sophisticated approach would use open-world verifiers, where each legitimate user is paired to its own classifier in a one-class/one-vs.-all formulation. Such a verification approach is naturally suited to the open-world scenario, where possible imposters can originate outside the set of legitimate users (e.g., an intruder from outside an office who takes over an unlocked computer, instead of a malicious colleague). However, this paper considers the case of a closed set of possible users as a baseline for future verification-based classifiers.

In the preprocessing phase, the keystroke data files were parsed to produce a list of documents (text windows) consisting of overlapping windows for each user with time-based sizes of 10, 30, 60, 300, 600 and 1,200 seconds. For the first three settings, a sliding window was advanced with steps of 10 seconds of the stream of keystrokes; the last three settings used steps of 60 seconds. The step size determines how often a decision can be made by the sensor. In addition, although the window generation was configured with fixed parameters (e.g., a time-wise size of 300 and step of 60), in practice, the timestamps of the generated windows correlated with the keystroke events by relaxing the generation to no greater than 300 and no less than 60 with empty windows being discarded. In a live system, a similar approach is expected to be used: a window is "closed" and a decision is made when the size limitation time is up; hence, it is no greater than 300. In addition, when determining the beginning of a window followed by another window, a difference of at least one character is expected (otherwise the second window is a subset of the first). Therefore, if the time span between the first character in a window and the one that follows is greater than the determined step size, effectively a greater step size is applied; hence, it is no less than 60.

In this research, the generated windows were set to ignore idle periods, as if the data stream was continuous with no more than two minutes of delay between one input character and the next. This was applied in the dataset by preprocessing the keystroke timestamps such that an idle pe-riod longer than two minutes was artificially narrowed down to precisely two minutes. Furthermore, the data was aggregated and divided into five equally-sized folds for analysis purposes, thus potentially containing windows with an idle period between days. Although this preprocessing suffers from the problems associated with mixed legitimate/nonlegitimate user-input windows or mixed time-of-day windows (e.g., end of one day and beginning of the next day) if applied in a real system, in our analysis, the processing was applied to allow the generation of as many windows as possible. Since the analysis presented does not involve legitimate/non-legitimate mixed windows, idle-crossing windows are reasonable for the purpose of this work.

In the case of stylometry-based biometrics, selecting a window size involves a delicate trade-off between the amount of captured text (and the probability of correct stylistic profiling for the window) and the system response time. In contrast, other biometrics can perform satisfactorily with small windows (with sizes in the order of seconds). The problem is somewhat overcome using small steps (and overlapping windows), leaving the problem only at the beginning of a day (until the first window is generated). Similar to the analysis in [START_REF] Juola | Towards active linguistic authentication[END_REF], only keystrokes were considered during preprocessing (key releases were filtered) and all special keys were converted to unique single-character placeholders. For example, BACKSPACE was converted to β and PRINTSCREEN was converted to π. Representable special keys such as \t and \n were taken as is (i.e., tab and newline, respectively).

The chosen feature set is probably the most crucial part of the configuration. The constructed feature set, which we refer to as the AA feature set, is a variation of the Writeprints [START_REF] Abbasi | Writeprints: A stylometric approach to identity-level identification and similarity detection in cyberspace[END_REF] feature set that includes a vast range of linguistic features across different levels of text. Table 2 summarizes the features of the AA feature set. This rich linguistic feature set can better capture a user's writing style. With the special-character placeholders, some features capture aspects of a user's style that are not found in standard authorship problem settings. For example, frequencies of backspaces and deletes provide an evaluation of a user's typo-rate (or lack of decisiveness).

The features were extracted using the JStylo framework [START_REF] Mcdonald | Use fewer instances of the letter "i:" Toward writing style anonymization[END_REF]. JStylo was chosen for analysis because it is equipped with fine-grained feature definition capabilities. Each feature is uniquely defined by a set of its own document preprocessing tools, one unique feature extractor (core of the feature), feature post-processing tools and normalization/factoring options. The features available in JStylo are frequencies of a class of related features (e.g., frequencies of "a," "b," ..., "z" for the "letters" feature class) or some numeric evaluation of an input document (e.g., av- erage word length or Yule's Characteristic K). Its output is compatible with the Weka platform [START_REF] Hall | The Weka Data Mining Software: An update[END_REF], which was employed during the classification process. The definition and implementation of all the features in the AA feature set are available in JStylo, making it easily reproducible. Two important procedures were applied in the feature extraction phase. First, every word-based feature (e.g., function words class or different word-grams) was assigned a tailor-made preprocessing tool developed for the dataset; each tool applied the relevant special characters on the text. For example, the character sequence chββCchββhicago becomes Chicago, where β represents backspace. Second, since the windows are determined by time and not by the amount of collected data as in [START_REF] Juola | Towards active linguistic authentication[END_REF], normalization is crucial for all frequency-based features (which constitute the majority of the feature set). Each of these features was simply divided by the most relevant measurement related to the feature. For example, character bigrams were divided by the total character count of the window.

The classification used sequential minimal optimization support vector machines [START_REF] Platt | Fast training of support vector machines using sequential minimal optimization[END_REF] with a linear kernel and complexity parameter C = 1 available in Weka. Support vector machines are commonly used for authorship attribution [START_REF] Abbasi | Identification and comparison of extremistgroup web forum messages using authorship analysis[END_REF][START_REF] Koppel | Ad-hoc authorship attribution competition -Approach outline[END_REF][START_REF] Zheng | A framework for authorship identification of online messages: Writing-style features and classification techniques[END_REF] and are known to provide high performance and accuracy. As mentioned earlier, they are closed-world classifiers, i.e., they classify each window to one of the known candidate users (with the legitimate user as the true class). No acceptance thresholds were integrated in the classification process.

Finally, the data was analyzed with the stylometric sensors using a varying threshold for minimum characters-per-window spanning from 100 to 1,000 with steps of 100. For every threshold set, all the windows with less than the threshold amount of characters were discarded and the sensors outputted no decisions for these windows. The different thresholds help assess the trade-off in sensor performance in terms of accuracy and availability: as the threshold increases, the window has richer data and is potentially classified with higher accuracy, but the portion of total windows that pass the threshold decreases, making the sensor less available. Note that even the largest threshold (1,000 characters) is considerably smaller than that used in most previous stylometry analyses (minimum of 500 words). After filtering, only configurations with training data available for all users were retained; as expected, this resulted in the removal of sensors configured to small windows with high minimum number of character thresholds.

After eliminating sensors according to the rule mentioned above, 37 stylometric sensors were retained. These sensors spanned a variety of time-wise window sizes and minimum character-wise window sizes. In the remainder of this paper, the stylometric sensors are denoted as S n,m , where n denotes the time-wise window size in seconds and m denotes the minimum characters-per-window configuration.

Evaluation

The generated user data streams, divided into five equally-sized folds, are intended to be evaluated in a multi-modal decision fusion active authentication system. Such a system requires knowledge of the expected FAR and FRR rates of its sensors in order to make a cumulative weighted decision. Therefore, the intended evaluation is based on five-fold crossvalidation where, in each of the five validations, three folds were used for training, one fold for characterization of the FAR and FRR of the sensor, and the last fold for testing. Thus, each of the five validations outputted a decision for each test instance (from the last fold) and a global FAR and FRR characterization of the sensor. Eventually, the results of all five validations were averaged to determine the performance of the system. The configuration of the validations was cyclic, such that, in the first validation, folds 1, 2 and 3 were used for training, fold 4 for characterization and fold 5 for testing; in the second, folds 2, 3 and 4 were used for training, fold 5 for characterization and fold 1 for testing, and so on.

To evaluate the performance of the stylometric sensors, we use the results averaged over all train-characterize-test configurations described above to provide performance evaluations of the sensors when combined in a centralized decision fusion algorithm. Since the FRR and FAR produced in the characterization phase of the main experiments provide an evaluation of the reliability of the decisions made during the test phase, it is reasonable to use them to evaluate the standalone performance of the stylometric sensors.

Figures 1 and2 show the averaged FRR and FAR results. In particular, Figure 1 shows the averaged FRR for all characterization phases using the stylometric sensors with varying time-wise window sizes and varying thresholds for the minimum number of characters per window. Only windows passing the threshold (i.e., with at least that many characters) participated in the analysis. This measurement accounts for the portion of legitimate user windows that were not detected as belonging to the user (i.e., false alarms).

Figure 2 shows the averaged FAR for all characterization phases using the stylometric sensors with the same configurations. Note that the FAR accounts for the portion of intruder windows that were classified as belonging to a legitimate user (i.e., security breaches).

Figure 3 shows the averaged percentage of the remaining windows after all the windows that did not pass the minimum characters-perwindow threshold were removed.

The high FRR and low FAR suggest that the majority of the sensors are rather strict: they almost never falsely identify an intruder as le- gitimate, but a cost is paid in terms of a high FAR. The FRR results indicate that as the window size (in seconds) increases, the less the minimum characters-per-window threshold affects performance. The same trend is seen with the FAR results: the large windows (300, 600 and 1,200) show insignificant differences for different minimum charactersper-window thresholds. The availability of decisions as a function of the minimum charactersper-window thresholds (shown in Figure 3) completes the evaluation of the performance of the stylometric sensors. For example, S 1200,100 triggered every 60 seconds (the step configuration of the 1,200-secondwindows sensors) produces a decision 95% of the time with an accuracy of approximately 0.5/0 FRR/FAR.

Conclusions

A preliminary study by Juola, et al. [START_REF] Juola | Towards active linguistic authentication[END_REF] indicates the effectiveness of stylometric biometrics for active authentication. This paper puts the shortcomings of the preliminary study to the test by using settings that simulate a more realistic active authentication environment, with many users and high frequency decision making constraints. The results obtained under these settings demonstrate that the effectiveness of stylometric sensors deteriorates drastically, down to 0.5 FRR and 0 FAR. Nevertheless, the results are promising because the sensors may be used in a mixture-of-experts approach that fuses multi-modal sensors.

Although the configuration of data with overlapping sliding windows is realistic, the classification methodology is still limited and focused on closed-world SVM classifiers with an extensive linguistic feature set. Future analysis must include other classifiers, especially open-world verifiers that can be applied in scenarios where the set of suspects is not closed. In addition, because of the noisiness of the data, other feature sets should be considered, including sets that focus less on high linguistic characteristics of the text (like POS-taggers) and more on typing patterns. A mixture of writing style and typing style quantification could, perhaps, achieve better profiling with this type of data.

The immediate next step in evaluation is to use a multi-modal fusion system employing multiple sensors. Sensors to be considered include mouse movements, keyboard dynamics (i.e., low-level key patterns) and web-browsing behavior. The configuration of such a system based on closed-world sensors and data-based windows evaluated on a subset of nineteen users achieves approximately 1% FAR/FRR, but its performance in open-world settings with the complete dataset is as yet unknown and is currently the subject of investigation.

Figure 1 .

 1 Figure 1. Averaged FRR for all characterization phases.

Figure 2 .Figure 3 .

 23 Figure 2. Averaged FAR for all characterization phases.

Table 1 .

 1 Character count statistics over five workdays.

	Minimum per user	17,027
	Maximum per user	263,165
	Average	84,206
	Total	5,641,788

Table 2 .

 2 AA feature set.

	Group	Features
	Lexical	Average word length
		Characters
		Most common character bigrams
		Most common character trigrams
		Percentage of letters
		Percentage of uppercase letters
		Percentage of digits
		Digits
		Two-digit numbers
		Three-digit numbers
		Word length distribution
	Syntactic Function words
		Part-of-speech (POS) tags
		Most common POS bigrams
		Most common POS trigrams
	Content	Words
		Word bigrams
		Word trigrams