Wilfried Mayer
email: wmayer@sba-research.org

Martin Schmiedecker
email: mschmiedecker@sba-research.org

Turning Active TLS Scanning to Eleven

Keywords: Cipher Suite Scanning, SSL, TLS, Network Security

Transport Layer Security (TLS) is the fundament of today's web security, but the majority of deployments are misconfigured and left vulnerable to a phletora of attacks. This negatively affects the overall healthiness of the TLS ecosystem, and as such all the protocols that build on top of it. Scanning a larger number of hosts or protocols such as the numerous IPv4-wide scans published recently for a list of known attacks in TLS is non-trivial. This is due to the design of the TLS handshake, where the server chooses the specific cipher suite to be used. Current scanning approaches have to establish an unnecessary large number of connections and amount of traffic. In this paper we present and implemented different optimized strategies for TLS cipher suite scanning that, compared to the current best practice, perform up to 3.2 times faster and with 94% less connections used while being able to do exhaustive scanning for many vulnerabilities at once. We thoroughly evaluated the algorithms using practical scans and an additional simulation for evaluating current cipher suite practices at scale. With this work full TLS cipher suite scans are brought to a new level, making them a practical tool for further empiric research.

Introduction

Transport Layer Security (TLS) is the fundament of today's web security and provides confidentiality and authentication for application layer protocols like HTTPS, e-mail-related protocols or smartphone applications. Successful attacks against TLS are irritating the security community on a regular basis. Many of these attacks exploit vulnerabilities in the underlying cryptographic primitives, which, when grouped together, form so-called cipher suites. Often the mitigation of these vulnerarbilites is achieved by simply discontinuing the use of insecure cipher suites. Although easily done, this is a manual configuration step, which results in a slowly adopting TLS ecosystem. This progress is only observable through Internet-wide measurements.

Full cipher suite scans are important in order to understand in-depth the TLS ecosystem and the impact of discovered vulnerabilities, as demonstrated recently [START_REF] Aviram | Breaking TLS using SSLv2[END_REF][START_REF] Mayer | No Need for Black Chambers: Testing TLS in the E-mail Ecosystem at Large[END_REF]. Only with detailed information it is possible to thoroughly assess the state of online security, ranging from the security of a single host up to the security of the whole ecosystem. With the recent advent of fast-paced scanning tools it has become possible to proactively scan the entire range of IPv4 on a regular basis. This data is invaluable when reacting to newly released attacks.

In this work, we developed three new scanning algorithms that efficiently test TLS configurations in detail. These full cipher suite scans can then be used to cluster configurations based on different cipher suites, identifying common misconfigurations and facilitate TLS stack fingerprinting. We evaluated these algorithms and estimated the performance gain for Internet-wide full cipher suites scans. We then used these algorithms to scan parts of the IPv4-wide Internet and analyzed the results. The specific contributions of this paper are:

-We introduce highly optimized scanning methodologies to perform TLS scanning at scale. -We evaluate our improved methodologies against the top-10k websites, and are on average 3.2 times faster. -We show that current cipher suite recommendations are hardly used.

-We publicly release the source code and collected data from our experiments under an open source license 1 .

The remainder of this paper is organized as follows: In Section 2 we present the relevant background as well as the body of related work. Section 3 introduces our optimized scanning methodologies and the data inputs used for our evaluation. Section 4 illustrates the achievable gain in overall performance and provides insights into the current TLS deployment. We discuss the results in Section 5 before we conclude in Section 6.

Background

TLS itself is specified in a variety of RFCs. The most important one is RFC5246 [START_REF] Dierks | The Transport Layer Security (TLS) Protocol Version 1.2. RFC 5246[END_REF]. It defines the most modern version of TLS, version 1.2, introduced in 2008. Version 1.3 contains significant changes, but is still a working draft [START_REF] Rescorla | The Transport Layer Security (TLS) Protocol Version 1.3 draft[END_REF]. One of the goals of TLS is extensibility, i.e., the possibility of exchanging the used cryptographic functions. This is accomplished through the concept of cipher suites. Cipher suites are combinations of cryptographic primitives, defined as a two-byte value [START_REF]Transport Layer Security (TLS) Parameters[END_REF]. The used cipher suite and the TLS version are negotiated in the first two exchanged TLS messages (client hello, server hello). First, the client sends a client hello message including a list of supported cipher suites. Second, the server replies with a server hello choosing one of these cipher suites. This cryptographic primitives are subsequently used. A large number of cipher suites exists (over 140), and they can be used in different TLS versions (SSLv3, TLSv1, TLSv1.1, TLSv1.2). This results in approx. 550 different combinations that can be tested.

Many security problems are caused by the use of old and deprecated features of TLS, e.g., the support of export-grade algorithms, old TLS versions or insecure ciphers. Two examples are POODLE [START_REF] Möller | This POODLE bites: exploiting the SSL 3.0 fallback[END_REF] which is caused by the use of deprecated SSLv3 and DROWN [START_REF] Aviram | Breaking TLS using SSLv2[END_REF] which is based on the active support of SSLv2. Some attempts to get rid of old cryptography were made, e.g., the ban of export-grade crypto in modern TLS versions or RFC7465 [START_REF] Popov | Prohibiting RC4 Cipher Suites[END_REF] that forbids the use of the insecure RC4 cipher. A secure deployment is non-trivial, therefore several guidelines give recommendations on how to (i) configure cipher suite settings and (ii) improve the configuration of TLS-enabled server applications [START_REF]Applied Crypto Hardening[END_REF][START_REF] Sheffer | Recommendations for Secure Use of Transport Layer Security (TLS) and Datagram Transport Layer Security (DTLS)[END_REF]. However, these methods all rely on the administrator to actively improve the setup by changing the supported cipher suites manually -hence, the ecosystem is adopting slowly.

In the early days, nmap was used to perform these types of scan on a larger scale, but it is rather slow and does not scale to a larger number of hosts in reasonable time. A breakthrough was achieved with the development of zmap [START_REF] Durumeric | ZMap: Fast Internet-wide Scanning and Its Security Applications[END_REF] and masscan [START_REF] Graham | Masscan: the entire Internet in 3 minutes[END_REF]. Both tools use new methods to optimize large-scale scanning and are so far mainly used for port and vulnerability scanning. However, these improved methods are not applicable to fine-grained TLS scanning. With zgrab it is possible to establish TLS connections, but it is still not feasible for examining full cipher suite configurations. A more intense scanning behavior is necessary due to the design of TLS. Tools like SSLyze implement naive algorithms that conduct a full scan of all cipher suites by using one TLS handshake for each cipher suite. This is slow and produces a lot of traffic, thus a huge potential for optimization exists. With the results of this work, we are able to efficiently measure cipher suite configurations for TLS, also for large-scale studies.

Related Work

Prior studies that measured the TLS ecosystem focused primarily on the certificate ecosystem, the overall security was rarely evaluated. An early study was conducted by Lee et al. in 2007 [18]. With only 19,000 evaluated servers, this is a long way from an Internet-wide scale. Nevertheless, the size of measurement studies increased constantly, with larger studies conducted by the EFF [START_REF] Eckersley | An Observatory for the SSLiverse[END_REF] a few years later. Also, additional passive data was taken into account (Amann et al. [START_REF] Amann | No Attack Necessary: The Surprising Dynamics of SSL Trust Relationships[END_REF][START_REF] Amann | Revisiting SSL: A Large-Scale Study of the Internet's Most Trusted Protocol[END_REF]). With new scanning methods (e.g., zmap [START_REF] Durumeric | ZMap: Fast Internet-wide Scanning and Its Security Applications[END_REF]), studies were suddenly able to cover the IPv4-wide Internet. These methods implemented new ideas, e.g., no per-connection state. This improved the speed and quality of large-scale scans. Studies that used this new scanning behavior are, e.g., the certificate ecosystem study by Durumeric et al. [START_REF] Durumeric | Analysis of the HTTPS Certificate Ecosystem[END_REF], that doesn't cover supported cryptographic primitives, and studies on vulnerabilities like Heartbleed [START_REF] Durumeric | The Matter of Heartbleed[END_REF] that solely examine one exclusive issue.

Most of these studies focused on specific details in the configuration, e.g., the properties of a certificate. Fewer studies scanned all cryptographic primitives at once, i.e., all supported cipher suites. Huang et al. [START_REF] Huang | An Experimental Study of TLS Forward Secrecy Deployments[END_REF] describes the results of a complete cipher suite scan to measure perfect forward-secrecy support, but scanned only hosts from the Alexa top 1 million list [START_REF][END_REF]. Mayer et al. [START_REF] Mayer | No Need for Black Chambers: Testing TLS in the E-mail Ecosystem at Large[END_REF] performed cipher suite scans for all e-mail-related ports at an IPv4-wide scale. Both studies used naive algorithms to perform the scan. Other projects like the Qualys SSLTest [START_REF]SSL Server Test[END_REF] also scan full TLS configurations, but these projects are designed for single host configuration tests and not for Internet-wide studies. Newer studies tried to draw a complete picture of the certificate ecosystem [START_REF] Van Der Sloot | Towards a Complete View of the Certificate Ecosystem[END_REF] while missing the underlying security primitives, others decided against scanning full TLS cipher suites, because it would require to establish too many connections [START_REF] Holz | TLS in the wild: an Internet-wide analysis of TLS-based protocols for electronic communication[END_REF].

Methodology

To improve the scan rate for TLS-specific scanning, we defined the following requirements: First, time as the overall time consumption of the scanning process; second, the support for parallelization -can different scans be executed in parallel or do they rely on partial results and therefore require a sequential execution? These two requirements are especially important for large-scale scans. Third, the number of connections necessary for a scan: How many connections are necessary for a full configuration scan? Also, the generated traffic is derived from the number of connections. Lastly, the completeness of the scan, or how much information we can gather from the results: Is it possible to draw a complete picture of the TLS configuration or is it just one specific detail?

The anticipated use cases range from an interested system administrator or CISO who wants to scan infrastructure for security vulnerabilities up to Internetwide scans for either specific questions or complete ecosystem analysis.

We identified three existing approaches to scan and identify TLS configurations: The naive approach establishes one connection for each cipher suite, starting at the same time. For each connection the server replies with either this cipher suite or with an alert that the cipher suite is not supported. This method is currently implemented by the command line tool SSLyze [START_REF]SSLyze -Fast and full-featured SSL scanner[END_REF]. It highly parallelizes all requests, which results in a fast execution time, especially for non-delaying networks. The number of connections and produced traffic is rather large. This can lead to errors for some hosts, because the number of parallel connections may exceed their limit. This disadvantage leads to error-prone results and affects the completeness in a negative way. SSLyze (version 0.12) produces exactly 543 connection attempts to test all cipher suite/TLS version combinations and approx. 500KB of traffic (inbound and outbound) per tested host. These numbers clearly don't scale for Internet-wide scans, making SSLyze impractical for this task.

The second approach is implemented by zmap. This command line tool, created by Durumeric et al. [START_REF] Durumeric | ZMap: Fast Internet-wide Scanning and Its Security Applications[END_REF], is primarily used for Internet-wide port scans. With zgrab they also implemented an application layer scanner capable of scanning TLS configurations. To minimize the number of connections to exactly one per server, the cipher suites in the client hello message are fixed to a specific research question, e.g., in order to test if RC4 is supported, all cipher suites that use RC4 are included. The server then responds with a server hello message (showing the support of RC4) or an alert (showing that RC4 is not supported). This one-connection-based approach minimizes traffic and performs fast. The downside is that it does not completely evaluate all cipher suites. It is limited in its expressiveness, since only one question per scan can be evaluated.

The third approach is used by the SSL Server Test [START_REF]SSL Server Test[END_REF]. This web service is designed to test and evaluate one specific web server configuration. Therefore it utilizes the cipher suite settings from different browsers and browser versions as well as settings to test common misconfigurations. It then establishes one TLS connection per setting to completely evaluate one server configuration. It also includes HTTPS-specific settings and security features, e.g., HSTS or HPKP. The information collected is comprehensive, but the service's design is not suitable for large-scale ecosystem studies.

Introducing New Approaches

We propose the following new approaches for cipher suite scanning:

Connection-Optimal Approach This approach tests all cipher suites per TLS version in a serialized way. The process is illustrated in Figure 1. It starts with one TLS client hello message that includes all available cipher suites for this TLS version. The server then responds with one cipher suite that it accepts. The next handshake includes all cipher suites except the one that was accepted earlier. This procedure is repeated until the server does not accept any of the offered cipher suites and responds with an alert. All remaining cipher suites are then evaluated as rejected. This approach uses the optimal, lowest number of connections necessary, but is not parallelizable for one host. Therefore it needs more time, especially for networks with a delayed round trip.

Grouped by Cryptographic Primitives

The second approach, presented in Figure 2, groups cipher suites according to their used cryptographic primitives. It is based on the assumption that server operators disable or enable all cipher suites with a common primitive (e.g., deactivate all RC4-based cipher suites). After the cipher suites are split up in groups, the process follows the methodology of the connection-optimal approach. We currently use groups based on keywords in the cipher suite name, i.e., SRP, PSK, EXP, NULL, (DSA, DSS), (ADH, AECDH), (CAMELLIA, SEED, IDEA, DES-CBC-), RC4. Primitives that are Fig. 1. Connection-Optimal Approach not supported can be filtered out in the very first round. This approach supports parallel execution of the group tests so that it works with fewer round trips than the connection-optimal approach.

Based on Existing Results

The third approach, as presented in Figure 3, combines the ideas from the former approach with data from already conducted cipher suite scans. It is based on the fact that many server operators use the same configuration, e.g., a default configuration. The most likely configuration based on former results is calculated before a client hello is sent. After the first round of concurrent handshakes, an intermediary result is evaluated. Based on this result, the next, most probable configuration is computed. The cipher suites used in the next round of parallel sent client hello messages are then adjusted. This goes on until all cipher suites are either rejected or accepted. This approach is based on data described in the next paragraph. Existing Data For the last algorithm, we rely on the dataset of an Internetwide study we conducted from April to August 2015 [START_REF] Mayer | No Need for Black Chambers: Testing TLS in the E-mail Ecosystem at Large[END_REF]. We additionally use cipher suite scans of the HTTPS ecosystem, performed in August 2015. These datasets are very extensive w.r.t. the number of scanned cipher suites. Because of the large dataset (approx. 12 million error-free results), we transformed each result for each single host/port combination to a string. This string has a length of 551 characters2 . This represents the total number of TLS version and cipher suite combinations. Each TLS version/cipher suite is either accepted or rejected, which is represented by the characters a respectively r. We did not take different behaviors of key exchange algorithms or different error messages (for rejected cipher suites) into account. In Table 1, the five most-used combinations for HTTPS are shown (a black bar represents an accepted cipher suite, a white bar a rejected cipher suite). We see that 7.8% of all hosts share one configuration in which all cipher suites for SSLv2 and SSLv3 are rejected and the supported cipher suites for TLSv1 and TLSv1.1 are identical. As an example, the first two bars represent the accepted AES128-SHA and AES256-SHA, whereas the next cipher suites are rejected (CAMELLIA128-SHA, CAMELLIA256-SHA). When we take a closer look at the number of existing patterns per TCP port and the percentage of hosts that use these patterns, we see that a small number of patterns are used for most of the hosts. This is especially true for SMTP, where we see that the two most-used patterns cover more than 50% of all SMTP-enabled hosts. In Figure 4 the percentage of hosts that is covered by an increasing number of patterns for various TCP ports is shown. We assume that it is possible to optimize scanning methods by using this information. Also, the raw data of this patterns is publicly available.

Implemented Approaches

We implemented all approaches by creating an additional mechanism to store partial results. Based on this partial result, the next requests are computed and executed, adding information to the partial result until it is complete. Users are able to choose the algorithm by specifying a command line argument (e.g., --algorithm=connopt). The required connections and time are logged for every run. Based on the existing data, we also implemented a simulation that calculates the number of necessary connections and rounds per approach. The complete source code is publicly available.

Results

We evaluated the proposed improvements by simulating an Internet-wide scan on IPv4 with existing scan data. We computed two performance values: The number of average established connections necessary to scan one host (C ) and the number of average rounds (round trips) to scan one host (R). These two parameters are a good indicator for the defined requirements. Generated traffic and connections are directly mapped to C, the degree of parallelization and therefore the time needed is mapped to R. The results of this simulation are shown in Table 2. We can see that all new approaches use fewer connections than the naive approach. The optimum is achieved with the connection-optimal approach, although this method uses a lot of rounds and is therefore not parallelizable (and probably the slowest of all algorithms), except of the different TLS versions. Thus, the number of connections is five times bigger than the number of rounds. The group-based algorithm lies in between, with further potential to optimize the chosen groups. The algorithm based on existing data shows a low number of connections as well as a low number of rounds. For HTTPS on port 443 it minimizes the number of connections to an average of 37.3 (6.8%) with an average of 1.8 rounds. The simulation is based on the same dataset as the algorithm, so the expressiveness of this method will decrease in the future (as configurations change), but can be easily readopted with newer results.

Experimental Results

We tested the performance of our algorithms with scans in the wild. We used SSLyze version 0.12 and scanned a predefined set of hosts out of the Alexa Top10k list. We shuffled it and created batches of 100 hosts. With each algorithm we scanned 25 batches and measured the time needed and connections performed. We restrained from changing other aspects of SSlyze, like multiprocessing, multithreading or the general process. We also did not optimize kernel settings or other parameters on operating system level in order to compare only the algorithms with the default behavior. We used commodity hardware with an 100MBit/s uplink. The results are presented in Table 3. The naive approach performs worst in terms of speed. Also, connection-wise every new approach performs better than the naive approach. Although it has a more complex implementation, the approach based on existing data performs only slightly better than the algorithm based on crypto groups. Also listed in Table 3 are scans with a slightly larger set of hosts, used in the Section 4.2. Figure 5 visualizes the large performance gain we can achieve with our approaches. It shows the average time for one host and the average number of connections per valid, scanned host of every tested batch. These results show a large improvement in TLS cipher suite scanning algorithms. The connection-optimal algorithm is 3.2 times faster than the naive implementation (avg. connection-optimal compared with avg. naive) and uses only 6% of the connections (avg. connection-optimal compared with avg. naive) to execute a full TLS cipher suite scan in the wild. The connection-optimal approach and the group-based approach are correctly simulated, but we see that the results of the method based on existing data differ from the simulated results. We argue that this is due to two reasons: First, the algorithm and the simulation are based on the same data. If configurations change, the algorithm gets slower. The second reason is that we practically evaluated top-10k web services and not random hosts, whereas the simulation also considers a large number of small hosts.

Cipher Suite Results of Top-10k Domains

We used the connection-optimal algorithm to perform an additional cipher suite scan on the Alexa top-10k domains [START_REF][END_REF]. Cisco Umbrella recently proposed an alternative to the often-used Alexa Top 1 million list [START_REF] Hubbard | Cisco Umbrella 1 Million[END_REF], so we decided to scan these top-10k domains as well. We analyzed which patterns occur, if these patterns are secure and if we can find a trend to common and secure TLS configurations. First, we looked at the most-used patterns in the Umbrella top-10k list. Although the three most-used patterns are used by 39.6% of the Umbrella top 10k resp. Finally, we tried to compare the results with pattern statistics we used for our simulation. We see that there are differences in the pattern usage, and we argue that the average top-10k host is differently configured than the average host from an Internet-wide scan.

Discussion

Internet scanning is not only a technical challenge. It also has to deal with ethical issues. Other studies already pointed out current best practices [START_REF] Durumeric | ZMap: Fast Internet-wide Scanning and Its Security Applications[END_REF] which include to "scan no larger or more frequent than is necessary". This discouraged studies from performing a full scan, e.g., Holz et al. [START_REF] Holz | TLS in the wild: an Internet-wide analysis of TLS-based protocols for electronic communication[END_REF]. They stated that a full TLS cipher suite scan "is a poor trade-off in terms of good Internet citizenship versus lessons that can be learned". With our work, full TLS cipher suite scans can be conducted with less than 6% (approx. 32) of the connections compared to the currently used naive algorithm (543 connections). This minimizes the load each target host has to handle to a manageable minimum and makes the trade-off in terms of good Internet citizenship absolutely arguable. Good Internet citizenship is not only about minimizing the impact of one scan. It is also about avoiding unnecessary scans at all. One solution are publicly available results of scans, for which Censys [START_REF] Durumeric | A Search Engine Backed by Internet-wide Scanning[END_REF], a search engine for Internet-wide scans, is a good example. They use the scanning approaches mentioned in Section 3, but an integration with the results of full TLS cipher suite scans is possible. We publish all our datasets and the source code.

In this work we optimized the methodology for full TLS cipher suite scans. For the practical evaluation we didn't change important factors of SSLyze to speed up the process. Important factors to optimize the bandwidth usage are, e.g., TCP port reuse, optimal settings for the TCP/IP stack or TCP connection reuse. The most influential factor is the parallelization of the scanning infrastructure. SSLyze (version 0.12) uses a maximum of 12 processes with 15 threads for all hosts; if the number of hosts is larger than that, the hosts are queued internally. This behavior is not optimal, since the server is idling. The solution is to split up all hosts amongst a large number of concurrent processes to minimize idling. With some optimizations applied, we were able to scan 27K hosts per hour with the naive approach on commodity hardware (100MBit/s uplink). We did not bundle these optimizations with our new approaches in order to focus on our comparison.

The approaches are created for TLS versions up to TLS v1.2. With TLSv1.3, which is currently a working draft, many things will change. Many insecure features are dropped, e.g., static RSA or DH key exchanges, insecure ciphers or hash-functions like MD5. Also, the handshake mechanism will be changed, so only one round trip is necessary to establish a full TLS connection. This -and also the question how TLSv1.3. is going to be deployed in the wild -affects the problem of how to efficiently scan full TLSv1.3 configurations.

Conclusion

In this paper we presented existing and new approaches for cipher suite scanning which is an important tool to evaluate the current status of the TLS ecosystem. Until now, naive approaches were used which are not optimal in terms of connections, scanning time or traffic transmitted over the wire. We introduced three new approaches that make use of the TLS protocol specification, common configurations and existing results. We evaluated the performance gain of these methods and found that we were able to perform scans 3.2 times faster with only 6% of the connections. We implemented a version of the described methods to work with a commonly used tool, simulated them and then evaluated them in practice by conducting a cipher suite scan for Alexa and the Umbrella top-10k hosts, describing the results and common patterns.

Fig. 2 .

 2 Fig. 2. Approach Based on Grouping Cryptographic Primitives

Fig. 3 .

 3 Fig. 3. Approach Based on Cipher Suite Statistics

3

 3

Fig. 4 .

 4 Fig. 4. Host Coverarage By Number of Patterns

Fig. 5 .

 5 Fig. 5. Experimental Results of Different Approaches

Table 1 .

 1 Most-used cipher suite patterns for HTTPS, Internet-wide scan in Aug. 2015

	7.8%
	5.5%
	5.3%
	4.3%
	2.7%

Table 2 .

 2 Comparison of Simulation Results with Existing Scan Data

		Port 25	Port 110		Port 143		Port 443
		C	R	C	R	C	R	C	R
	Naive	551.0	1.0 551.0	1.0 551.0	1.0 551.0	1.0
	Connection-optimal 110.0	22.0	42.7	8.5	42.7	8.5	28.2	5.6
	Crypto-group-based 252.0	7.3 199.9	3.8 199.8	3.8 187.7	2.4
	Existing-data-based 141.0	1.5	52.3	1.5	51.4	1.4	37.3	1.8

Table 3 .

 3 Experimental Results of the Different Approaches

		# Scans Valid	Time (s)	Connections
	Approach	(Hosts) Results Total Min Avg Max	Total Min Avg Max
	Naive	25 (100) 1,866 14,356 0.92 7.69 15.42 1,012,976 542.6 542.9 543.0
	Connection-optimal 25 (100) 1,896 4,473 0.92 2.36 4.70	60,723 28.7 32.0 34.7
	Crypto-group-based 25 (100) 1,914 5,462 0.64 2.85 4.96 351,534 182.1 183.7 185.8
	Existing-data-based 25 (100) 1,870 4,672 0.50 2.50 5.98 268,814 126.1 143.8 156.4
	Connection-optimal 5 (2,000) 9,262 5,951 0.56 0.64 0.75 314,398 33.3 33.9 34.4
	Connection-optimal 5 (2,000) 7,534 9,493 1.13 1.26 1.38 244,644 31.7 32.5 33.5

Table 4 .

 4 Cipher Suite Patterns

		Umbrella Top10k Alexa Top10k
	e.g., xx.fbcdn.net	18.53%	1716 8.51%	641
	e.g., google.com	13.43%	1244 6.15%	463
	e.g., configuration.apple.com 7.63%	707 1.87%	141
	Mozilla Modern Conf.	0.02%	2 0.05%	4
	Mozilla Intermediate Conf.	0.98%	91 3.28%	247
	Mozilla Old Conf.	0.35%	32 0.15%	11

  [START_REF] Van Der Sloot | Towards a Complete View of the Certificate Ecosystem[END_REF].9% of the Alexa top-10k, 524 and 954 (Umbrella/Alexa) different configurations exist. This indicates a highly diverse ecosystem. Second, we analyzed proposed cipher suite settings. Mozilla introduced a tool, the Mozilla SSL Configuration Generator[START_REF]Mozilla SSL Configuration Generator[END_REF], to generate secure configurations for various compatibility requirements, i.e., modern, intermediate and old. We see that the cipher suite pattern for a modern configuration is only used by 2 resp. 4 hosts in the top-10k lists. Their intermediate configuration is used by a recognizable number (91, 247). The exact numbers are also shown in Table4. Third, we looked at differences between these patterns. All patterns disabled SSLv2 and SSLv3.In contrast to the modern Mozilla configuration (only TLSv1.2), the other configurations support TLSv1 to TLSv1.2. In contrast to the intermediate configuration, TripleDES with DH key exchanges is not supported. The xx.fbcdn.net configuration is supporting more cipher suites (CAMELLIA, non-elliptic-curve Diffie-Hellman), whereas configurations like google.com support only one cipher suite more than configurations like configuration.apple.com, i.e., AES256-SHA.

The patterns, the mappings and the source code are available online at: https://github.com/WilfriedMayer/turning-active-tls-scanning-to-eleven

551 cipher suites were tested with SSLyze version 0.11. Because the underlying TLS implementation changed, version 0.12 does not test two specific cipher suites for four TLS versions, thus only 543 connections. Existing results for these cipher suites are ignored in the algorithm.

https://scans.io/study/sba-email

Acknowledgments. The research was funded by COMET K1 and by grant 846028 (TLSiP) from the Austrian Research Promotion Agency (FFG).


Lydia Kraus
email: lydia.kraus@qu.tu-berlin.de

Robert Schmidt

Marcel Walch
email: marcel.walch@uni-ulm.de

Florian Schaub
email: fschaub@umich.edu

Sebastian Möller
email: sebastian.moeller@tu-berlin.de

On the Use of Emojis in Mobile Authentication

Keywords: Mobile authentication, Security, Usability, User experience, Emoji

Mobile authentication methods protect smartphones from unauthorized access, but also require users to remember and frequently enter PINs, passwords, or graphical patterns. We propose the EmojiAuth scheme with which we study the effects of Emoji use on the usability and user experience of mobile authentication. We conducted two between-subjects studies (lab study: n=53; field study: n=41) comparing EmojiAuth to standard PIN entry. We find that Emo-jiAuth provides good memorability for short passwords and reasonable memorability for longer passwords. Moreover, we identify diverse Emoji-password selection strategies and provide insights on the practical security of Emoji-based mobile authentication. Our results suggest that Emoji-based authentication constitutes a practical alternative to traditional PIN authentication.

Introduction

Usability of mobile authentication is an active research topic [START_REF] Egelman | Are You Ready to Lock?[END_REF][START_REF] Harbach | It's a hard lock life: A field study of smartphone (un)locking behavior and risk perception[END_REF][START_REF] Zezschwitz | Patterns in the wild: a field study of the usability of pattern and pin-based authentication on mobile devices[END_REF], given that users spend a considerable amount of time unlocking their phones [START_REF] Harbach | It's a hard lock life: A field study of smartphone (un)locking behavior and risk perception[END_REF]. Knowledge-based authentication mechanisms, such as PIN and unlock pattern (on Android), have been widely deployed for smartphone locking; alphanumerical passwords are also a common option. While PINs, especially 4-digit PINs, are susceptible to user choice [START_REF] Bonneau | A birthday present every eleven wallets? The security of customer-chosen banking PINs[END_REF] and shoulder surfing [START_REF] Schaub | Exploring the design space of graphical passwords on smartphones[END_REF], they balance short log-in time and good memorability with sufficient protection against casual attackers [START_REF] Schaub | Exploring the design space of graphical passwords on smartphones[END_REF]. Biometric authentication, such as fingerprint and face recognition emerged recently as alternatives, but still rely on knowledgebased authentication as a fallback [START_REF] Bhagavatula | Biometric authentication on iphone and android: Usability, perceptions, and influences on adoption[END_REF]. Therefore, knowledge-based authentication remains relevant for smartphones and is unlikely to be replaced soon. However, if users need to spend mental effort and time to protect their smartphone, the required interactions should be as pleasant and positive as possible.

Designing positive interactions has gained considerable attention in user experience research. Concepts such as hedonic (product) qualities, joy of use, and stimulation evolved as important aspects of user experience design [START_REF] Bargas-Avila | Old wine in new bottles or novel challenges: a critical analysis of empirical studies of user experience[END_REF]. We argue that considering positive interaction aspects is also relevant in the design of usable security mechanisms.

An interesting direction for positive interaction in mobile authentication is the use of Emojis as password characters. Emojis are largely used in positive contexts [START_REF] Novak | Sentiment of emojis[END_REF] and are popular among users. Thus, providing potential for offering positive user experiences. Emoji-based passwords have recently been introduced by a commercial application [START_REF]Intelligent Environments: Now you can log into your bank using emoji[END_REF].

In this paper, we study opportunities of Emojis for creating a positive mobile authentication experience. We further study how Emoji-based authentication influences password selection and shoulder surfing. To gather insights, we developed an Emojibased authentication scheme (EmojiAuth) and evaluated it in a lab study (n=53) and a field study (n=41), including a shoulder-surfing experiment (n=38). Our contributions include [START_REF] Egelman | Are You Ready to Lock?[END_REF] the identification of five main Emoji-password selection strategies; (2) a comparative evaluation of PIN-and Emoji-based passwords regarding their susceptibility to shoulder surfing, indicating a slight improvement with Emojis; and (3) an analysis of the user experience of Emoji-based passwords. While Emoji and PIN show similarly high usability, users indicated that they would prefer Emoji over PIN as a screen lock.

Related Work

Mobile authentication has received considerable research attention [START_REF] Egelman | Are You Ready to Lock?[END_REF][START_REF] Harbach | It's a hard lock life: A field study of smartphone (un)locking behavior and risk perception[END_REF][START_REF] Zezschwitz | Patterns in the wild: a field study of the usability of pattern and pin-based authentication on mobile devices[END_REF][START_REF] Harbach | Keep on lockin' in the free world: A multi-national comparison of smartphone locking[END_REF]. A multinational survey showed that 50.4% (Italy) to 76.4% (UK) of users use a screen lock on their phone [START_REF] Harbach | Keep on lockin' in the free world: A multi-national comparison of smartphone locking[END_REF]. Authentication schemes can be divided into knowledge-based, tokenbased, and biometric schemes [START_REF] O'gorman | Comparing passwords, tokens, and biometrics for user authentication[END_REF]. The Emoji-based password scheme belongs to the class of graphical authentication schemes which is a subclass of knowledge-based authentication. In the following we detail related work on these two areas.

PINs and passwords are commonly deployed knowledge-based authentication schemes. While PINs can be entered quickly and accurately [START_REF] Schaub | Exploring the design space of graphical passwords on smartphones[END_REF][START_REF] Zezschwitz | Patterns in the wild: a field study of the usability of pattern and pin-based authentication on mobile devices[END_REF], they lack entropy. With a 4-digit PIN the password space is constrained to about 14 bit. Users tend to weaken PINs by choosing easy-to-remember numbers, e.g., birth dates [START_REF] Bonneau | A birthday present every eleven wallets? The security of customer-chosen banking PINs[END_REF]. Random passwords are more secure but harder to remember [START_REF] Yan | Password memorability and security: Empirical results[END_REF]. PINs generated under a security policy are more secure, but also harder to remember than freely-chosen PINs [START_REF] Kim | Pin selection policies: Are they really effective? computers & security[END_REF].

Graphical authentication schemes are motivated by the fact that graphics are easier to remember than alphanumeric passwords [START_REF] Biddle | Graphical passwords: Learning from the first twelve years[END_REF]. As for PINs and passwords, major issues of graphical passwords arise from the susceptibility to capture and guessing attacks [START_REF] Biddle | Graphical passwords: Learning from the first twelve years[END_REF]. For instance, image-based cued-recall schemes are prone to hotspots [START_REF] Biddle | Graphical passwords: Learning from the first twelve years[END_REF], i.e., image regions users are likely to select, which can be used in guessing attacks. Graphical passwords can also take longer to enter. A study with Android pattern unlock found that participants needed twice as long to enter a pattern and made more mistakes compared to a PIN [START_REF] Zezschwitz | Patterns in the wild: a field study of the usability of pattern and pin-based authentication on mobile devices[END_REF]. Yet, users tend to rate pattern usability and likability similar to PIN, likely due to easy error recovery [START_REF] Zezschwitz | Patterns in the wild: a field study of the usability of pattern and pin-based authentication on mobile devices[END_REF]. However, to be practical, a login attempt should not take longer than for a PIN or a pattern lock mechanism [START_REF] Harbach | It's a hard lock life: A field study of smartphone (un)locking behavior and risk perception[END_REF]. Patterns have a smaller theoretical password space as PINs and their security is considered low in general [START_REF] Elenkov | Android Security Internals: An In-Depth Guide to Android's Security Architecture[END_REF].

Icon-based graphical authentication schemes are a promising approach enabling fast log-in times [START_REF] Schaub | Exploring the design space of graphical passwords on smartphones[END_REF], while potentially providing a theoretical password space similar to PIN or larger. The Story scheme [START_REF] Davis | On User Choice in Graphical Password Schemes[END_REF] is somewhat similar to our proposed Emojibased scheme as users create a password from a 3x3 set of photo icons from different categories (objects, food, people). An interesting finding is that Story did not result in a skewed password probability distribution [START_REF] Davis | On User Choice in Graphical Password Schemes[END_REF]. Emoji-based authentication has been recently suggested [START_REF]Intelligent Environments: Now you can log into your bank using emoji[END_REF]. Shortly after our lab study was conducted, Golla et al. conducted an online study to investigate the susceptibility of Emoji-based passwords to guessing attacks [START_REF] Golla | Emojiauth: Quantifying the security of emoji-based authentication[END_REF]. Their Emoji-based authentication scheme features a keyboard with 20 Emojis. With their scheme, they found that the distribution of Emoji-passwords is skewed, but 4-digit user-chosen Emoji-passwords were still more resistant to guessing attacks than 4-digit user-chosen PINs.

User experience and authentication should be considered together. To create a positive user experience, psychological needs, such as stimulation and popularity, should be addressed in the interaction design of mobile authentication mechanisms [START_REF] Kraus | Exploring psychological need fulfillment for security and privacy actions on smartphones[END_REF]. Also, while mobile and graphical authentication schemes have been investigated intensively in terms of usability and security, user experience evaluations beyond usability, have received little attention [START_REF] Kraus | User experience in authentication research: A survey[END_REF].

While Emojis have been used in authentication, we are the first to study usability and user experience of an Emoji-based scheme in the lab and in the wild, as well as its resistance to shoulder-surfing attacks.

EmojiAuth: Emoji-based Authentication Scheme

The use of Emojis may lead to a positive and pleasing user experience and positive perception of EmojiAuth: Emojis have been shown to enable the expression of moods, emotions and nuances in written text [START_REF] Cocozza | Crying with laughter: how we learned how to speak emoji[END_REF]. Thus, Emojis may also make authentication more (personally) meaningful for users. Emojis further have positive associations which may lead to authentication being perceived positively as well. The most frequently used Emojis are rated significantly more positive than the remaining Emojis [START_REF] Novak | Sentiment of emojis[END_REF].

Similar to PIN entry, our EmojiAuth scheme features twelve buttons (cf. Figure 1(a)). We further designed a PIN lock as a baseline comparison (cf. Figure 1(b)). In both schemes, if users enter their password correctly, the entry field turns green and the screen unlocks automatically. If the password is incorrect, the phone vibrates and a respective message appears above the entry field. The use of a keyboard with twelve Emoji buttons is grounded in the advantages of PIN keyboards: PIN entry is easy and fast [START_REF] Zezschwitz | Patterns in the wild: a field study of the usability of pattern and pin-based authentication on mobile devices[END_REF]. Simple keyboards have further been linked to authentication usability [START_REF] Schaub | Password entry usability and shoulder surfing susceptibility on different smartphone platforms[END_REF].

In EmojiAuth's keyboard generation, three Emojis are randomly selected from each of four categories (Person and Face: 226 Emojis, Object: 287 Emojis, Nature: 204 Emojis, and Activity: 44 Emojis) to support easy assembly of passwords. Once a userspecific keyboard has been initialized, the Emojis and their position remain static to reduce search time and thus enable shorter login times [START_REF] Schaub | Exploring the design space of graphical passwords on smartphones[END_REF][START_REF] Stobert | Memory Retrieval and Graphical Passwords[END_REF].

The theoretical password space of EmojiAuth is more than two times larger than the password space of PINs for 4-digit passwords (EmojiAuth: 20,736; PIN: 10,000), and almost three times larger than PIN for 6-digit passwords (EmojiAuth: 2,985,984; PIN: 1,000,000). However, that users favor certain Emojis is evident from rankings of currently popular Emojis [START_REF] Rothenberg | emojitracker: realtime emoji use on twitter[END_REF] and has been also shown as an issue in related work on Emoji-based authentication [START_REF] Golla | Emojiauth: Quantifying the security of emoji-based authentication[END_REF]. To mitigate the issue of hotspot Emojis, EmojiAuth generates an individual keyboard for each user during password enrollment. Keyboards generated from a large set of Emojis may increase the practical password space as specific Emojis have low probability to appear on individual keyboards, thus decreasing the probability that certain Emojis are favored across the whole user population.

We conducted a lab study and a field study, both between subjects, to evaluate Emo-jiAuth (treatment) in comparison to PIN entry (control). In the lab study, we evaluated memorability, selection strategies, and user experience of Emoji-passwords. In the field study, we validated our findings in the wild. We further conducted a shoulder-surfing experiment at the end of the field study.

Lab Study

Methodology and Procedure

In the lab study, the Emoji and PIN conditions were further divided into two subgroups to investigate effects of varying password length (4 and 6 digits). Groups are subsequently referred to as Emoji-4, Emoji-6, PIN-4 and PIN-6. The lab study was conducted in two sessions. The first session started with participants signing the consent form and completing an entry questionnaire on demographics and smartphone use. They were informed that passwords they create in the study will be stored in plain text to enable scientific analysis, but will not be linked to their identity. Participants were then assigned round-robin to an Emoji or PIN group. Participants who currently used a PIN (or fingerprint and PIN combination) on their smartphone were assigned to the Emoji group, in order to reduce the impact of prior habituation to PIN entry.

After a training task with randomly generated passwords, participants were asked to choose their own password and instructed that they will have to remember it. After creating their password, they had to enter it three times with a mental rotation task (MRT) between attempts. The MRTs served to distract participants and clear their short-term memory between login attempts [START_REF] Chiasson | Graphical Password Authentication Using Cued Click Points[END_REF][START_REF] Schaub | Exploring the design space of graphical passwords on smartphones[END_REF]. Participants then completed a usability and user experience questionnaire (AttrakDiff 2 mini [START_REF] Hassenzahl | The inference of perceived usability from beauty[END_REF]) and a five-minute semi-structured interview, in which they were asked to describe how they selected their password and their level of confidence in remembering their password. AttrakDiff 2 mini measures different aspects of user experience [START_REF] Hassenzahl | The inference of perceived usability from beauty[END_REF][START_REF] Diefenbach | Handbuch zur Fun-ni Toolbox[END_REF]: pragmatic quality (PQ), hedonic quality (HQ), and attractiveness (ATT). Each dimension is measured on a semantic differential with 7 rating levels between differentials. Pragmatic quality is related to usability, i.e., functional aspects of a product [START_REF] Hassenzahl | The thing and i: understanding the relationship between user and product[END_REF]. Hedonic Quality (HQ) relates to the capability of a product to address aspects of personal relevance [27, p. 38]. The hedonic quality scale is further divided into the sub-dimensions Stimulation and Identity [START_REF] Diefenbach | Handbuch zur Fun-ni Toolbox[END_REF]. Stimulation refers to a products' capability to provide stimulating experiences (e.g. in terms of providing new impressions, opportunities, insights), whereas identity refers to a products' capability to communicate identity [27, p. 35]. Attractiveness is related to the overall judgment of a product [START_REF] Diefenbach | Handbuch zur Fun-ni Toolbox[END_REF].

One week after the first session, participants returned to the lab for the second session. Participants had to enter the password they created in the first session and completed the same usability and UX questionnaire. They were also asked in a short interview how they memorized their password and whether they had written it down. All participants conducted the study on the same smartphone (LG Nexus 5, Android 5.1.1). The interviews were recorded and transcribed verbatim for further analysis. Participants received 4e compensation for the first study session, and 8e for the second one in order to incentivize participants to return and thus reduce drop-outs. Participants were recruited through a participant panel of TU Berlin, classified ads posted on an online service similar to Craigslist, flyers, and e-mail.

Results

In total, 53 smartphone users participated in the lab study: 14 in the Emoji-4 group, 13 in each of the other three groups. Participants were 18 to 70 years old (M =31, Mdn.=27); 28 were male, 25 female. The average time between sessions was 7 days (SD=1.2 days; range 3-12 days due to scheduling). Over half the participants were students (58.5%), despite not targeting campus populations. Other participants were employees (15.1%), self-employed (7.5%), retired (5.7%), and others (13.2%). Most (75.5%) did not have a professional or educational IT background. In the sample were 69.8% Android users, 22.6% iOS users, and 7.6% other smartphone users. Most participants (69.8%) reported to use authentication on their phone; most common were PIN (28.3%), unlock pattern (22.6%), and fingerprint with PIN as fallback (11.3%).

Password memorability The lab study results indicate high memorability of both

EmojiAuth passwords and PINs. After one week all participants (EmojiAuth and PIN) were able to successfully authenticate within three attempts. Long Emoji-passwords seem to be slightly harder to remember after a week of non-use, as a lower number of participants managed to enter their password correctly for all three trials in week 2 (Emoji-4: 92.9% in both weeks; Emoji-6: 100% in week 1 and 69.2% in week 2; PIN-4: 100% in both weeks; PIN-6: 100% in week 1 and 92.3% in week 2). A Fisher's exact test did not reveal statistically significant differences between groups. Only four PIN participants reported writing down their passwords after the first session.

Password selection Interviews on password selection strategies were first coded openly by one coder, who created separate code books for Emoji and PIN with some overlapping codes. Two coders then independently re-coded all interviews with the code books. Multiple codes could be assigned. Interrater agreement was high for both groups (Emoji: Cohen's κ=.83; PIN: κ=.72). Coders subsequently reconciled the remaining cases. Participants in the PIN group relied on predictable password selection strategies, e.g., birth dates as PIN [START_REF] Bonneau | A birthday present every eleven wallets? The security of customer-chosen banking PINs[END_REF]. The selection strategies of the Emoji participants overlapped only partially with the PIN strategies. Emoji participants often selected passwords based on a preference for certain Emojis and remembered them by creating stories, memorizing spatial patterns or repeating characters. We identified five main password selection strategies each for Emoji passwords and PINs (frequencies are provided in Table 1):

-Emoji preference (Emoji): Emojis are selected based on personal preference, e.g., "Well I clicked those Emojis I was interested in" (P33). The PIN selection strategies are consistent with findings in related work. For instance, dates as PINs or parts of passwords are commonly observed [START_REF] Bonneau | A birthday present every eleven wallets? The security of customer-chosen banking PINs[END_REF][START_REF] Fahl | On the ecological validity of a password study[END_REF] and were also the most frequent selection strategy in our study. We further observed spatial patterns as PIN selection strategies, which are known user strategies to improve memorability [START_REF] Bonneau | A birthday present every eleven wallets? The security of customer-chosen banking PINs[END_REF][START_REF] Fahl | On the ecological validity of a password study[END_REF]. The re-use of passwords is another well-known issue [START_REF] Fahl | On the ecological validity of a password study[END_REF] that also surfaced in our study. Participants reported that they used former or current PINs.

The emergence of the Emoji-password selection strategy "Preference" suggests that passwords generated with EmojiAuth may also follow a skewed password distribution. We analyzed the set of Emoji-passwords created in both our studies to further explore this issue (cf. Section 5.3).

User experience Pragmatic Qualtiy (PQ) for Emoji was medium-high in week 1 (M =4.5, SD =1.4), but lower compared to PIN ((M =5.9, SD =0.77). A Kruskal-Wallis test revealed a significant difference between the groups, H(3)=16.25, p=.001, with PQ for Emoji-4 and Emoji-6 being significantly lower than for PIN-4. In week 2, PQ increased for Emoji (M =5.5, SD =1.2) and approximated the ratings for PIN (M =5.9, SD =0.71). The Kruskal-Wallis test did not reveal significant differences in PQ between groups in week 2. Hedonic Quality in terms of Stimulation was medium-high for Emoji (week 1: M = 4.8; SD = 1:36; week 2: M = 4.9; SD = 1.38) and mediumlow for PIN (week 1: M = 3.8; SD = 1.19; week 2: M = 4.0; SD = 1.13). Differences were significant in both weeks (Mann-Whitney U ; week 1: U =185;p=.003; week 2: U , U =209;p=.018). This suggests that Emoji users found the authentication more stimulating in both weeks compared to PIN.

Field Study

Methodology and Procedure

The field study consisted of a pre-study questionnaire, an introductory session, a field phase of 15-17 days, and an exit session. In order to ensure meaningful use of the authentication methods during the study, we deployed EmojiAuth and PIN as a protection mechanism for the participants' email app on their own phone. E-mails have been shown to often contain sensitive information [START_REF] Egelman | Are You Ready to Lock?[END_REF] worth protecting. Consequentially, we recruited only Android users who use an email app on their device and verified this in a screening survey. Participants were recruited through a participant panel of TU Berlin and classified ads posted on an online service similar to Craigslist. Participants from the first study could not participate. Participants received 25e compensation of which 5e were paid at the introductory session and 20e at the end.

During the introductory session participants received information about the study and were asked for consent. Then, either EmojiAuth or PIN was installed as a lock for their email app on their own devices. We used Android accessibility services to monitor whether the e-mail app is currently in the foreground. In order to activate this service, the participants had to select one or more e-mail apps which they currently use from the list of installed apps. After they created an Emoji-password or PIN (depending on the group), opening their email app required participants to authenticate with their password/PIN. Our apps had a 30 second time-out for an authentication session, i.e., if participants left their e-mail app for 30 seconds or more, they had to re-authenticate.

Participants were asked to pick their password/PIN at home. It had to be at least 4 digits. For the PIN group, only meta-data of the user-chosen PINs was collected (PIN length and number of differing characters).

Directly after creating the password, participants received a questionnaire asking about the importance of different password/PIN selection criteria, which were derived from the lab study results. Participants could change their password or PIN during the study (within our app) and EmojiAuth users could further generate a new Emojikeyboard. In case that they had forgotten their password or PIN, users could enter a pre-defined backup-password in our app and select a new password/PIN. If the password/PIN was entered five times incorrectly in a row, users also had to provide their backup-password to unlock their e-mail app and to select a new password.

The field phase lasted 15-17 days, depending on when participants scheduled their exit session. Similar to Wechsung et al.'s study [START_REF] Wechsung | View from a distance: comparing online and retrospective ux-evaluations[END_REF], participants received a daily reminder to complete a daily feedback questionnaire, which asked participants to rate on a Smiley-scale how they liked interacting with EmojiAuth or PIN that day. Participants could further explain their rating in a free-text field. On days 2, 8, and 14, participants further received the AttrakDiff2 mini-questionnaire to assess user experience.

After the field phase, participants returned to the lab for the exit session in which they completed an exit survey (on paper) followed by the shoulder-surfing experiment. Furthermore, EmojiAuth/PIN was uninstalled from their devices.

Shoulder-Surfing Experiment

The field study's exit session contained a shoulder surfing experiment, modeled after similar experiments in related work [START_REF] Schaub | Exploring the design space of graphical passwords on smartphones[END_REF][START_REF] Tari | A comparison of perceived and real shoulder-surfing risks between alphanumeric and graphical passwords[END_REF], in which the threat model is a casual observer. Participants acted as shoulder surfers for either EmojiAuth or PIN (based on their field study condition), whereas the experimenter served as the observation target. In contrast to related work, our shoulder surfers were experienced with the authentication scheme they tried to observe after two weeks of use. Participants could position themselves either left, right or behind the experimenter who sat at a table to enter the password. Participants were provided with pen and paper for note taking. To ensure that passwords are entered with similar speed and in the same position, the experimenter trained password entry beforehand.

To test shoulder surfing susceptibility for passwords created with different password selection strategies, the procedure was repeated with five passwords in counterbalanced order. Emoji-and PIN-passwords used the same spatial position of keys on the keyboard in order to facilitate direct comparison between the two schemes. The first and second passwords were random 6-digit ('341779') and 4-digit passwords ('1706'). The third ('134679') and fourth passwords ('5802') were patterns lab study participants had created. The fifth password was an association based on the Christmas Eve date ('2412') for the PIN users and a Christmas-related story created by a lab study participant for the Emoji users ('bear -Christmas tree -snowman -heart' or '23#4' on a numerical keyboard). After a password was entered by the experimenter, the participant had three trials to enter the observed password on a LG Nexus 5 smartphone (Android 5.1.1).

Results

In total, 41 smartphone users participated in the field study: 21 in the Emoji group, 20 in the PIN group. Three PIN users had to be exluded (2 due to issues with participants' phones; one due to out of scope/inappropriate responses in almost all daily feedback questions). Thus, the PIN group decreased to 17.

Participants were 19-63 years old (M =34, Mdn.=28, SD=12.1); 24 were female (59%). Most were students ( 22), although we did not target students. The second largest group were employees (8), followed by job seekers [START_REF] Schaub | Exploring the design space of graphical passwords on smartphones[END_REF], self-employed (2), and others (4). Most (80.5%) did not have a professional IT background. 19 participants currently used a PIN, 3 a password, 9 an Android pattern, and 11 did not use any locking method.

Success rates In both groups, few incorrect unlocks were recorded during the field study (Emoji: 3% of total unlocks; PIN: 1.5%). In total, 3,514 correct and 83 incorrect unlocks were recorded. EmojiAuth accounted for 1,924 correct (M =91.6, SD=66.1) and 58 incorrect unlocks (M =2.8, SD=4.2); PIN accounted for 1,590 correct (M =93.5, SD=70.4) and 25 incorrect unlocks (M =1.5, SD=1.6). Fisher's exact tests did not reveal significant differences in the number of correct and incorrect unlocks between the groups.

Success rates for PIN were high, suggesting that PIN performs well in the wild. This confirms related work that found PIN to be a practical authentication method with low error rates [START_REF] Zezschwitz | Patterns in the wild: a field study of the usability of pattern and pin-based authentication on mobile devices[END_REF]. Emoji success rates were also high, suggesting that EmojiAuth is a practical authentication method, too.

Password length and changes

The majority of participants in the Emoji group [START_REF] Kraus | User experience in authentication research: A survey[END_REF] initially picked a 4-digit password, whereas 2 participants picked a 5-digit password. Participants in the PIN group initially picked diverse PIN lengths: 10 picked a 4-digit PIN, 2 picked a 5-digit PIN, 3 picked a 6-digit PIN, and 2 an 8-digit PIN. A Mann-Whitney U test did not indicate significant differences in the mean password length between groups (Emoji: M =4.1, SD=.3; PIN: M =4.9, SD=1.4).

Four Emoji participants changed their password once, 3 changed their password twice. In the PIN group, 4 participants also changed their PIN once, 1 changed their PIN twice. A Mann-Whitney U test did not indicate significant differences in the mean number of password changes between groups (Emoji: M =.48, SD=.75; PIN: M =.35, SD=.61).

Password selection

The same password selection strategies identified in the lab study also surfaced in the field study (cf. Table 1). Figure 2 provides examples of Emojipasswords created by study participants in the lab and in the field study.

Based on the results of the lab study, we asked questions (available online at: http://bit.ly/2imyb2H) about Emoji and PIN password selection in the field. For EmojiAuth, the questionnaire contained 17 5-point items (1='does not apply at all'; 5='completely applies'), with 2-4 items to measure each selection strategy. For PIN, the questionnaire contained 15 items, with 1-6 items per selection strategy. Lab study frequencies for Table 1 were calculated by counting the occurrences of each interview code. Field study frequencies were calculated as the number of participants who rated at least half of the items of a scale (selection strategy) as important or very important.

The overlaps between selection strategies in both studies suggest reasonable validity of the identified strategies. The PIN selection strategies in both studies align with findings in related work [START_REF] Bonneau | A birthday present every eleven wallets? The security of customer-chosen banking PINs[END_REF]. For Emoji-password selection, Preference, Pattern & Position, and Association & Story played an important role in both studies.

The importance of the Preference selection strategy for Emoji passwords is also visible from the distribution of selected Emojis across passwords. Figure 3 depicts three examples of the most popular and three examples of the most unpopular Emojis together with their occurrences on the keyboards (lab and field study). Due to the different sizes of the category lists from which Emojis were selected in EmojiAuth, some Emojis appear more often on the keyboard than others. Although we expected the individual keyboards to decrease the probability of hotspots, Figure 3 suggests that the distribution of password-Emojis is skewed.

Shoulder surfing

We calculated the minimal Levenshtein distance for each user ("attacker") and each password, i.e., the number of deletions, insertions, or substitutions, needed to obtain the correct password from the entered password [START_REF] Schaub | Password entry usability and shoulder surfing susceptibility on different smartphone platforms[END_REF][START_REF] Levenshtein | Binary codes capable of correcting deletions, insertions and reversals[END_REF]. There was a significant difference in the minimal Levenshtein distance between Emoji (M =2.45, SD=1.64) and PIN (M =.72, SD=.83) for the 6-digit random password (Mann-Whitney-U, U =289.0; p=.001; r=.53) with medium effect. Thus, the 6-digit random password was significantly harder to shoulder surf on the Emoji keyboard. For the other passwords, there were no significant difference between the authentication methods.

We also compared shoulder surfing susceptibility of passwords from the same scheme. For Emoji, a Friedman's test revealed significant differences in the minimal Levenshtein distance between passwords (χ 2 =40.44; p<.001). Post-hoc analysis with Bonferroni correction revealed that the 6-digit random password was significantly harder to shoulder surf than the 4-digit random password (M =.75, SD=.93; Z=1.45, p=.037, r=.46), the 6-digit pattern (M =.15, SD=.67; Z=2.75, p<.001, r=0.72), and the 4-digit pattern (M =.15, SD=.37; Z=2.2; p<.001, r=.70). All post-hoc results for Emoji had medium to large effect sizes. For PIN, a Friedman's test revealed significant differences between passwords (χ 2 =10.78; p<.029), but post-hoc tests were not significant. The post-experiment questionnaires revealed four different strategies attackers used to observe the password: paying attention to the numbers on the keyboard ("numbers"), the password's spatial pattern ("pattern"), a mix of both strategies ("mix"), or they reported observing password entry with high concentration ("observation"). The frequencies of strategies significantly differed between Emoji and PIN (p=.026; Fisher's exact). "Attackers" in the Emoji group were more likely to use the pattern observation strategy (Emoji: 16; PIN: 8). Not surprisingly, "attackers" in the PIN group were more likely to use the numbers observation strategy (Emoji: 0; PIN: 4).

In summary, the 6-digit random password was harder to shoulder surf with the Emoji keyboard compared to PIN and was also harder to shoulder surf with the Emoji keyboard compared to the 4-digit random password and the 4-and 6-digit pattern passwords on the Emoji keyboard. The casual "attackers" in the Emoji group largely relied on the pattern observation strategy which may make Emoji passwords that are based on spatial patterns more suscebtible to shoulder surfing attacks.

User experience

The daily feedback questionnaires answered during the field study indicate that the user experience of EmojiAuth and PIN was perceived similarly well. This is supported by the AttrakDiff 2 mini ratings, with the difference that EmojiAuth users perceived the authentication method more stimulating at the beginning of the study. In total, participants reported 342 (Emoji: 184) positive experiences, 99 neutral experiences (Emoji: 51), and 14 negative experiences (Emoji: 10). A Mann-Whitney U test did not reveal significant differences between distribution of positive, neutral, and negative experiences between groups.

To further analyze users' experiences, the free-text answers of the daily feedback were open-coded by one coder. This led to a code list of 17 codes. The qualitative data was then independently coded with the code list by another coder. Inter-rater agreement was high (Cohen's κ=.83). The coders jointly reconciled the remaining cases. A third of participants' comments (35%) expressed that everything was well (e.g., "everything's ok," "fine," "works"). The second most common comments (10%) concerned good usability of the methods (e.g., "really easy and not annoying", "fast [PIN] entry, no problems, I don't have concerns regarding memorability as long as the positions of the numbers don't change"). Six percent of comments indicated participants got familiar with the methods (e.g., "I've became accustomed to it," "it [the authentication] already belongs to my daily routine"). Thereby, Emoji participants reported this twice as much as PIN participants (14 vs. 7 comments). Four percent of codes concerned hedonic aspects. Hedonic aspects were mostly mentioned by Emoji users (11 out of 14, e.g., "I liked choosing the Emojis as I could select them on my own without restrictions," "it was fun to open the e-mail app with the Emojis while sitting next to my friends," "I changed my password twice today as I was curious which other Emojis are available"). A few comments (2.5%) also concerned perceived security vulnerabilities of the schemes ("when I open the app in quick succession, EmojiAuth didn't work properly" [participant was likely corollary not aware of 30 s time-out]; "it's relatively easy for others to find out the [Emoji] combination").

The AttrakDiff 2 mini ratings align with the daily feedback: Pragmatic quality was perceived as high (M > 5) for both methods at all measurement points (day 2, 8, and 14). Emoji users rated hedonic quality in terms of stimulation higher than PIN users on day 2 (Emoji: M = 4.62, SD = 0.89; PIN: M = 3.22, SD = 0.60; Mann-Whitney U , U = 34; p < .001; r =.70). However, this effect disappeared over time: there were no significant differences in stimulation between the groups for day 8 and 14.

Despite negligible quantitative differences in user experience, 17 of 20 Emoji users reported in the exit questionnaire that they would prefer using Emojis over PIN as a screen lock, mainly due to the high perceived memorability of Emoji-passwords (12 answers) and the appeal of the Emoji-based method (six answers).

Discussion and Conclusion

Limitations. Our study has a few potential limitations. Participants self-selected to participate in a study on mobile authentication, thus our participants may have higher technology affinity than the general population. As the sample size in both studies was limited, generalizations should be made with caution. However, our results facilitate a meaningful comparison of EmojiAuth to the current baseline: PIN entry. Furthermore, the consistency between lab and field study findings indicates a reasonable validity of our results.

Practical Emoji authentication. We have gained valuable insights into the practical aspects of Emoji-based mobile authentication. The results suggest that EmojiAuth may be a practical authentication method with a good password memorability of short passwords and a reasonable memorability of longer passwords. Study participants created their Emoji-based passwords with five different strategies: Emoji preference, as-sociation & story, pattern & position, repetition & similarity, and color & shape. The results suggest that the distribution of Emoji passwords may be skewed, even with individual keyboards. We plan to conduct further studies to quantify the frequency of each selection strategy and its contribution to the practical password space. Results from the shoulder-surfing experiment suggest that EmojiAuth performs better for longer passwords that do not follow distinct spatial patterns. As the "attackers" in this experiment mostly focused on the pattern strategy, we recommend that spatial patterns should not be used for password creation. We also plan to conduct further studies to investigate whether password creation policies could help users create Emoji passwords that are resistant to guessing and capture attacks, as well as memorable. For example, such policies could blacklist most popular Emojis or spatial patterns.

The role of UX in mobile authentication. Both, EmojiAuth and PIN, were perceived as highly usable and as providing a good user experience in the lab and the field study. In the field study, EmojiAuth users mentioned hedonic aspects slightly more often in their daily feedback. However, for both methods, the overall number of experiences related to hedonic aspects was rather low. The Hedonic Quality/Stimulation ratings indicate that EmojiAuth users perceived their authentication method as more stimulating in the beginning of the field study compared to PIN users. The majority of EmojiAuth users (field) also indicated that they would prefer EmojiAuth over PIN as a screen lock, which is a promising result. We plan to conduct further studies to investigate how hedonic quality could be further increased and maintained in authentication methods and whether it contributes to long-term user "relationships" with the authentication method.

Fig. 1 .

 1 Fig. 1. EmojiAuth and PIN user interfaces. The original UIs were in German. Emojis are depicted in the Noto Emoji Font by Google Inc. https://github.com/googlei18n/ noto-emoji.

  -Association & story (Emoji): Participants leverage an association between Emojis and their own knowledge or experience, and/or a password is selected or memorized by creating a story connecting the Emojis, e.g., "[I selected the password] after a song. [...] each Emoji stands for one word and depending on the song which words came first, I typed [the Emojis] in." (P22); "I just thought about the weekend [laughing]" (P3). -Pattern & position (Emoji): A spatial pattern is used to create or remember the password and/or the position on the keyboard is used to remember certain Emojis, e.g., "And then I went from the upper left down to the bottom right" (P16). -Repetition & similarity (Emoji): Either single characters or character sequences are repeated to create a password and/or a password is assembled from Emojis which are (subjectively) similar to each other, e.g., "[I chose the password so] that the pictures look similar" (P39). -Color & Shape (Emoji): A Password is selected based on color or shape of Emojis, e.g., "Well... first I chose four symbols with the same color." (P16); "I chose [the Emojis] according to circular shape" (P18). -Date (PIN): A date of personal importance (birthday, anniversary, etc.) is used to create a PIN. -Repetition & sequence (PIN): Single numbers or number sequences are repeated to create a PIN and/or a PIN is created with consecutive numbers. -Re-use (PIN): A PIN is selected by re-using a current or former PIN. -Pattern & position (PIN): A spatial pattern is used to create or remember the PIN and/or the keyboard position is used to remember certain numbers.-Association (PIN): An association between numbers and the user's knowledge or experience is used to select the password (e.g., choosing a name that contains a number or a phone number as PIN).

Fig. 2 .

 2 Fig. 2. EmojiAuth passwords created by lab and field study participants. Passwords are grouped according to password selection strategies.

Fig. 3 .

 3 Fig. 3. Password-Emojis examples of the most popular (left) and unpopular (right) password-Emojis together with their occurrences on the keyboards.

Table 1 .

 1 Frequencies of password selection strategies. Note that some participants used multiple strategies.

		Emoji	PIN
	Strategy	Lab	Field	Lab	Field
		(n=27) (n=20) (n=26) (n=17)
	Color and Shape	2 (7%) 9 (43%)	-	-
	Icon Preference	10 (37%) 12 (60%)	-	-
	Repetition	9 (33%) 4 (20%) 7 (27%) 7 (42%)
	Pattern and Position 12 (44%) 8 (40%) 5 (19%) 3 (18%)
	Association and Story 10 (37%) 8 (40%) 5 (19%) 12 (71%)
	Password re-use	1 (4%)	-	7 (27%) 4 (24%)
	Date	-	-	13 (50%) 8 (47%)

Acknowledgement The authors thank Christopher Krügelstein and Felix Kaiser for their assistance to this research. This work was partially funded by the German Federal Ministry of Education and Research (BMBF) under the project Softwarecampus, grant no. 01IS12056.