Melanie Volkamer

Karen Renaud
email: karen.renaud@glasgow.ac.uk

Benjamin Reinheimer

TORPEDO: TOoltip-poweRed Phishing Email DetectiOn

scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Phishing is merely a modern equivalent of a confidence trick that has been carried out for centuries: to deceive someone to derive some personal benefit. The first time that the term "phishing" was used to refer to this digital version of confidence tricking was on January 2, 1996 4 . Phishing messages offer a link embedded in an email message that entices the recipient to click. Email recipients are likely to click on links due to their widespread legitimate use. If they do click, it redirects them to a website masquerading as the real thing or downloads some malware onto their computer. Twenty years after its emergence, phishing still succeeds [START_REF] Erkkilä | Why we fall for phishing[END_REF][START_REF] Webroot | Threat Brief[END_REF].Automated detection is a powerful tool against phishing, but the fact that it takes, on average, 28.75 hours to detect new phish websites [START_REF]Global Phishing Survey: Trends and Domain Name Use in 2H2013[END_REF] means that users have to detect phishing messages themselves during the discovery window. However, many people are unable to distinguish legitimate from phish messages. Since there is no financial bar on the number of emails phishers can send, this means a sizeable number of people are snared every day.

The goal of our research was to propose a solution to reduce phishers' success in the email environment significantly (note, we studied the approach in the email environment but it could be easily adopted to other message formats such as Facebook and Twitter messages). To achieve this goal, we needed first to understand why people fall for phishing. We thus carried out a literature review and a cognitive walk-through analysis of emails as displayed by commonly-used desktop and webmail clients. Based on our findings, we proposed a concept called TORPEDO (TOoltip-poweREd Phish Email DetectiOn) to assist users by providing a just-in-time, just-in-place, trustworthy tooltips that display the actual URL with the domain highlighted in bold (see Fig. 1). Furthermore, we disable the link briefly, introducing a short delay, to increase the likelihood that people will check the URL before clicking on it. Finally, we provide users with an extended information diagram to explain the phish detection process. An evaluation delivered significant improvements (85.17% for phish, 91.57% for legitimate emails compared to 43.31% and from 63.66% when only providing the URL in the status bar, as Thunderbird does it for instance). We implemented a corresponding Thunderbird Add-On, as a proof-of-concept.

Identifying ons Why People Fall for Phish

It is important to understand why people fall for phish in order to support them the better. To identify possible reasons for people falling victim to phish, we carried out a literature review and conducted a cognitive walk-through analysis.

Literature Review

A phishing email contains a number of signals that may indicate that the email is a phish, the most reliable of which is the actual URL as explained in [START_REF] Garera | A framework for detection and measurement of phishing attacks[END_REF] and [START_REF] Ma | Beyond Blacklists: Learning to Detect Malicious Web Sites from Suspicious URLs[END_REF].

Many people do not realise this but, just in case they do, phishers routinely obfuscate the URL to dampen down this signal (e.g. using amazon.shop-secure.com to phish amazon accounts). Our literature review revealed a number of papers in which the authors showed that the reality is different since many people focus on other signals, applying various flawed heuristics, namely:

-The Sender: People are likely to trust emails from friends [START_REF] Jagatic | Social Phishing[END_REF] or from reputable businesses [START_REF] Xu | Victimized by Phishing: A Heuristic-Systematic Perspective[END_REF]. -The Look and Feel: People judge emails' trustworthiness based on their first impression, informed by a recognisable logo [START_REF] Blythe | F for fake: four studies on how we fall for phish[END_REF][START_REF] Naidoo | Analysing Urgency and Trust Cues Exploited in Phishing Scam Designs[END_REF], attractive design [START_REF] Naidoo | Analysing Urgency and Trust Cues Exploited in Phishing Scam Designs[END_REF][START_REF]The psychology of scams: Provoking and committing errors of judgement[END_REF], the use of their name or the provision of the company's contact details [START_REF] Jakobsson | What instills trust? A qualitative study of phishing[END_REF]. -The Email Content: People read the email in order to judge the trustworthiness thereof. Relied-upon indicators are the grammar and spelling quality [START_REF]The psychology of scams: Provoking and committing errors of judgement[END_REF][START_REF] Naidoo | Analysing Urgency and Trust Cues Exploited in Phishing Scam Designs[END_REF]. Researchers also showed that people are more likely to fall for a phish when: emotions such as excitement, fear or anxiety [START_REF] Blythe | F for fake: four studies on how we fall for phish[END_REF][START_REF]The psychology of scams: Provoking and committing errors of judgement[END_REF] are provoked, a sense of urgency is invoked [START_REF] Naidoo | Analysing Urgency and Trust Cues Exploited in Phishing Scam Designs[END_REF][START_REF] Stajano | Understanding scam victims: seven principles for systems security[END_REF], existing attitudes and beliefs (wanting to believe that the scammer is honest) are exploited [START_REF] Rusch | The "social engineering" of Internet fraud[END_REF], or persuasive and influencing techniques are used [START_REF]The psychology of scams: Provoking and committing errors of judgement[END_REF][START_REF] Wang | An empirical exploration of the design pattern of phishing attacks[END_REF]. Researchers argue that under such conditions of arousal people's decision-making abilities are impaired and they are less likely to pick up danger signals [START_REF] Vishwanath | Why do people get phished?Testing individual differences in phishing vulnerability within an integrated, information processing model[END_REF]. -Wrong Parts of the URL: Some people do look at the URL [START_REF] Jakobsson | What instills trust? A qualitative study of phishing[END_REF]. However, they misinterpret the URL due to a lack of knowledge of the semantics of URLs [START_REF] Dhamija | Why Phishing Works[END_REF][START_REF] Wu | Do security toolbars actually prevent phishing attacks? In CHI[END_REF]. Some people are reassured by the mere presence of HTTPS in the embedded link and look no further [START_REF] Friedman | Users' conceptions of web security: a comparative study[END_REF]. Others are reassured by the brand name being embedded 'somewhere in the URL' [START_REF] Jakobsson | What instills trust? A qualitative study of phishing[END_REF].

Cognitive Walk-through Analysis

For one week, we carefully considered emails that we received, examining the embedded URLs to identify possible challenges which could impair or encourage phish detection. We examined Thunderbird and Apple Mail as well as Web interfaces from three popular Web email clients. We made the following observations:

-Information not provided where expected/needed: Thunderbird5 as well as the Web email clients, display the actual URL destination in the status bar at the bottom of the window. Problem: The status bar is some distance away from the user's current attentional focus and might easily be missed. The text of the email is far more prominent and thus likely to be focused on. -Tooltip provided by sender: Some email senders provide a tooltip which appears when the mouse hovers over the link when using Thunderbird or the Web email clients6 while the actual URL is still displayed in the status bar. Providing such tooltip is actually a very simple attack since the phisher only needs to provide a "title=" attribute. Problem: The tooltip encourages examination of the URL by appearing where the user's attention is focused.

If the recipient relies only on the tooltip, a phisher would be successful when providing a reassuring URL. -Redirection: Some email providers seem to make phish detection difficult, if not impossible. These clients do not display the actual URL in the status bar, but rather an (obfuscated) URL, a so-called dereferer (see Fig. 2). Web mail providers argue that they do this to protect their users (due to some checks before redirecting users to the actual web page). Problem: This approach makes it almost impossible for even the most security aware to detect the perfidy of the link. Problem: It is likely that users are not cautioned enough and instead of only moving the mouse to the link they already click before having checked.

Reasons Why People Fall for Phishing

From the above findings, the following reasons can be deduced:

1. Not being aware that the URL is the only reliable signal: making a decision based on the wrong signal. 2. Not knowing which displayed URL to trust. There are three options: embedded in email text, in the displayed tooltip, or in the status bar. 3. Not having access to the actual URL (destination) due to URLs being obscured -either because of redirection or the use of tiny URLs. 4. Not consulting the URL carefully enough before clicking due to accidental clicks and/or habituation effects. 5. Not knowing how to distinguish authentic from phish URLs.

TORPEDO as possible Solution

We try to address all of these reasons with TORPEDO. TORPEDO proves justin-time and just-in-place trustworthy tooltips which contain the actual URL with the domain highlighted. It delays link activation for a short period. Furthermore, TORPEDO consists of an information diagram to explain phish detection, to be used together with the tooltips (when first used and on demand). We explain in the following paragraphs the different aspects and how they are supposed to address the identified reasons .

Just-in-time means that the tooltip appears when the person hovers their mouse over an embedded link. This addresses 'Reason 1' by making the reliable signal more prominent (at least compared to the status bar used by Thunderbird and the Web email clients). Just-in-place means that it appears right next to the link (i.e. more precisely right below the link) and only there. This addresses 'Reason 1' and 'Reason 2' by making the most important signal the most prominent one and always displaying it at the same position. Highlighting the domain in bold letters (similar to the highlighting in the addressbar of some Web browser) focuses attention on the most important part of the URL addressing 'Reason 5'.

Disabling the link for a short period , perhaps three seconds while providing continuous feedback in terms of a counter showing the time left to click (3, 2, 1s) increases the likelihood of people examining the link before clicking, addressing 'Reason 4'. Note, the delay is configurable to give users control. Furthermore, a white list is maintained to remember domains users have already clicked on twice before (requiring two clicks means that domains will not as easily be accidentally white listed). Whitelisted links will be activated immediately and not be subject to any delay to not annoy users.

Trustworthiness, first, requires overwriting tooltips provided by the email sender. This, together with the tooltip appearing just-in-time and just-in-place, addresses 'Reason 2'. It also addresses 'Reason 3' partially by providing the actual URL, instead of the obfuscated one the phisher wants the user to see. Fig. 3 (a) shows how we propose to handle the redirections ('redirectUrl=') aspect of 'Reason 3', i.e. providing the actual URL and informing the user that this is the second but final destination. There are two possibilities to address the 'tiny URLs'7 aspects of 'Reason 3': (1) automatically replace these URLs by the actual one using the service from http://longurl.org, or (2) inform users and let them decide whether to check for the actual URL using this service. From a usability point of view the first option looks more promising; however, from a security and privacy perspective the second one is more promising (as e.g. the tool would send requests although the user does not want to visit the corresponding page). We decided to go for option (2) by default but allowing to configure option [START_REF] Alnajim | An Anti-Phishing Approach that Uses Training Intervention for Phishing Websites Detection[END_REF]. Thus, users would first see the upper tooltip of Fig. 3 (b) and the other one once decided to check for the actual URL. -In Step 1, we suggest focusing only on the URL. This addresses the fact that people do not focus on the URL ('Reason 1'). It is also explained that the remaining parts of the URL can easily be faked. -In Step 2 we recommend that people actually only consider the highlighted part of the URL displayed in the tooltip. -In Step 3, we explain that they should check whether the brand name is highlighted (to address misleading URLs such as amazon.shop-secure.com but also obvious phishes such as IP addresses). More precisely, we explain that they should ignore the remaining parts of the URL. -In Step 4, we advise them to check for extensions of the brand name such as in amzon-shopping-in-America.com. This the most difficult phishing URL to detect as some companies use such extensions in their authentic domain. We, thus, recommend that they search at Google if they are not sure. -In Step 5, we recommend that they check letter by letter to identify small modifications in the domain name (to pick up mircosoft.com).

Evaluation

We wanted to evaluate TORPEDO's effectiveness, efficiency and user-engendered confidence in terms of properly judging the authenticity of emails, as compared to the status quo status bar display in Thunderbird and the considered Web email clients. To do so, we conducted a between-subjects online study launched on SoSciSurveywith participants randomly associated to one of two groups:

-Status bar: The group sees the URLs in the status bar.

-TORPEDO: The group sees the URL in a tooltip with domain highlighted in bold while having seen the information diagram.

Moreover, we formulated the following hypotheses:

-H1 -Phish detection: The TORPEDO group will detect more phishing emails, as compared to the status bar group. -H2 -Authentic eMail identification: The TORPEDO group will identify more authentic emails, as compared to the status bar group. -H3 -Efficiency: The TORPEDO group will judge emails more quickly, as compared to the status bar group8 . -H4 -Confidence: The TORPEDO group will be more certain of their judgements, as compared to the status bar group.

Study Procedure

The study comprised the following three phases9 : Phase 1 -Welcome: General information was provided, including the goal of the study, number of phases, the estimated duration, and data protection. We explained that it was important not to seek assistance (we did not elaborate by citing Google, as this could have been counter productive). We introduced the main tasks: They should imagine that their friend Max Müller is about to work through his emails. Since Max has accounts at all the companies in question, it is important for him to know which emails are authentic and which are phish. Therefore, they were asked to help him to judge the emails based on screenshots which Max provides to them on the following pages. Phase 2 -Judging screenshots: Participants were presented with screenshots of 16 emails (8 authentic / 8 phish) each on a different web service and in random order. The TORPEDO group got in addition the information diagram, both before starting to answer questions as well as below each screenshot. Participants were asked: Is the email authentic? Then participants were then asked: How certain are you that you properly judged the displayed emails. The TORPEDO group was also asked to comment on the information diagram. Phase 3: Demographics: We requested demographic information.

Creation of Email Screenshots

We selected 16 web service providers based on the degree of popularity based on Alexa (see Table 1). For all of these, we determined what authentic emails look like (including the 'from' address). All emails addressed the 'Heartbleed-Bug'. The text recommended that the recipient change their password and provided a link to facilitate this. The text slightly differed from one email to the next but the meaning remained the same. All raising some (but not strong) pressure to change the password. Then, half of the screenshots were 'turned into' a screenshot of a phish email by modifying the URL. For the TORPEDO group a tooltip was added to display the relevant link. We decided to simulate a worst-case scenario, i.e. advanced phishing emails which can only reliably be detected by checking the URL. We wanted to investigate the difference between the status bar and tooltip, and not the impact of various different signals. All emails were personalised. We also used HTTPS for both phish and non-phish displays because we did not want the absence or presence of HTTPS to constitute a cue due to the findings in Section 2.1. Next, we considered which URL manipulation techniques to apply to get a representative set of manipulated URLs. Researchers have identified different URL manipulation classifications [START_REF] Canova | NoPhish App Evaluation: Lab and Retention Study[END_REF][START_REF] Garera | A framework for detection and measurement of phishing attacks[END_REF][START_REF] Lin | Does domain highlighting help people identify phishing sites? In CHI[END_REF]. We used the categories from [START_REF] Canova | NoPhish App Evaluation: Lab and Retention Study[END_REF] with each type's anticipated success depending on how well users understand URLs and the thoroughness of their URL checking:

-Obfuscate: The phish URL is composed of an arbitrary name or IP address. Note, the brand name of the authentic website does not appear. -Mislead: The phish URL embeds the authentic name somewhere (e.g. in the subdomain or the path) in order to allay suspicions. -Mangle: The phish URL includes letter substitutions, different letter ordering, or misspelling e.g. arnazon instead of amazon. -Camouflage: The domain name of the phish URL contains the brand name together with an extension or a different top level domain.

Ethics, Recruitment, and Incentives

Our University's ethical requirements with respect to respondent consent and data privacy were met. Participants first read an information page on which they were assured that their data would not be linked to their identity and that the responses would only be used for study purposes. Furthermore, using SoSciSurvey ensured that data was stored in Germany and thus subject to German data protection law. No debriefing was necessary. We recruited participants through a platform called Workhub, which is a German equivalent of Amazon Mechanical Turk. Every Workhub participant receives e3.

Results and Discussion

The demographics are summarized for both groups in versus 67.44%. Furthermore, the corresponding numbers for the answer 'I do not know' are (TORPEDO/status bar): 3.49%/6.98%, 2.91%/7.56%, 2.33%/8.14%, and 5.81%/5.23%. The descriptive data for H3 and H4 is depicted in Fig. 4.

As to the violation of homogeneity of variances for the compared groups we started our analyses with Mann-Whitney U tests for every hypothesis supplemented with an approximated effect size. H1 -Phish Detection: Our analysis shows a significantly improved detection rate for phish Emails for participants in the TORPEDO group, as compared to those in the statusbar group (U = 210, p < .001, η 2 = 0.455).

H2 -Authentic Email Identification: Our analysis shows a significantly improved identification rate of authentic Emails for participants in the TOR-Fig. 4: Descriptive data for timing and certainty of correct decision.

PEDO group, as compared to those in the status bar group (U = 304, p < .001, η 2 = 0.374).

H3 -Efficiency: Our analysis shows that participants in the TORPEDO group were significantly more efficient than participants in the status bar group (U = 676.5, p = .032, η 2 = 0.053).

H4 -Confidence: Our analysis shows that participants in the TORPEDO group were significantly more certain about their decisions than participants in the status bar group (U = 536.5, p < .001, η 2 = 0.155).

Diagram Feedback.

We used open coding to analyse the free text answers. We came up with the following categories: 'grateful', 'nothing to improve', 'confusing', 'too much information', 'too little information', and 'small improvements'. Most of the participants (29 from 43 in total) were happy with the diagram, not mentioning anything to improve. Three were grateful. Eight mentioned that the diagram was confusing and five added that the confusion cleared once they read it. Two participants considered the diagram to contain too much information while another two participants complained about it too little information (requesting more examples). Three provided small suggestions for improvement: provide a title, and reconsider the usage of terms such as phish and URL. Discussion. The results show that, in the studied scenario, we significantly improved phish detection as well as the identification of legitimate emails with TORPEDO. The detection rates for all four phishing types increased, too. The participants in the TORPEDO group are also more confident that they made the proper decision in comparison to the status bar group. The decision making process is also significantly more efficient. Operating more quickly can, in the long run, lead to more errors being made. It is worth providing people with information such as that given in the information diagram since it is easy to apply and requires the email recipient to spend less time checking each individual email. The feedback to the information diagram showed that there is not much need to improve the diagram other than making the numbers clearer and adding a title. We acknowledge that the diagram might not provide sufficient information for some people. In these few cases, we recommend extending our defence approach with existing proven training approaches (see Section 6).

Limitations. We acknowledge that we evaluated the approach in a best-case scenario as their primary task was security. Phishing effectiveness evaluations in field studies are not possible due to ethical and legal constraints. Lab studies also have their limitations because participants would not use a study laptop instead of their own. We also acknowledge that the URLs were displayed the entire time and not only when hovering the mouse over the link. This only partially simulates the proposed delay. Note, we only used HTTPS since we wanted to assess their ability to check the actual URL, not the presence or absence of HTTPS. Finally, we acknowledge that the sample was not representative.

Related Work

Researchers have proposed different ways of addressing phishing:

Automated Detection. Phishing emails can be detected either pre-or post-click. Emails can be analysed by the email provider before being forwarded to the user. This analysis includes checking the integrated URLs against several blacklists provided by companies such as Microsoft, Google and phishTank. Other checks can also be carried out. For example, to look at differences between displayed and actual domain names [START_REF] Fette | Learning to detect phishing emails[END_REF] or carry out an NLP analysis of the actual email text [START_REF] Verma | Detecting phishing emails the natural language way[END_REF]. If the email is delivered and the person clicks, post-click detection can also occur. Web browsers or Add-ons can check the URL against various blacklists or check the web site content in combination with the actual URL. A number of different approaches for these checks have been proposed [START_REF] Marchal | Proactive discovery of phishing related domain names[END_REF][START_REF] Bar-Yossef | Do not crawl in the DUST: Different URLs with similar text[END_REF][START_REF] Maurer | Using Visual Website Similarity for Phishing Detection and Reporting[END_REF][START_REF] Prakash | PhishNet: Predictive Blacklisting to Detect Phishing Attacks[END_REF]. In both pre-and post-click checks a risky situation can either lead to blocking or a warning e.g. [START_REF] Li | Usability evaluation of anti-phishing toolbars[END_REF][START_REF] Maurer | Using data type based security alert dialogs to raise online security awareness[END_REF][START_REF] Zhang | Phinding Phish: Evaluating Anti-Phishing Tools[END_REF][START_REF] Wu | Do security toolbars actually prevent phishing attacks? In CHI[END_REF]. As a final comment, there is an inherent flaw to post-click warnings. The human tendency to consistency makes it less likely for people to even want to detect the deceptive nature of any site if they have already committed to the process [START_REF] Cialdini | Low-ball procedure for producing compliance: commitment then cost[END_REF]. They have judged the email to be legitimate. Withdrawing at this stage is unlikely. TORPEDO does not aim to replace detection approaches but to complement them in order to help people to protect themselves in case none of the checks detects the phish or it is simply during the pre-detection window [START_REF]Global Phishing Survey: Trends and Domain Name Use in 2H2013[END_REF].

Training. A number of researchers have focused on training users to spot phish [START_REF] Alnajim | An Anti-Phishing Approach that Uses Training Intervention for Phishing Websites Detection[END_REF][START_REF] Canova | NoPhish: An Anti-Phishing Education App[END_REF][START_REF] Canova | Learn to Spot Phishing URLs with the Android NoPhish App[END_REF][START_REF] Jansson | Simulating malicious emails to educate end users on-demand[END_REF][START_REF] Kirlappos | Security Education against Phishing: A Modest Proposal for a Major Rethink[END_REF][START_REF] Kumaraguru | Protecting People from Phishing: The Design and Evaluation of an Embedded Training Email System[END_REF][START_REF] Kumaraguru | Getting Users to Pay Attention to Anti-phishing Education: Evaluation of Retention and Transfer[END_REF][START_REF] Sheng | Anti-Phishing Phil: The Design and Evaluation of a Game That Teaches People Not to Fall for Phish[END_REF] but most of them address phish detection in a web browser context. Researchers have shown that training improves phish detection rates. Training has two drawbacks as compared to TORPEDO. First, people need to be aware that there is a problem and that they need to be active in dealing with it. Otherwise they will not undergo training. This problem was addressed by Kumaraguru [START_REF] Kumaraguru | Teaching Johnny Not to Fall for Phish[END_REF] by employing the concept of teachable moments, where people are given instructions or training when they almost fall for a phish. In their scheme, instead of blocking a link they allow it, and then show them that they almost fell for a phish. Second, people may forget the information the training imparted as they are not confronted with phishing emails every day.

Again the teachable moment approach can help. However, we think providing the information graphic on demand, as and when required, is the safer approach as it might be that the next time they forget how to check the teachable moment mechanism may not be installed and they would be unprotected. Dodge et al.

[10] report a different approach, post-click training. They send out fake phish messages and then train people who click on the links. They report a positive effect. However, this approach can only be taken within an organisation.A few participants in our evaluation had trouble understanding our diagram. For those, more exhaustive training may help them. Note, the training, as such, would be shorter than what would be required without TORPEDO.

Combining Approaches. We propose TORPEDO to complement existing approaches to address the email phishing problem. Other researchers have also proposed combining approaches to maximise phish protection. Khonji et al. [START_REF] Khonji | Phishing detection: a literature survey[END_REF] suggest a two-pronged approach, the first prong being user training, and the second being automated detection. The latter includes blacklists, machine learning and visual similarity detection. Frauenstein and Von Solms [START_REF] Frauenstein | Phishing: How an Organization can Protect Itself[END_REF] propose combining human, organisational and technical measures. The first includes awareness and training, the second policies and procedures and the last one includes automated measures to detect phish.

Conclusion and Future Work

Phishing is a thorny issue. Trying to filter out phishing emails before they reached the end users reduces the problem. Great strides have been made in this direction but no one will claim that any automated system will catch 100% of phishing emails. So, it is up to the end users to protect themselves. The research we have presented here offers a way to support end users by deploying TORPEDO providing just-in-time, just-in-place, trustworthy tooltips; disabling links for a short period of time; detection of re-directions and tiny URLs, and providing a diagram at installation, or on demand, that encapsulates phish detection advice in a step by step fashion. This approach was evaluated and improved. We found that it highly significantly improved phish and legitimate email detection, made such detection significantly faster and led to people feeling more confident about their judgements compared to the status bar approach as used in Thunderbird and Web email clients. With 85.17% phish detection compared to 43.31% with the status bar URL display, it can only be hoped that the different email clients and email providers deploy this approach as soon as possible. Until then, people can use the TORPEDO Add-on we developed. As future work, we plan to conduct acceptance tests to determine whether TORPEDO will indeed be used. We also plan to extend this approach to mobile email clients.

Fig. 1 :

 1 Fig. 1: Just-in-time, just-in-place, trustworthy tooltips as shown in the upper-left part. The entire figure is displayed when more information is requested.

Fig. 2 :

 2 Fig. 2: Status bar displays an obfuscated URL for an embedded URL

  Fig. 3: Example tooltips

Table 2 .

 2 Participants in the TORPEDO group, on average, detected phishing emails 85.17% of the time and they, on average, identified legitimate emails as such 91.57% of the time. The corresponding percentages for the control group are: 43.31% for phish

Table 1 :

 1 Legitimate(L) and Manipulated(M) URLs incl. type of manipulation. detection and 63.66% for identifying legitimate emails. Note, participants, on average, detected Camouflaged URLs 72.09% of the time in the TORPEDO group and 38.37% in the status bar group, Misleading URLs 87.21% versus 30.23%, Mangled URLs 91.86% versus 37.20%) and Obfuscated URLs 89.53%

	Brand	URL (abbreviated with '...')
	Postbank	L: https://banking.postbank.de/rai/login
	Ebay	L:https://signin.ebay.de/ws/eBayISAPI.dll?SignIn&UsingSSL...
	Xing	L: https://login.xing.com/login?dest_url=https\%3A\%2F\%2Fwww...
	Google	L: https://accounts.google.com/login?hl=de
	Dropbox	L: https://www.dropbox.com/s/VPrize8EppElIxxWOwETRB87 Pe733AR...
	Telekom	L: https://accounts.login.idm.telekom.com/oauth2/auth?response...
	Zalando	L: https://www.zalando.de/login/
	MediaMarkt	L: https://www.mediamarkt.de/webapp/wcs/stores/servlet/Logo...
	Facebook	L: https://www.facebook.com/login
	(Obfuscate)	M: https://130.83.167.26/login
	Flickr	L: https://login.yahoo.com
	(Obfuscate)	M: https://www.xplan.com/signing/flickr/
	Twitter	L: https://twitter.com/login
	(Mislead)	M: https://twitter.webmessenger.com
	Amazon	L: https://www.amazon.de/ap/signin
	(Mislead)	M: https://www.amazon.de.buecherkaufen.de/ap/signing?...
	DeutscheBank L: https://www.deutsche-bank.de
	(Mangle)	M: https://meine.cleutsche-bank.de/trxm/db/i nit.do?login...
	Maxdome	L https://www.maxdome.de/
	(Mangle)	M: https://www.maxdorne.de/?fwe=true&force-l ogin-layer=true
	Paypal	L: https://www.paypal.com/signin/?country.x=DE&locale...
	(Camouflage)	M: https://www.paypalsecure.de/webapps/mpp/ho me
	GMX	L: https://www.gmx.net
	(Camouflage)

M: https://meinaccount.gmxfreemail.de/

Table 2 :

 2 Demographics# Participants Average Age Median Youngest Oldest Male IT Expert

	Status bar	43	25.70	23	17	54	25	5
	TORPEDO	43	27.86	26	18	60	25	2

The mention occurred in alt.online-service.america-online.

Note, this is different for Apple Mail and also for Outlook.

Again, this is different for Apple Mail and for Outlook.

According to Wikipedia popular shortening services are: bit.ly, goo.gl, ow.ly, t.co, TinyURL, and Tr.im. The URL is parsed accordingly. Those services are addressed.

Note, on the one hand it is important that people take their time to check the URL, however in addition, if they know what to consider, they can make decisions faster.

Questions were translated from German for this paper.

Acknowledgement. This work was developed within the project 'KMU AWARE' which is funded by the German Federal Ministry for Economic Affairs and Energy under grant no. BMWi-VIA5-090168623-01-1/2015. The authors assume responsibility for the content.


Hongzhe Li
email: hongzhe@korea.ac.kr

Jaesang Oh
email: jaesangoh@korea.ac.kr

Hakjoo Oh
email: hakjoooh@korea.ac.kr

Heejo Lee

Automated Source Code Instrumentation for Verifying Potential Vulnerabilities ⋆

Keywords: automatic instrumentation, security sinks, security constraints, vulnerability verification

With a rapid yearly growth rate, software vulnerabilities are making great threats to the system safety. In theory, detecting and removing vulnerabilities before the code gets ever deployed can greatly ensure the quality of software released. However, due to the enormous amount of code being developed as well as the lack of human resource and expertise, severe vulnerabilities still remain concealed or cannot be revealed effectively. Current source code auditing tools for vulnerability discovery either generate too many false positives or require overwhelming manual efforts to report actual software flaws. In this paper, we propose an automatic verification mechanism to discover and verify vulnerabilities by using program source instrumentation and concolic testing. In the beginning, we leverage CIL to statically analyze the source code including extracting the program CFG, locating the security sinks and backward tracing the sensitive variables. Subsequently, we perform automated program instrumentation to insert security probes ready for the vulnerability verification. Finally, the instrumented program source is passed to the concolic testing engine to verify and report the existence of an actual vulnerability. We demonstrate the efficacy and efficiency of our mechanism by implementing a prototype system and perform experiments with nearly 4000 test cases from Juliet Test Suite. The results show that our system can verify over 90% of test cases and it reports buffer overflow flaws with P recision = 100%(0 FP) and Recall = 94.91%. In order to prove the practicability of our system working in real world programs, we also apply our system on 2 popular Linux utilities, Bash and Cpio. As a result, our system finds and verifies vulnerabilities in a fully automatic way with no false positives.

Introduction

Even though security experts are making best efforts to ensure the software security, the number of software vulnerabilities is still increasing rapidly on a yearly basis, leaving great threats to the safety of software systems. According to the Common Vulnerabilities and Exposures(CVE) database [START_REF]Common Vulnerabilities and Exposures(CVE)[END_REF], the number of CVE entries has increased from around 1000 CVEs yearly in 2000 to over 8000 yearly in 2015. The discovery and removal of vulnerabilities from software projects have become a critical issue in computer security. Nowadays, because of enormous amount of code being developed as well as limited manpower resource, it becomes harder and harder to audit the entire code and accurately address the target vulnerability.

Security researchers have devoted themselves into developing static analysis tools to find vulnerabilities [START_REF] Wheeler | Flawfinder, Access[END_REF]. The large coverage of code and access to the internal structures makes these approaches very efficient to find potential warnings of vulnerabilities. However, they often approximate or even ignore runtime conditions, which leaves them a great amount of false positives.

Recently, more advanced static analysis methods are proposed [START_REF] Yamaguchi | Chucky: exposing missing checks in source code for vulnerability discovery[END_REF] [5] [START_REF] Perl | Vccfinder: Finding potential vulnerabilities in open-source projects to assist code audits[END_REF]. They either encode insecure coding properties such as missing checks, un-sanitized variables and improper conditions into the analyzer for vulnerability discovery, or they model the known vulnerability properties and generate search patterns to detect unknown vulnerabilities. Even though these approaches can find vulnerabilities using search patterns and exclude the majority of code needed to be inspected, they still require security-specific manual efforts to verify the vulnerability at the very end, which is neither efficient for vulnerability discovery nor feasible for non-security specialists to use it.

According to the previous efforts of researchers, finding exact vulnerabilities in a fully automatic way has been challenging. To automate the overall process of vulnerability detection and verification, we classify the potential vulnerable security sinks into basic 4 types and apply security constraint rules(corresponding to each type) to automatically instrument vulnerability triggering probes into the source code in order to verify vulnerabilities. In this paper, we propose an automatic mechanism to detect and verify software vulnerabilities from C code by using program source instrumentation and concolic testing. In the beginning, we leverage CIL [START_REF] Necula | CIL: Intermediate language and tools for analysis and transformation of C programs[END_REF](C intermediate language) to statically analyze the source code including extracting the program CFG(control flow graph), locating the sensitive security sinks and backward tracing the sensitive variables. Subsequently, we classify the security sinks into 4 basic types and perform automated program instrumentation to insert security probes according to different properties of the security sink types, ready for the vulnerability verification. Finally, the instrumented program source is passed to the concolic(CONCrete + sym-bOLIC) testing engine [START_REF] Kim | Industrial application of concolic testing on embedded software: Case studies[END_REF] [START_REF] Cadar | Klee: Unassisted and automatic generation of high-coverage tests for complex systems programs[END_REF] to report and verify the existence of an actual vulnerability. We here focus on buffer overflow vulnerabilities since this type of vulnerability is the major cause for malicious intensions such as invalid memory access, denial of service(system crach) and arbitrary code execution. We demonstrate the efficacy and efficiency of our mechanism by implementing a prototype system and perform experiments with 4000 buffer overflow test cases from Juliet Test Suite [START_REF] Boland | Juliet 1.1 C/C++ and Java Test Suite[END_REF]. The results show that our prototype system gets a detection result with P recision = 100% and Recall = 94.91%. In order to prove the practicability of our system working in real world programs, we also apply our mechanism on Linux utilities such as Bash and Cpio. As a result, our system finds and verifies vulnerabilities in a fully automatic way with no false positives.

Main contributions of our study are described as follows:

-Fully automated verification for vulnerability discovery. We propose, design and implement a fully automated verification mechanism to detect and verify vulnerabilities with zero interference of manual efforts, which can expand the system usability to more general users such as non-security specialist.

-Memory space analysis(MSA) for verifying security requirements. We verify the existence of vulnerabilities by generating triggering inputs which violate security constraints(SC). The memory space analysis(MSA) enables us to track the size of buffer space at runtime and set the SC conditions accurately. It decreases the false positives for vulnerability verification.

Related work

Source code auditing have been actively researched by security researchers for software assurance and bug finding. Previous researchers have proposed different approaches for static source code auditing and have developed source code static analysis tools for vulnerability discovery.

Flawfinder [START_REF] Wheeler | Flawfinder, Access[END_REF] applies a pattern matching technique to match the security sinks in the source code and report them as vulnerabilities. Even though these approaches can analyze large amount of source code and report vulnerabilities fast, they generate too many false positives due to a lack of analysis about program data flow and control flow information.

Chucky [START_REF] Yamaguchi | Chucky: exposing missing checks in source code for vulnerability discovery[END_REF] statically taints the source code and identifies missing conditions linked to security sinks to expose missing checks in source code for vulnerability discovery. VCCFinder [START_REF] Perl | Vccfinder: Finding potential vulnerabilities in open-source projects to assist code audits[END_REF] proposes a method of finding potentially dangerous code with low false positive rate using a combination of code-metric analysis and meta-data from code repositories. It also trains a SVM classifier by the vulnerability commits database to flag code as vulnerable or node. Yamaguchi et al. [START_REF] Yamaguchi | Automatic Inference of Search Patterns for Taint-Style Vulnerabilities[END_REF] models the known vulnerability properties and generate search patterns for taint-style vulnerabilities. The generated search patterns are then represented by graph traversals which is used for vulnerability mining in a code property graph database [START_REF] Yamaguchi | Modeling and discovering vulnerabilities with code property graphs[END_REF]. Even though these approaches can find vulnerabilities using search patterns and exclude the majority of code needed to be inspected, they still require security-specific manual efforts to verify the vulnerability at the very end, which is neither efficient for vulnerability discovery nor feasible for general users(nonspecialist) to use it. Driven by this, there is an urgent need to build a fully automatic system to accurately catch vulnerabilities within reasonable time as well as the expansion of usability to more general users.

Based on the weakness of the above discussion, we are looking into an automatic and efficient way to do vulnerability verification. Symbolic execution has been proposed to do program path verification but it cannot resolve complex programs with enormous amount of path constraints [START_REF] Zhang | Detecting vulnerabilities in c programs using trace-based testing[END_REF]. Concolic testing [START_REF] Kim | Industrial application of concolic testing on embedded software: Case studies[END_REF] [START_REF] Cadar | Klee: Unassisted and automatic generation of high-coverage tests for complex systems programs[END_REF] was proposed to improve symbolic execution in order to make it more practical in real world programs. KLEE [START_REF] Cadar | Klee: Unassisted and automatic generation of high-coverage tests for complex systems programs[END_REF] was developed to automatically generate high-coverage test cases and to discover deep bugs and security vulnerabilities in a variety of complex code. CREST-BV [START_REF] Kim | Industrial application of concolic testing on embedded software: Case studies[END_REF] has shown a better performance than KLEE in branch coverage and the speed of test case generation. However, these approaches suffer from path explosion problem which stops them from scaling to large programs.

CLORIFI [START_REF] Li | CLORIFI: software vulnerability discovery using code clone verification[END_REF] is the closest research to this paper. It proposes a method to detect code clone vulnerabilities by the combination of static and dynamic analysis. However, it has not been fully automated and it still requires manual efforts to do source instrumentation for concolic testing, which is still a tedious task. In this paper, we propose, design and implement a fully automated verification mechanism to detect and verify vulnerabilities. In our mechanism, we do vulnerability verification using concolic testing after an automated sink detection and source code instrumentation process, which reduces false positives. We also applies the runtime buffer memory space analysis(MSA) to track the real size of a buffer space which helps us to improve the accuracy of vulnerability discovery.

Proposed Mechanism

Discovery of vulnerabilities in a program is a key process to the development of secure systems. In order to find exact vulnerabilities in a fast and accurate way and to reduce the tedious manual efforts for vulnerability verification, we propose a fully automated mechanism to detect and verify software vulnerabilities by taking advantage of both static analysis and concolic testing.

Before we go into detailed description of our approach, the general process is illustrated in Fig. 1. Our mechanism mainly consists of 3 phases which are code transformation, automated instrumentation, and vulnerability verification. In the phase of code transformation, we first leverage the library of CIL to parse the source code into CIL program structures such as function definitions, variables, statements, expressions and so on, and calculate the control flow graph(CFG) information of each function. The reason why we do CIL code transformation is to simplify code structures for efficient static analysis. We then identify the security sinks(potential vulnerable) to get potential vulnerable points. In the second phase, we apply backward data tracing on sensitive variables of each sink to find the variable input location. Then, we perform automatic program instrumentation and prepare the testing object for vulnerability verification. In the last phase of automated instrumentation, we verify each potential security sink to report vulnerabilities using concolic testing.

Using the CIL

CIL (C Intermediate Language) [START_REF] Necula | CIL: Intermediate language and tools for analysis and transformation of C programs[END_REF] is a high-level representation along with a set of tools that permit easy analysis and source-to-source transformation of C programs. The reason why we use CIL is that it compiles all valid C programs into a few core constructs without changing the original code semantics. Also CIL has a syntax-directed type system that makes it easy to analyze and manipulate C programs. Moreover, CIL keeps the code location information which enables us to pinpoint the exact location when reporting vulnerabilities(an example is shown in Fig. 5).

By using the provided APIs of CIL, we are able to flatten complex code structures into simple ones(e.g., all looping constructs are reduced to a single form, all function bodies are given explicit return statements). Subsequently, we extract and calculate the control flow graphs(CFGs) of each function using some wrapped module of CIL. Each created node in CFG corresponds to a single instruction in the source code, referred from Sparrow [START_REF] Oh | Selective context-sensitivity guided by impact pre-analysis[END_REF], a stable and sound source code analysis framework. Treating the most basic code unit(instruction level) as a CFG node can help us precisely and rapidly address the sensitive sinks and variables as well as providing convenience for the backward sensitive data tracing which will be explained in detail in the following sections.

Identification of security sinks and sensitive variables

Since most of buffer overflow vulnerabilities are caused by attacker controlled data falling into a security sink [START_REF] Di | Sinks: Dom Xss Test Cases Wiki Project[END_REF], it is crucial to first dig out security sinks. In this paper, we focus on the security sinks which often lead to buffer overflow vulnerabilities. Before that, we explain the definition of security sinks and how they can be classified according to argument properties.

Security sinks : Sinks are meant to be the points in the flow where data depending from sources is used in a potentially dangerous way. Typical securitysensitive functions and memory access operations are examples of security sinks. Several typical types of security sinks are shown in Table 1.

As we can see from the Table 1, basically, security sinks are either security sensitive functions or a buffer memory assignment operation by index. Further more, according to the number of arguments and whether there is a format string("%s") argument inside a security sensitive function, we classify the security sensitive functions into 3 types in order to generalize the automatic process of backward tracing and program instrumentation which will be explained in following parts. Along with the last type(buffer assignment), we classify the security sinks of buffer overflow vulnerability into 4 types.

-Type 1: Security sensitive functions with 2 arguments: a destination buffer pointer(dst) and a source buffer pointer(src). The typical argument format is: fn(dst,src) and the sensitive variable is the 2nd variable(src) in the argument list. The instances of sinks include: strcpy, wcscpy, strcat and wcscat.

dstbuf[i] = 'A'
After the classification of security sinks, we identify the security sinks as potential vulnerabilities using a fast pattern matching approach over the CIL structures of the source code and extract sensitive variables needed to be backwardly traced in the following step based on the table above.

Backward data tracing

Since instrumentation points are needed before performing automated instrumentation, we propose backward data tracing to find the program input place and treat it as an instrumentation point. Backward data tracing finds the program input location for the corresponding sensitive variables in the sink which reduces the whole input search space of a program. This will help us greatly improve the efficiency for the vulnerability verification. We perform intra-and inter-procedure backward tracing based on the nodes of the control flow graph extracted from CIL. Fig. 2 shows the concept of intra-procedure and interprocedure backward tracing respectively.

Concepts and definitions.

As shown in Fig. 2(1), starting from a security sink, we begin to backwardly trace the corresponding sensitive variable until we reach the source of the sensitive data. In order to understand the process of backward tracing, there are several terms that we need to know.

Source : Source is the original definition point of a variable. It is the node where the value of the variable does not depend on any other variable. For instance, the starting points where un-trusted input data is taken by a program. The Source is one the following 2 cases.

v 0 = gets(); Assignment node where the left variable is assigned by a user input function such as gets(), scanf() and read(). We also maintain a user input function list. v 0 = 5; Assignment node where the left variable is assigned by a constant.

Ancestor:

The ancestor A of a node N is described as: the traced sensitive data of N gets its value from A. Ancestor nodes are intermediate nodes while sensitive data gets propagated. The ancestor node of a certain variable v 0 could be one of the 4 cases below:

v 0 = expression(v 1 ); Node where variable assigned by expression v 0 = f (m, n, ...); Node where variable assigned by function return value f (v 0 ); Node where variable assigned inside a function call void f (char v 0 ); Node for function declaration

The description of procedure. As shown in Fig. 2, the intra-procedure backward tracing starts from the initial sensitive variable in the security sink such as sink(x) in Fig. 2(1) and recursively find its ancestor and identify a new variable needed to be traced. The intra-procedure backward tracing stops when the current node is the Source of the traced sensitive variable(whole backward tracing also stops) or it stops when the current node is the function entry node. In the later case, Source cannot be found in the current function and the data flow comes from argument passing of the function call, so we need further do inter-procedure backward tracing to find the Source of the sensitive variable. The inter-procedure tracing(see Fig. 2(2)) starts from intra-procedure tracing. It then checks the return node of intra-procedure backward tracing. If the node is Source, the procedure returns and backward tracing ends. If the node is the function entry node, the procedure finds out all the call sites of the current function and identifies the corresponding sensitive variable in each call site. Then it applies intra-procedure backward tracing on sensitive variables in each call site as a new starting point. The whole backward tracing stops and exits when the Source of the sensitive variable in the security sink is found.

Program source instrumentation

After backward tracing, we get security sinks and Sources of the corresponding sink and store them into a list of sinks(sink list) and a list of Sources(source list) accordingly. We also establish a sink-source mapping between the 2 lists which helps us to correctly find the corresponding source for a certain sink. To instrument the program source, we make security probes(assertions) based on our predefined security requirements(Table 2) right before the security sink and replace the source input statement with symbolic values. To automate the overall process of source code instrumentation, we generalize the security constraint rules for the 4 basic types of vulnerability to automatically instrument bug triggering probes into the source code in a more generalized way.

Program Constraints(PC) and Security Constraints(SC): Program constraints are generated by following a specific branch according to conditional statements in the program. Program inputs which satisfy a set of program constraints are meant to execute a certain path of the program. Security constraints are clearly high-level security requirements, which is also used as our baseline to make security probes before the security sinks. For instance, the length of the string copied to a buffer must not exceed the capacity of the buffer. We define security requirements for security sinks such us security-sensitive functions and buffer assignment with index based on the condition that related arguments must satisfy to ensure the security of software systems. In our work, we have collected 14 library functions from Juliet Test Suite which are well known to be "insecure" for buffer overflows as security-sensitive functions and generated security requirements for them. Table 2 shows part of our predefined security constraints for security sinks. When there are inputs which satisfy program constraints but violates security constraints(P C ∧ SC) at a certain point during the execution, the program is considered to be vulnerable. To suggest a way to extend the method to cover other types of vulnerabilities, we will investigate the vulnerability features and define more sinks and the corresponding security requirements for vulnerability types such as integer overflows, format strings and divide-by-zeros.

Instrument input location and the security sink:

To instrument the input location, we first iterate over all the element in the source list, if the current Source takes the data from user input such as command line, network data, or a file, we replace the user input data with symbolic input values. For example, "a = readf ile(); " will be replaced by "a = sym input(); ".

To instrument the security sink, we insert an assertion statement right before the security sink based on the security rules defined in Table 2. For example, before the sink strcpy(dst, src), we insert an assertion statement assert(dst.space > strlen(src)). However, there is a problem here. We can easily get the length of a string by using "strlen()" function(C library function), but the space of a buffer is hard to be determined at runtime as well as at compile time. Someone may say, we can always get the size of a buffer by "sizeof ()". This is not correct when we measure the size of the buffer memory space. E.g., if we want to get the real space that dst points to, we cannot use "sizeof (dst)" because it will always return the actual size of a pointer itself which is the number 4 at 32 bit architectures. In order to get the real buffer size of a buffer pointer, we propose a pointer analysis approach to correctly get the buffer size at program runtime.

Memory Space Analysis(MSA) at runtime: As we can see from Table 2, in the security requirement rules, we have to get the "buffer.space" value to accurately set the bug triggering condition and instrument the sinks. However, the common static analysis of source code cannot get the runtime updating information about memory size of a buffer pointer. This usually will result in an inaccurate assertion of SC violating condition, which makes the system generate possible false positives when reporting vulnerabilities. The runtime memory space analysis enables us to accurately track the real size of a pointer's buffer space and helps us to correctly instrument the program so as to ensure high accuracy of vulnerability verification. We make a library called "libmysizeof" and it provides 3 major functions: mysizeof store(), mysizeof propagate() and mysizeof getspace(). We insert mysizeof functions in the corresponding place in the source code. The steps are shown below:

-Iterate over all the instructions in the source code and identify buffer allocation statement such as buf = malloc(n); and char buf[100];. Then, we store the buffer pointer name and the corresponding size in a global map M by inserting mysizeof store() function at the current location. -Identify each pointer propagation instruction such us p 2 = p 1 + 5;. The mysizeof propagate() will propagate the size of p 1 to get the size of p 2 according to the operation and store the size of p 2 into the map M. We then insert this mysizeof propagate() function at the current location. -When we need to get the runtime size of a pointer's buffer space, we insert mysizeof getspace() function at a certain location in the source code to get the correct buffer space.

After inserting "mysizeof" functions, we can get the accurate size of a pointer's buffer space at runtime. The buffer size information can then be used in the assertions. For instance, assert(mysizeof getspace(dst) > strlen(src)). Fig. 3 shows an example of instrumenting pointer analysis functions along with input and sink instrumentation.

Until here, we prepare a testing source object from the program input to the potential vulnerable sinks. This object is usually a small part of the whole program source which helps us to release the burden of next stage.

Vulnerability verification using concolic testing

We apply concolic testing in our mechanism to verify the potential vulnerabilities. The general principle of the verification is to find inputs which satisfy all the program constraints(PCs) but violate the security constraints(SCs) as shown in Fig. 1. Symbolic execution and concolic execution have been widely used in software testing and some have shown good practical impact, such as KLEE [START_REF] Cadar | Klee: Unassisted and automatic generation of high-coverage tests for complex systems programs[END_REF] and CUTE [START_REF] Sen | Cute: a concolic unit testing engine for C[END_REF]. However, they suffer from path explosion problem which makes them cannot scale well to large real world programs. In our scenario, the backward Fig. 3: Automatic instrumentation.

tracing module helps us to find the program inputs which are related to the sensitive data instead of the whole program input space. This can mitigate the path explosion problem mentioned before. Our approach for concolic testing to verify potential vulnerabilities mainly follows a general concolic testing procedure [START_REF] Kim | Industrial application of concolic testing on embedded software: Case studies[END_REF]. However, the difference is that we focus on generating an input to execute the vulnerable branch instead of generating inputs to traverse every possible paths of the program. Hence, it is more cost efficient when doing concolic testing.

Experimental Results

Implementation

System prototype: We have implemented our mechanism by developing a prototype system. Our system consists of 3 phases: code transformation, automated instrumentation, and vulnerability verification. Its architecture is described in Fig. 4. The system is used to discover buffer overflow vulnerabilities in software projects. 1Environment setup : We performed all experiments to test our automatic vulnerability detection and verification system on a desktop machine running running Linux Ubuntu 14.04 LTS (3.3 GHz Intel Core i7 CPU, 8GB memory, 512GB hard drive).

Dataset : For the experiment, we prepared 2 kinds of datasets -Juliet Test Suite and Linux utilities. First one is Juliet Test Suite provided by US National Security Agency(NSA) which has been widely used to test the effectiveness of vulnerability detection tools. To test our tools, we prepared 3,969 files for stack based buffer overflow vulnerabilities, each of which belongs to 4 basic types based on our sink classification in Table 1. The second dataset is 2 famous Linux utilities, Bash and Cpio, which is used to prove the practicability of our system.

Experimental Results

We have conducted our experiments with two types of dataset. Juliet Test Suite We tested our approach with 3,969 testcases of Stack Based Buffer Overflow vulnerabilities of Juliet Test Suite. The number of samples covered in Table 3 states the number of cases that we verified(over 90% in total). Our processing time for 3,969 testcases is nearly 20 minutes, which includes about 2 minutes of instrumentation and 17 minutes of concolic testing. Table 3 shows the number of testcases processed, number of good and bad sinks, and elapsed time of instrumentation and concolic testing for each type.

We checked the rest of the cases that we couldn't verify and found out that our frontend parser cannot handle non-standard coding style such as wide string L"AAA"(222 cases). Besides, our tool failed to correctly instrument the source code when handling function pointers(76 cases) and complex use of pointers such us pointer's pointer(105 cases).

A working example is shown in Fig. 5. Sub figures in Fig. 5 indicate the sequence of our mechanism. Fig. 5(1) shows the result of CIL transformation of a file named CWE121 Stack Based Buffer Overflow CWE131 memcpy 01.c. Then we show the instrumentation result of this case in Fig. 5(2). The assertion assert(dst.space ≥ n) ∧ (n ≥ 0) is automatically inserted before the sink memcpy. Fig. 5(3) shows the execution result of concolic testing. In this step, it actually verifies the vulnerability by checking whether the execution of program violates the assertion or not. By using this mechanism, our approach can detect and verify the vulnerabilities in the Juliet Test Suite in a fully automatic way. The comparative detection results. We show the number of false positives regarding to each type of sink in Table 4. As we can see, when we apply apply our system with MSA, we get no false positives while there are some false positives(233 in total) when MSA is not applied. The memory space analysis technique finds the real size of the buffer space at runtime and accurately set the condition for violating security constraint(bug triggering condition). When the MSA is not applied, the real runtime memory size of a pointer in the violation condition can only be set by approximation, which results in false positives. MSA can help our system completely reduce false positives.

We also compare our system with Flawfinder [START_REF] Wheeler | Flawfinder, Access[END_REF] which is a source code auditing tool widely used in security communities. We measure the precision ( T P T P +F P ), recall( T P T P +F N ) and F1 Value( 2P * R P +R ) for each tool: 1) our system with MSA; 2) our system without MSA; 3) Flawfinder. As shown in Fig. 6, our system applied with MSA gets the highest P recision of 100% which means 0 false positives. In terms of Recall, Our system with MSA gets Recall of 94.91%. Flawfinder has the highest Recall value, however, its P recision is quite low. F 1 value is a more comprehensive indicator for evaluation, our system with MSA gets the highest F 1 value of 97.43%. For the false negatives, our tool failed to correctly instrument the source code when handling the cases involving function pointers and complex use of pointers such as pointer's pointer(total 181 cases). This makes the tool cannot trigger the failure of the assertion when a real vulnerability exists, which contributes to false negatives. Case1: Cpio-2.6(CVE-2014-9112). We also demonstrate the effectiveness of our system by real open source projects. Fig. 7 shows a vulnerability in program Cpio-2.6 which is a program to manage archives of files. This vulnerability is caused by an integer overflow induced buffer overflow. at line 8, the numeric operation can cause an integer overflow and results in 0 bytes allocation for "link name". The buffer overflow is at line 10 when the program is trying to write "c filesize" number of bytes to 0 space buffer. We apply our system to automatically report out this vulnerability a fully automatic way by generating an input which makes "f ilesize c = 0xf f f f f f f ". Case2: Bash-4.2(CVE-2012-3410) We also apply our system to Bash-4.2 and successfully verifies the vulnerability in Fig. 8. Our system identifies the sink "strcat", backwardly tracing the user input and set the violating condition of security constraint by automatic instrumentation. The system reports out this vulnerability with a triggering input "path = /dev/f d/aaaa...aa(35 ′ a ′ s)".

Conclusion

In this paper, we propose, design and implement an automated verification mechanism for vulnerability discovery. Different from other source code auditing methods, our mechanism needs no human interference with extremely low false positive rate and it can expand the system usability to non-security specialist. It takes source code as input, detects and verifies the existence of vulnerability in a fully automatic way. What's more, the memory space analysis(MSA) enables us to set violating condition of security requirements accurately so as to report vulnerabilities with higher accuracy. We developed a prototype system and conducted several experiments with Juliet test cases and also real open source projects. The results show that our system can detect and verify vulnerabilities with low false positives within reasonable time.

However, there are concerns and limitations as well. To the current stage, our system focuses on buffer overflow vulnerability. In future research, we will study the features of other kinds of vulnerability and expand the vulnerability type coverage. Moreover, due to the incapability of handling complex data types such as nested structures in C code, function pointers and pointer's pointer, the system is limited to be working on programs with relatively small amount of source code. The source code analysis and automatic instrumentation will be further generalized to cover large programs.

Fig. 1 :

 1 Fig.1: General overview of our approach.

Fig. 4 :

 4 Fig. 4: The system architecture.

Fig. 5 :

 5 Fig. 5: Snapshot results of different phases

Fig. 6 :

 6 Fig. 6: Detection performance and comparison

Fig. 7 :Fig. 8 :

 78 Fig. 7: CVE-2014-9112 vulnerability from Cpio-2.6

Table 1 :

 1 Sink classification and variable location

	Sink type Description	Argument format	Sensitive functions	Variable positions
	Type 1	functions with two arguments	fn(dst,src)	strcpy, wcscpy strcat, wcscat	2
	Type 2	functions with three arguments	fn(dst,src,n)	memcpy, memmove, strncpy strncat, wcsncpy, wcsncat	2,3
	Type 3	function with fn(dst,n,"%s",src) format strings	snprintf, swprintf	2,4
	Type 4	memory operations	buffer[i] = expr	dstbuf[i] = 'A'	index:i

Table 2 :

 2 Security requirements for security sinks

	Security sinks	Security requirement	Description
	strcpy(dst,src)	dst.space > src.strlen	Space of dst should be bigger than the string length of src
	strncpy(dst,src,n)	(dst.space ≥ n) ∧ (n ≥ 0)	Space of dst should be bigger or equal to the positive integer n
			Space of dst should be bigger
	strcat(dst,src) dst.space > dst.strln + src.strlen	than the total string length
			of dst and src
	getcwd(buf,size) (buf.space ≥ size) ∧ (size ≥ 0)	Space of buf should be bigger or equal to the positive integer size
	fgets(buf,size,f) (dst.space ≥ size) ∧ (size ≥ 0)	Space of dst should be bigger or equal to the positive integer size
	buf[i] = expr.	buf.space > i	Space of buf should be bigger than the index value i

Table 3 :

 3 Type of vulnerabilities and base information of experiment

	Type	Number of Samples Covered (Instrumentation) (Concolic Testing) Bad Sinks Good Sinks Elapsed Time Elapsed Time Number of Number of
	1	2,065/2,217	78.816s	547.879s	2,217	3,126
	2	695/732	25.893s	175.986s	732	1,008
	3	292/296	10.535s	68.504s	296	408
	4	715/724	26.150s	197.690s	724	1,048
	Total	3,767/3,969	141.394s	990.059s	3,969	5,590

Table 4 :

 4 False positives with or w/o MSA

	Sink type # of FP with MSA # of FP without MSA
	1	0	102
	2	0	58
	3	0	25
	4	0	48
	Total	0	233

Our program and sample scripts are available at http://cssa.korea.ac.kr/ clorifi.

⋆ This research was supported by Institute for Information & communications Technology Promotion(IITP) grant funded by the Korea government(MSIP)(R0190-15-2011, Development of Vulnerability Discovery Technologies for IoT Software Security). ⋆⋆ To whom all correspondence should be addressed.