Kathryn Parsons
email: kathryn.parsons@dsto.defence.gov.au

Agata Mccormac

Malcolm Pattinson
email: malcolm.pattinson@adelaide.edu.au

Marcus Butavicius
email: marcus.butavicius@dsto.defence.gov.au

Cate Jerram
email: cate.jerram@adelaide.edu.au

Phishing for the Truth: A Scenario-Based Experiment of Users' Behavioural Response to Emails

Keywords: phishing, information security, security behaviours, email security, security training

de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Phishing is a term that describes an attempt to deceptively acquire personal and financial information via electronic communication with malicious intent. Social engineering strategies in conjunction with computer knowledge are used to gather usernames, passwords, and bank account and credit card details (Anti-Phishing Working Group, 2010). Phishing attacks are commonly committed via email, and victims are often directed to fraudulent websites that appear legitimate [START_REF] Clayton | An empirical analysis of the current state of phishing attack and defence[END_REF]. Such breaches can have serious consequences, including direct consequences, such as financial loss if a phisher obtains access to a bank account, and indirect consequences, such as damaged reputation [START_REF] Tam | The psychology of password management: a tradeoff between security and convenience[END_REF].

Although there is a growing body of phishing studies, there is a lack of research examining the impact of the cognitive bias known as the subject expectancy effect [START_REF] Anandpara | Phishing IQ tests measure fear, not ability[END_REF]. Essentially, studies where participants know they are participating in a phishing study have been criticised because they lack real world validity. This criticism is based on the assumption that individuals who are aware that they are taking part in a phishing study may be more suspicious, and this may therefore result in a bias towards 'phishing' decisions [START_REF] Anandpara | Phishing IQ tests measure fear, not ability[END_REF]. It is unlikely that individuals would have this level of suspicion when checking their personal inboxes [START_REF] Furnell | Phishing: can we spot the signs?[END_REF].

In response to the possible influence of the subject expectancy effect, researchers have begun incorporating a role play scenario into the design of their phishing studies. This approach aims to minimise the bias caused by the subject expectancy effect, because participants are not informed that they are participating specifically in a phishing study. A study of this nature was conducted by [START_REF] Downs | Behavioral response to phishing risk[END_REF]. Participants were informed that they were participating in a study about computer use, not computer security. They were given the identity of 'Pat Jones' and were shown images of emails from 'Pat's' inbox. Some emails were legitimate and some were phishing emails and participants were given options about how they would respond to each email. [START_REF] Downs | Behavioral response to phishing risk[END_REF] found that participants who were more knowledgeable and experienced with the internet environment were less susceptible to phishing attacks. They also found that participants' perceptions of the consequences of emails did not reliably predict their behaviour (2007). This study used only five email screen shots, which limits the ability to generalise these findings to other types of emails. [START_REF] Downs | Decision strategies and susceptibility to phishing[END_REF] conducted a similar study that used a role play design and also incorporated qualitative interviews on computer security and trust. The study focused on decision strategies and susceptibility to phishing and concluded that participants were most likely to use subjective cues, such as relying on the text within an email, to determine the trustworthiness of emails, rather than relying on more objective cues, such as using information contained within URLs and links [START_REF] Downs | Decision strategies and susceptibility to phishing[END_REF]. They also found that participants were more vulnerable to unfamiliar phishing scams, and were generally less susceptible to scams they had seen previously. However, the authors acknowledge that their findings were significantly limited by a small sample size of only 20 participants [START_REF] Downs | Decision strategies and susceptibility to phishing[END_REF], which, once again, limits the generalisability of findings.

The limitations and shortcomings of these previous studies provided the incentive for this current study, which was designed to address these issues, and further the vital work in the area of electronic mail fraud. For example, this is the first study that we are aware of that tested the influence of the subject expectancy effect on par-ticipants' response to phishing emails. This was achieved by using a role play design where, although all of the participants were aware they were participating in a study about email, only half of the participants were informed they were participating in a phishing study. This will therefore reveal whether the knowledge that participants are participating in a phishing study influences their decisions. Our examination of the influence of the subject expectancy effect may highlight the need to reevaluate or interpret the findings of all previous phishing studies in light of the effect. This study will therefore provide vital knowledge regarding the design of future studies.

Participants in the current study were also exposed to a comparatively larger number of emails than in the previous studies, and these emails varied widely. Furthermore, to better understand what makes some people more susceptible to phishing emails than others, this study included a demographics questionnaire and a measure of impulsivity. This provided a more comprehensive assessment of individual differences than the previous research.

Methodology

Participants

A total of 117 students from the University of Adelaide were recruited via email and participated in the study. Of the 117 participants, 27 were male and 90 were female. The majority of participants were first year students (93), there were also 19 second year and 4 third year students, and 1 participant was completing post-graduate studies. Most participants were 25 years of age and younger (108) and only 5 participants were over the age of 30. Participants received $25 cash for their participation.

Materials

Emails

The study consisted of 50 images of emails; half of these images were genuine emails, and half were phishing emails. The selected emails were comprised of 'actual' real and phishing emails that were either received by the authors, or found online. A range of emails, including banking, shopping and social networking emails were utilised to ensure that they were representative of the types of topics that would be expected in a typical inbox. An example of a phishing email can be seen in Figure 1, and an example of a real email used in the experiment can be seen in Figure 2.

A fictitious character, by the name of 'Sally Jones' was created, and the original emails that contained personal information were altered to include her details as if she was the intended recipient. Participants were informed that they were viewing emails from the inbox of 'Sally Jones', and were asked to make a decision regarding how they would manage each email. They were not provided with any other information regarding the persona of Sally Jones.

Measures of Individual Differences

A number of demographics were collected including information about gender and education level. The Cognitive Reflection Test (CRT), which is a very quick and efficient measure of impulsivity [START_REF] Frederick | Cognitive reflection and decision making[END_REF], was also utilised. The test includes three questions and the most obvious response is not correct. To answer correctly, participants should stop and consider the question before providing an answer. A higher score on this test relates to a better ability to control impulsivity. This particular test was selected because findings indicate that the predictive validity of this measure was equal or above other cognitive measures [START_REF] Frederick | Cognitive reflection and decision making[END_REF].

It is hypothesised that individuals who are better able to effectively manage impulsivity may be less susceptible to phishing emails, as they may be more likely to thoroughly deliberate the legitimacy of the email. This hypothesis was tested in a study by Kumaraguru and colleagues (2007). Participants with higher CRT scores were less likely to click on the phishing emails, but the results were not statistically significant. This may be due to the small number of phishing emails in the experiment, and the relatively small number of participants. The current study will retest this hypothesis with a larger number of emails and participants.

Method

Participants were informed that they were completing an experiment on how peo-ple manage emails. They were told that they would be required to view images of 50 emails, taken from the inbox of Sally Jones. In order to test the influence of the subject expectancy effect, the participants were divided into two groups, the 'Control' Group and the 'Alerted' Group. The 'Control' Group, which consisted of 59 participants, were given the following description:

"Managers are often inundated with an extremely large number of emails on a daily basis, and the management of these emails is often very difficult. We're interested in assessing how people manage emails. You will be presented with a number of emails, both personal and work related, taken from the inbox of 'Sally Jones'.

Your job is to examine each email, with the aim of assisting Sally to process her Inbox. You will be asked what action you would recommend to her. You will also be asked to provide a rating of how confident you are with your recommendation, and what aspect of the email most influenced your recommendation."

The 'Alerted' Group, which consisted of 58 participants, were informed that they were participating in a phishing study and were given the same description with the following sentence added to the end: "We are specifically interested in assessing the ability to identify 'phishing' emails. These are fraudulent email messages that are used to obtain personal information for the purposes of identity theft."

The research assistant also gave a verbal description of what phishing emails are to be sure that all participants had this knowledge.

For each of the 50 emails, all participants were asked to respond to the question , "How would you manage this email?" with one of four replies: a) leave the email in the inbox and flag for follow up; b) leave the email in the inbox; c) delete the email; or d) delete the email and block the sender. For each email, participants were also asked, "what aspect of this email influenced your decision?"

After responding to all 50 emails, participants were required to complete the demographics questionnaire and the cognitive test. The logic of this process ensured that the demographic questions could not alert the participants in the 'Control' Group that they were participating in a phishing study until after they had completed the main task.

Results

Phishing Emails

As shown in Figure 3, when responding to phishing emails, participants in the 'Control' Group most frequently responded with 'Flag for follow up' (37%), whereas the participants in the 'Alerted' Group were mostly likely to respond with 'Delete' (38%). The pie chart below also indicates that participants in the 'Alerted' Group were far more likely to respond with 'Delete and block' (11% of 'Control' Group responses versus 23% of 'Alerted' Group responses). For the phishing emails, a response of 'Delete and block' was considered most appropriate, and a response of 'Flag for follow up' was deemed least appropriate. A total score was calculated for phishing emails, where a response of 'Delete and block' was assigned a score of 4, a response of 'Delete' was assigned a score of 3, a response of 'Leave in inbox' was assigned a score of 2, and a response of 'Flag for follow up' was assigned a score of 1. This assignment was such that the more appropriate the action when faced with a phishing email, the higher the values assigned to it.

A Mann-Whitney U-test was used to compare the ranks for the participants in the 'Control' Group and the 'Alerted' Group. The results indicated that there was a statistically significant difference, U(116) = 2644.00, Z = 5.089, p < .001. Participants in the 'Control' Group had a mean rank of 43.19, while participants in the 'Alerted' Group had a mean rank of 75.09. This means that the participants in the 'Alerted' Group were significantly better at correctly managing the phishing emails, indicating that knowledge that participants were undertaking a phishing study tended to improve performance.

Real Emails

Interestingly, for the real emails, there was very little difference between the groups in regards to the frequency of both 'Delete and block' and 'Flag for follow up' responses. Instead, as shown in Figure 4, the percentage of responses in those categories was very similar, and the groups differed in regards to the responses, 'Leave in inbox' and 'Delete'. When responding to real emails, the participants in the 'Control' Group were far more likely to respond with 'Delete' (37%), whereas the participants in the 'Alerted' Group were most likely to respond with 'Leave in inbox' (44%). A total score was calculated for real emails using a similar approach to that employed above for the phishing emails, i.e., a response of 'Flag for follow up' was assigned a score of 4, a response of 'Leave in inbox' was assigned a score of 3, a response of 'Delete' was assigned a score of 2, and a response of 'Delete and block' was assigned a score of 1.

A Mann-Whitney U-test was used to compare the ranks for the participants in the 'Control' Group and the 'Alerted' Group. There were no statistically significant differences between the groups, U(116) = 2008.50, Z = 1.624, p = .104, with a mean rank for participants in the 'Control' Group of 53.96, and a mean rank for participants in the 'Alerted' Group of 64.13. Although the difference was not statistically significant, the participants who were alerted that they would be viewing some phishing emails were more likely to correctly manage the real emails.

Bias and Discrimination

To further examine the nature of any subject expectancy effect, the signal detection theory measures of discrimination and bias were calculated from the data [START_REF] Green | Signal Detection Theory and Psychophysics[END_REF]. In this context, discrimination refers to the ability of a participant to distinguish real from phishing emails while bias refers to an overall tendency to keep or delete emails in the inbox. A' and B'' were used as measures of discrimination and bias respectively (refer to [START_REF] Stanislaw | Calculation of signal detection theory measures[END_REF]. These non-parametric measures are calculated directly from the commonly used measures 'Hit Rate' (HR) and 'False Alarm Rate' (FAR):

The HR refers to the probability that a phishing email is met with a phishing decision and the FAR refers to the probability that the participant responded with a phishing decision when it was a real email. For the purposes of this analysis, when faced with phishing emails, responses of 'Delete' or 'Delete and block' were deemed to be 'hits', and responses of 'Flag for follow up' or 'Leave in inbox' were deemed to be 'misses'. When faced with real emails, responses of 'Delete' or 'Delete and block' were deemed to be 'false alarms', and responses of 'Flag for follow up' or 'Leave in inbox' were deemed to be 'true misses'.

An A' value of 1 equates to perfect discrimination, and a value of 0.5 indicates that the respondent could not distinguish phishing emails from the real emails. A B" value of zero indicates that there was no bias in the responses, a value of -1 indicates an extreme bias towards 'phishing' decisions, and a value of 1 indicates an extreme bias towards 'real' decisions.

The results indicated that the participants in the 'Alerted' Group were better able to discriminate between the phishing and real emails than the participants in the 'Control' Group (A' Control = 0.52, CI 95% = [0.48, 0.56]; A' Alerted = 0.72, CI 95% = [0.68, 0.76]). Furthermore, the results showed that participants in both the 'Control' and 'Alerted' Group had a very small response bias (B" Control = 0.07, CI 95% = [0.01, 0.13]; B" Alerted = 0.04, CI 95% = [-0.04, 0.13]), indicating that this study did not find evidence of the subject expectancy effect.

Time Taken

An independent-samples t-test demonstrated a significant difference between the time taken to manage emails between the two groups, t(116) = 4.093, p < .001. The participants in the 'Control' Group (M = 21.47, SD = 5.83) took significantly less time to make their decisions than the participants in the 'Alerted' Group (M = 27.05, SD = 8.67). The eta squared statistic (.13) indicated a large effect size. This therefore suggests that informing participants that they were completing a phishing study may have resulted in an increase in diligence and vigilance.

Individual Differences

Since the manner in which participants managed emails is best captured via the four response categories, the mean ranks for phishing and real emails were used, and a series of Kruskal-Wallis and Mann-Whitney U-tests were conducted. The aim was to examine the influence of these variables on the mean ranks for phishing and real emails for the 'Control' Group and 'Alerted' Group.

Gender and Age

Contrary to the findings of Jagatic et al. (2005) and Sheng, Holbrook, [START_REF] Sheng | Who falls for phish? A demographic analysis of phishing susceptibility and effectiveness of interventions[END_REF], who found that females and participants aged below 25 years were most vulnerable, the current study found no evidence of a relationship between either gender or age and the ability to correctly manage emails. However, of the 117 participants in this study, 90 were female and 108 were under 26 years of age. Because of this bias in our population, we can not discount the findings of Jagatic et al. (2005) and [START_REF] Sheng | Who falls for phish? A demographic analysis of phishing susceptibility and effectiveness of interventions[END_REF], and this is an issue worthy of further research.

Level of Education and Knowledge

After ranking the total phishing scores, a Kruskal-Wallis test was used to evaluate any association between performance in managing emails and participants' level of education. When participants were not told that they were conducting a phishing study, the participants with a higher level of education were significantly better at correctly managing phishing emails, χ 2 = 8.186 (2, N = 59), p = .017 (Mean ranks; 'Year 12 or equivalent' = 27.33, 'Bachelor Degree' = 32.07, 'Honours Degree' = 56.17). However, there was no difference in the ability to correctly manage phishing emails when people were informed that they were completing a phishing study. This therefore suggests that people with more education were more likely to think about security without being prompted.

However, contrary to expectations, results indicated that the participants in the 'Control' Group who had completed a course in the area of information systems or information technology were less accurate in their ability to correctly manage phishing emails. A Mann-Whitney U-test revealed a mean rank of 21.50 for the n = 17 participants who had completed a course in the area of information systems or information technology, and a mean rank of 33.44 for the n = 42 who had not completed such a course, U(58) = 501.50, Z = 2.421, p = .015.

Employment Experience

A Kruskal-Wallis test revealed that, for the participants in the 'Alerted' Group, those who were currently employed (mean rank = 34.48) or had previous employment experience (mean rank = 30.69) were significantly better at identifying phishing emails than those without any employment experience (mean rank = 17.18), χ 2 = 7.817 (2, N = 58), p = .02. Interestingly, this was only true for the participants who were informed that they were conducting a phishing study. UNCLASSIFIED 11 3.5.4. Cognitive Impulsivity For participants in the 'Control' Group, those who obtained a higher score on the test of cognitive impulsivity (and were therefore better able to control impulsivity) were significantly better at identifying phishing emails, χ 2 = 8.241 (3, N = 59), p = .041. The mean rank for the participants who obtained a score of three, which was the maximum possible score, was 48.92, which is significantly higher than the mean rank obtained by participants with the other possible scores (mean ranks; '0' = 27.89, '1' = 27.11, '2' = 29.79). This is consistent with our hypothesis. However, there were no significant differences for the participants in the 'Alerted' Group, which suggests that, once participants knew they were undertaking a phishing study, they were more likely to stop and consider their response, regardless of whether they were usually more impulsive.

Qualitative Content Analysis

Participants' responses to the question, "What aspect of this email influenced your decision?" were analysed using content analysis. An analysis of the participants' responses to the open-ended question in conjunction with their response to the multiple choice question supported the hypothesis that participants were more likely to 'Delete' or 'Delete and block' emails when they were suspicious of their legitimacy. In a minority of cases, participants flagged an email for follow up to alert 'Sally' of a possible security threat that she should report to the purported organisation. However, this reasoning was rare, and an examination of these cases revealed that recategorising them would not impact on the statistical significance of the results reported above.

The justification of decisions supported the findings of [START_REF] Furnell | Security beliefs and barriers for novice Internet users[END_REF], that participants were influenced by their perception of trust. This was based on the perceived trustworthiness of the company that the email appeared to originate from. For example, participants responded with statements such as "[Company name] is a trusted chat program used all over the world, so emails from it would be legit". These participants did not appear to question whether the email actually did originate from that company, but rather, appeared to decide based solely on the face validity of the email.

Many participants also mentioned the visual presentation of emails. For example, many participants deleted a real email from a telephone company because it did not contain any company logos, and therefore concluded that it seemed suspicious. This is consistent with the findings of Everard and Galletta (2006), who found the perceived quality of an online Web site was strongly influenced by the style of the Web site, and poor style was associated with low perceived quality.

Another common justification was based on incentives within emails. Participants were more susceptible to phishing emails when the email promised a financial reward. For example, two survey requests with a financial incentive for participation resulted in responses such as "the $100 monetary compensation is a great incentive for me to participate in this survey". Hence, potential incentives may limit partici-pants' ability to make valid and considered security decisions. It should be noted that participants were paid $25 for their participation, and therefore, may have been more susceptible to offers of financial reward.

Other commonly cited reasons for decisions included spelling and grammatical errors, the personalisation of the email, and the perceived legitimacy of the URL. As also found by [START_REF] Furnell | Phishing: can we spot the signs?[END_REF], although these were cues that could conceivably prove useful, they often failed to assist participants in making the correct decisions concerning the legitimacy of an email. In support of previous research, which indicated that participants do not notice security indicators [START_REF] Herzberg | Why Johnny can't surf (safely)? Attacks and defenses for web users[END_REF], Schechter et al., 2007), only one participant in our study used 'HTTPS' as a justification for their decision.

Discussion

This study provides further evidence that people are poor at identifying phishing emails. Overall, approximately 42% of all emails were incorrectly classified in this experiment. Participants who knew they were undertaking a phishing study were better able to make the distinction between real and phishing emails, which means that this study found no evidence of the subject expectancy effect. Participants in the 'Alerted' Group were not simply biased towards 'phishing' decisions, but were instead more likely to correctly manage all emails. Although the improved ability to correctly manage the real emails was not statistically significant, they were still more likely to correctly manage the real emails than the participants in the 'Control' Group. Evidence suggests that priming participants with the notion of phishing may have resulted in more diligent decision making, as the participants in the 'Alerted' Group took significantly longer to complete the experiment.

The influence of the different instructions provided to the two groups in our experiment may be explained by the general phenomenon known as framing [START_REF] Tversky | The framing of decisions and the psychology of choice[END_REF]. In other words, it was the context in which the task was presented which influenced their decision making processes. Unlike the 'Control' group, the participants in the 'Alerted' group were specifically informed that the study was testing how well they could detect phishing emails and this had a positive influence on their decision-making. As discussed in the context of Signal Detection Theory [START_REF] Green | Signal Detection Theory and Psychophysics[END_REF], this change was not in decision bias towards classifying an email as phishing but instead reflected an improvement in discrimination ability. Framing the task as one of detecting phishing emails may have caused participants to focus more on cues in the stimuli that better distinguished real from phishing emails. This has important implications for training and education programs, as it suggests that when people were primed to think about phishing, they were better able to identify phishing emails, and hence, less susceptible to phishing attacks.

The findings in regards to individual differences also have important implications for education and training programs. Participants who had attended an information systems or technology course were, in fact, less likely to correctly manage emails. This may suggest that knowledge in this area could lead to complacency.

Instead, actual security behaviours (such as using spam filters and adjusting security preferences) were better predictors of the ability to deal with phishing emails. In addition, when the task was framed as a phishing test, participants with employment experience performed better, possibly as a result of more experience in dealing with categorising emails in a work environment. Our results also indicated that participants who were better able to control impulsivity were better at managing phishing emails. This suggests that it may be more effective to emphasise the importance of stopping and thinking before responding to any email rather than exclusively teaching security rules. This is supported by the training literature, which indicates that it is more effective to emphasise specific behaviours rather than rules [START_REF] Parsons | Human Factors and Information Security: Individual, Culture and Security Environment[END_REF].

Our findings also have implications for the research literature on users' susceptibility to phishing emails. Previous studies should be interpreted in the light of the 'framing' effect identified in this study and future research should carefully consider how the task is presented to the user. Critically, the discovery of the framing effect suggests that the risk of phishing may, on the whole, be underestimated in previous literature. Specifically, in our study the inferior results of the 'Control' Group, who were not informed that they were undertaking a phishing study, better represent the performance of real-world users. This is because in real life the frequency with which people are reminded about the risks of phishing emails is generally low.

It is, however, important to highlight the fact that phishing studies such as ours do not directly measure actual susceptibility. In our experiment, participants were not required to click on any of the links or provide personal information, and it is therefore possible that, in a real world situation, participants may have become suspicious before succumbing to any of the phishing attacks. This study was also a role play, and the manner in which participants deal with emails in an experimental environment may not relate precisely to how participants would deal with actual emails received in their personal inboxes. Furthermore, in this study, participants did not know which sites 'Sally' subscribes to, and therefore their ability to make context dependent decisions was limited. In a real life situation, whether someone is a member of a particular bank or social networking site is likely to influence the decision to delete or keep an email. Future research should investigate how to more accurately replicate these variables in an experimental context.

Fig. 1 .

 1 Fig. 1. Example of a phishing email. Email containing logo reprinted with permission from Google.

Fig. 2 .

 2 Fig. 2. Example of a real email. Email containing logo reprinted with permission from Google.

Fig. 1 .

 1 Fig. 1. Responses to phishing emails for the 'Control' and 'Alerted' Groups.

Fig. 2 .

 2 Fig. 2. Responses to real emails for the 'Control' and 'Alerted' Groups.


Wayne D Kearney
email: kearneys@iinet.net.au

Hennie A Kruger
email: hennie.kruger@nwu.ac.za

Phishing and Organisational Learning

Keywords: Phishing, Social engineering, Information security awareness, Organisational learning

des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Traditionally the mitigation of information security risks was addressed using a variety of technical controls. It is however widely accepted and recognised that technology on its own cannot deliver complete solutions to the security problem and that the human aspect of security should receive more attention [START_REF] Furnell | Power to the People? The Evolving Recognition of Human Aspects of Security[END_REF], [START_REF] Pfleeger | Leverage Behavioral Science to Mitigate Cyber Security Risk[END_REF], [START_REF] Van Niekerk | Organisational Learning Models for Information Security[END_REF]. One way of addressing the human side of security is to focus on awareness and educational activities [START_REF] Thomson | Combating Information Security Apathy be Encouraging Prosocial Organisational Behaviour[END_REF] making use of some form of an awareness program.

An information security awareness program normally focuses on a number of issues related to the correct security behaviour of users. In some instances it may also concentrate on one area such as social engineering which is one of the most serious threats to information security as criminals keep on focussing on deceptive techniques to attack computer users and organisations [START_REF] Jansson | Phishing for Phishing Awareness[END_REF]. Phishing, which is one of the social engineering techniques, occurs when people are manipulated by deception into giving out information [START_REF] Jansson | A Model for Cultivating Resistance to Social Engineering Attacks[END_REF] and is one of the major threats to modern organisations and information technology users in general. It requires an ongoing awareness not to become a victim of a phishing scam and various researchers have completed studies related to phishing experiments and awareness levels of users [START_REF] Jansson | Phishing for Phishing Awareness[END_REF], [START_REF] Kumaraguru | School of Phish: A Real-World Evaluation of Anti-Phishing Training[END_REF], [START_REF] Jagatic | Social Phishing[END_REF].

A popular technique to improve user awareness pertaining to phishing scams is to conduct unannounced phishing tests in order to evaluate users' propensity to respond to an attack [START_REF] Jansson | Phishing for Phishing Awareness[END_REF], [START_REF] Dodge | Phishing for User Security Awareness[END_REF]. Albrechtsen [START_REF] Albrechtsen | Barriers against Productive Organisational Learning from Information Security Incidents[END_REF] contend that these type of incidents and experiments present great opportunities to learn and improve information security. To ensure that learning does take place Van Niekerk and Von Solms [START_REF] Van Niekerk | Organisational Learning Models for Information Security[END_REF] suggested that an organisational learning model be used.

This paper describes a practical phishing exercise that was conducted in industry and shows how organisational learning took place as a result. The remainder of the paper is organised as follows. Section 2 presents the background to the study as well as appropriate references to related work. In section 3 the methodology used is discussed while section 4 details the results. Concluding remarks are presented in section 5.

Background and related work

Organisational learning theories deal with the idea of how organizations learn and adapting its behaviour [START_REF] Van Niekerk | Organisational Learning Models for Information Security[END_REF]. This concept has been subjected to a wide and growing variety of researchers and a number of definitions have been suggested in the literature [START_REF] Schermerhorn | Organisational Behavior[END_REF], [START_REF] Lopez | Organisational Learning as a Determining Factor in Business Performance[END_REF]. Despite all these definitions the concept of organizational learning is by no means an unambiguous concept, as no one irrefutable definition has emerged in literature [START_REF] Kennedy | A Critical Evaluation of the Organisational Learning that takes place in a Project Management Environment[END_REF]. Organisational learning originated from the work by Argyris and Schon during the 1970s and one of the definitions suggested by them will be assumed in this study. The definition is formulated as follows. Organisational learning occurs when individuals within an organisation experience a problematic situation and enquire into it on the organisational behalf [START_REF] Argyris | Organisational Learning II: Theory, Method and Practice[END_REF].

In an effort to enhance organisational learning, Buckler [START_REF] Buckler | Practical Steps towards a Learning Organisation: Applying Academic Knowledge to Improvement and Innovation in Business Processes[END_REF] proposed that an actual learning process, as depicted in figure 1, occurs in organisations. Buckler then argues that individuals will move through the different learning stages driven by their inherent individual motivations to learn. Associated with these motivational forces, there will be certain barriers to the learning process, and where the motivational restraining (barrier) forces are matched, learning will not take place. In order for organisational learning to result in performance improvement, the enactment stage (see figure 1) of the learning process needs to be achieved -this will imply behavioural change which is a requirement for successful organisational learning. To assess the effectiveness of the behavioural changes, the reflection stage should be entered. There are various applications of learning processes but in general three types of learning can be categorised. These three types are summarized by Kennedy [START_REF] Kennedy | A Critical Evaluation of the Organisational Learning that takes place in a Project Management Environment[END_REF] as follows.

-Single-loop learning, which occurs when errors are detected and corrected and organisations continue with the present status quo without modifying present policies and goals. In essence, single-loop learning focuses on improving the status quo through small incremental changes in how organisations functions. An example in the area of information security could be a case of unauthorized access by a user to privileged data. A single-loop response would be to simply deny future access to this specific user. The status quo is maintained and present policies and/or goals are not modified. -Double-loop learning challenges, and possibly makes changes to the status quo and the existing assumptions and conditions. It means that the organisation questions and modifies its existing norms, policies, procedures and objectives and it can lead to transformational change that radically alters the status quo. In the information security example mentioned, a double-loop response may be to investigate the circumstances and reasons for the unauthorized access. Doubleloop learning may then occur when a decision is taken to improve (change) the process of allocating access rights in order to minimize future unauthorized access risks. -Deutero learning involves focusing on the learning process itself. This type of learning seeks to improve how organisations perform single and double-loop learning. It can be described as "learning how to learn" and it occurs when organisations learn how to perform both single and double-loop learning.

Due to the focus on long term goals and the more complex nature of double-loop learning, most companies focus only on single-loop learning [START_REF] Ahmad | Incident Response Teams -Challenges in Supporting the Organisational Security Function[END_REF]. According to Van Niekerk and Von Solms [START_REF] Van Niekerk | Organisational Learning Models for Information Security[END_REF] this is also true in the information security discipline.

They pointed out that generative, or double-loop learning, emphasizes continuous experimentation and feedback.

Although there are a large number of studies on organisational learning, there are not particularly many studies that relate organisational learning to information security. Even so, the studies that have been conducted in this area prove that information security is an important area that offered ample opportunities, linked to organisational learning, that can make a significant contribution to organisations and their performance. Examples of studies where organisational learning and information security were explored include the following.

Van Niekerk and Von Solms [START_REF] Van Niekerk | Organisational Learning Models for Information Security[END_REF] investigated, amongst other models, the use of an organisational learning model for information security education. Their aim was to ensure that adequate attention is given to behavioural theories in information security education programs. Albrechtsen [START_REF] Albrechtsen | Barriers against Productive Organisational Learning from Information Security Incidents[END_REF] conducted a comprehensive study into the barriers that exist and that prohibit productive organisational learning from information security incidents while Ahmat et al [START_REF] Ahmad | Incident Response Teams -Challenges in Supporting the Organisational Security Function[END_REF] suggested that the practice of incident response may lead to organisational learning. They proposed a double-loop learning model for security incident learning to address potential systemic corrective action. An interesting and authoritative study was conducted by Pfleeger and Caputo [START_REF] Pfleeger | Leverage Behavioral Science to Mitigate Cyber Security Risk[END_REF] where it was argued that blending behavioural sciences and cyber security may lead to the mitigation of cyber security risks. Although organisational learning was not specifically mentioned, the study strongly supports the idea that behavioural sciences (of which organisational learning is at least a sub-section) is relevant to information security in general. To further motivate this idea, Thomson and Van Niekerk [START_REF] Thomson | Combating Information Security Apathy be Encouraging Prosocial Organisational Behaviour[END_REF] also contend that employee apathy towards information security can be addressed through the use of existing theory from the social sciences.

There are also a number of studies where the focus is not on information security per se but rather on how information technology in general relates to organisational learning. These studies usually concentrate on computer systems necessary to facilitate organisational learning and knowledge transfer [START_REF] Kane | Information Technology and Organisational Learning: Investigation of Exploration and Exploitation Processes[END_REF], [START_REF] Chou | Computer Systems to Facilitating Organizational Learning: IT and Organizational Context[END_REF].

In the context of this paper, where it is claimed that a phishing exercise may lead to organisational learning, the next few paragraphs will briefly refer to the phishing concept and examples of studies related to it.

The basic idea of phishing is when someone attempts to fraudulently acquire sensitive information from a victim by impersonating a trustworthy entity [START_REF] Jagatic | Social Phishing[END_REF]. A more formal definition can be obtained from the Oxford English Dictionary [19] where phishing is defined as the fraudulent practice of sending e-mails purporting to be from reputable companies in order to induce individuals to reveal personal information, such as passwords and credit card numbers, online.

Phishing attacks are on the increase and successful attacks may have devastating effects on both enterprises and individuals. The Symantec Intelligence Report [20] of June 2012 reported that one out of every 170.9 e-mails sent during the month of June 2012, in South Africa, was a phishing scam. In the Netherlands the figure for June 2012 was one out of every 54.4 e-mails. Considering the billions of e-mail messages that are transmitted worldwide during a specific month, it becomes clear to what extend phishing attacks form part of the day to day electronic communication activities. With this in mind it becomes more and more important to implement the right and effective countermeasures to mitigate or prevent phishing attacks. One way of dealing with this growing number of phishing incidents is to implement security awareness and training programs where users are made aware of phishing scams. The use of practical tests seems to be a popular and effective way of making people aware of the dangers of phishing and some examples of the work conducted by other researchers in this area will be highlighted below.

Pattison et al [START_REF] Pattinson | Why do some People Manage Phishing E-mails Better than Others?[END_REF] investigated the behaviour response of computer users when receiving either phishing e-mails or genuine e-mails. The study was conducted as a scenario-based role-play experiment where participants had to indicate what the appropriate response would be on certain e-mail messages. The study found that participants who were informed, prior to the experiment, that they are part of a phishing exercise performed better in handling phishing e-mail messages.

Simulated phishing attacks together with embedded training were used by Jansson and Von Solms [START_REF] Jansson | Phishing for Phishing Awareness[END_REF] in an effort to cultivate users' resistance towards phishing attacks, while Kumaraguru et al [START_REF] Kumaraguru | School of Phish: A Real-World Evaluation of Anti-Phishing Training[END_REF] also conducted a study on anti-phishing training to proof that user training should be used in conjunction with technological solutions for security problems. Other studies include Dodge et al [START_REF] Dodge | Phishing for User Security Awareness[END_REF] who performed a practical phishing experiment involving students from the United States Military Academy, Jagatic et al [START_REF] Jagatic | Social Phishing[END_REF] performed a study at the Indiana University, Steyn et al [START_REF] Steyn | Identity Theft -Empirical Evidence from a Phishing Exercise, New Approaches for Security, Privacy and Trust in Complex Environments[END_REF] conducted a practical experiment in South Africa and Hasle et al [START_REF] Hasle | Measuring Resistance to Social Engineering[END_REF] a study in Norway.

It is interesting to note that all the practical phishing experiments referred to so far, were conducted using students as participants. Although these studies produced many advantages and insights, it is doubted whether the results can be generalised and extrapolated to industry enterprises.

Consistent with the research projects mentioned above, this study also performs a practical phishing experiment but uses an industry enterprise for research purposes instead of students in a university environment. In addition, the exercise is aimed at creating a climate for organisational learning. To ensure that the exercise is not a once-off event, the objective is to initiate a learning process and to show how security incidents such as phishing can and should be used for single and doubleloop learning in an organisation.

The study was conducted at a large geographically dispersed utility. The organisation in question is a large multi-billion dollar entity with over 3500 IT users and they supply essential services to over 2 million customers. The organisation has an information security course that is mandatory for all employees and partners who have access to the IT infrastructure. The objective of the course is to make IT users aware of their responsibilities with regards to protecting the organisations' information and information systems from unauthorised access, loss or disclosure. Whilst the information security course is deemed mandatory, the records could not support this assertion as many staff was found not to have completed the course or no records could be found of their attendance.

Methodology

The successful implementation of an e-mail phishing exercise is dependent on how well certain issues, associated with the exercise, are considered. Jansson and Von Solms [START_REF] Jansson | Phishing for Phishing Awareness[END_REF] categorised these issues into principles to be considered before designing the exercise, before conducting the exercise, during the exercise, and after the exercise while Dodge et al [START_REF] Dodge | Phishing for User Security Awareness[END_REF] simply refer to them as general and specific considerations. In this study considerations are also presented as general and specific considerations. The general considerations are concerned with those issues that may have an impact on the exercise as a whole while the specific considerations deal with aspects specific to the enterprise where the study was conducted.

General considerations

The first and most important general consideration is the determination and definition of an objective. There should be a clearly defined goal and in this study the goal was simply stated as the evaluation of security awareness associated with phishing and the creation of an opportunity for organisational learning to take place. The next consideration is critical for success i.e. to get ethical clearance and top management approval. This was achieved by conducting personal meetings with the CEO, the CFO and the IT manager where the purpose, actual steps and possible outcomes were explained. A formal project proposal detailing aspects such as the basic process, different phases, measures of success and possible risks, was also submitted for approval to management.

Other general considerations which were appropriately addressed included the timing of the exercise; maintaining the privacy of respondents; the selection of a random and representative sample of respondents; measurements to ensure that no information was disclosed prior to the exercise; and, a debriefing exercise following the test.

Specific considerations

The central issue among the specific considerations was the construction of an appropriate e-mail message. The message had to be concise, credible and at the same time be enticing in order for participants to react.

To ensure that the phishing e-mail message complies with all the necessary requirements, it was decided to make use of aspects that may trigger certain emotions from participants. Jansson [START_REF] Jansson | A Model for Cultivating Resistance to Social Engineering Attacks[END_REF] presents a list of a large number of techniques that are based on negative, positive and neutral emotional exploits. For the construction of the e-mail message the following emotional exploits were used. Legitimacy -when a user is made to believe that the source of the e-mail message is legitimate. Authority -people tend to comply with instructions or requests issued by someone with authority. Scarcity -when users believe that the time to react is limited. Conformity -users who believe that other fellow-employees have already reacted to a request are inclined to also comply with the request.

Apart from these four techniques which were explicitly built into the e-mail message (see figure 2), three other important emotional exploits were also implicitly included. They were urgency (making users believe it is an emergency), carelessness (clicking on a link) and diffusion of responsibility (users believe that someone else is responsible for security). Users were asked to click on the link in the message which would then take them to another webpage where their usernames and passwords were requested. Figure 2 also indicates how the e-mail was constructed to provide clues to alert users that the message was likely not to be legitimate. The real name of the organisation has been changed in figure 2.

Fig. 2. Phishing e-mail message

There were a number of other specific issues that also needed clarification before the actual exercise could take place i.e. it was important not to refer to any specific IT, security or internal audit staff as this may compromise the trust between users and staff. Steps also had to be taken to ensure that the enterprise's anti-phishing tools and spam filters do not identify the message as spam or a phishing scam, and Helpdesk had to be provided with a predetermined response should there be any queries from users. Provision was also made for respondents who reply directly to the phishing e-mail. Some of the technical considerations include the deletion of duplicate records (if a user responds more than once) and also a check to see whether the correct usernames were supplied (password were requested but not recorded).

The e-mail message (figure 2) was first sent to a small group of 10 employees. The objective was to test whether all technical aspects are functioning correctly and also to get feedback on possible improvements. After some minor changes were made, following the small pilot study, it was decided to go ahead and implement the phishing test.

The phishing e-mail message was sent to all employees at 8:00pm on a weekday night. The organisation is a 24-hour operation with activities taking place on a continuous basis. Statistics of user logs showed that there are on average about 1700 active IT users signed on during any night and to ensure that the night workers are included in the test, the 8:00pm sending time was chosen. This sending time would also guarantee that day workers should have the phishing e-mail in their inboxes first thing in the morning. The idea was to get users to respond early before they can discuss it with fellow employees.

A number of senior managers found the phishing e-mail very annoying and some of them sent out general e-mail messages to object to the phishing message (and the test). The security personnel were also involved and concern was expressed regarding the possibility of an external attack aimed at disrupting essential services. Due to this, it was decided at 8:30am the next morning to remove the phishing message and to officially end the test. The reasons for withdrawing the phishing email relatively early the next morning were firstly, to prevent large-scale disruptions and secondly, because enough data has been recorded at that stage to draw meaningful conclusions. The data and the experience were sufficient and interesting results, presented in the next section, were obtained.

Results

The data recorded from the phishing awareness exercise include the employee name, department where the person is working and the username. Passwords were also requested but not recorded due to privacy considerations. As part of the exercise, passwords were validated but only the result was recorded in a simple yes/no format. Appropriate safeguards to ensure privacy were put in place. The recorded employee names were purely recorded for statistical purposes and nowhere during reporting were specific names linked to responses. The reason for recording usernames was to perform a validation test to ensure that users do enter valid usernames (and by implication valid passwords). All duplicate records (users who entered their details more than once) and records with invalid usernames were removed from the final data set.

The main result, before any further analyses were performed, was the number of negative responses received. A negative response is a response where a user provided his or her username and password. During the test 280 users responded to the phishing message of whom 231 (83%) entered their usernames and passwords on the webpage. Of the 231 users, 23 (10%) entered their valid details more than once. Although there were approximately 1700 active users logged on during the test, it would be incorrect to assume that all of those who did not respond acted in a positive way. Reasons for this may be the fact that many people do not respond immediately to e-mail messages, some users may have left their workstations logged on during the night while not there, some users may have been engaged in other tasks and simply did not check their mail inboxes, etc. A much more significant analysis was to link the 280 users who responded, to the information security course that all staff members are required to complete and which would have provided them with basic security information on how to react to possible phishing scams. Figures 3(a) and(b) show the results graphically. Figure 3(a) shows that an unexpected 69% of those users who entered their passwords did complete the security training in the past. Figure 3(b) shows the training details of those who responded without entering their usernames and passwords. These results indicate that there are at least two points of concern. Firstly, the high number of users who responded in a negative way despite their security training and secondly, the relatively high number of users that never completed the information security course. Figure 4 shows an analysis of responses (percentages) per experience category for those who entered their usernames and passwords. Experience in this case refers to the number of years a person is employed at the organisation. From figure 4 it can be seen that those employees with less experience at the organisation (and therefore less exposure to its security practices and policies) are more inclined to give away personal details. More than a third (35%) of those who entered their usernames and passwords have less than 5 years experience with more than half (52%) less than 10 years. The data that was captured during the exercise makes it possible to perform a number of analyses, e.g. responses per department, gender, age group etc. These types of analyses were not done in this study as the focus was more directed at possible organisational learning opportunities.

As explained earlier, organisational learning involves the adjustment of actions based on an experience. These adjustments, or learning, can then be categorised as single or double-loop learning. The results from this study have shown that the phishing experiment offers the ideal opportunity for learning and that both single and double-loop learning has taken place.

Single-loop learning took place in the form of small changes in making staff aware of the risks and consequences of phishing scams. Instructions concerning basic acceptable behaviour related to suspicious e-mail messages were also issued. Specific actions that can be attributed to single-loop learning include the following.

-The first day, following the phishing exercise, the Manager Risk and Assurance sent out an e-mail message to all staff informing them about the exercise and, more importantly, making them aware of the risks and giving them basic instructions on how to react to these type of e-mails (e.g. to report it to the Service Centre). -The company's weekly in-house bulletin was used to reinforce the security awareness message and to instruct staff to complete the company's computer based information security course. This was done for two consecutive months following the phishing exercise.

The single-loop learning examples mentioned here did not change the status quo of any process but were quick and effective corrective measures to address a specific problem area. There were, however, other issues that needed a more comprehensive investigation that may lead to a change in policies and procedures. These doubleloop learning issues include the following.

-All staff members are required to complete an information security course which will equip them with basic security knowledge for different security situations including phishing scams. An analysis of the phishing results showed that not all staff has completed the course. More importantly, a relatively large number of those who have completed the course had given their passwords away. An assessment of the course content and possible controls to ensure that everybody completes the course is planned. This may lead to a change in the current security policy on issues pertaining to basic security training. -Another issue, planned for the future, which was highlighted during the phishing exercise relates to the gap between the different security views and expectations of managers and users. This gap is sometimes referred to as the information security digital divide between managers and users [START_REF] Albrechtsen | The Information Security Digital Divide between Information Security Managers and Users[END_REF] and may lead to unrealistic security assumptions and management strategies that are not aligned with the dynamics of the user environment.

If one considers the results of the phishing exercise it seems permissible to draw the conclusion that the exercise has created opportunities for organisational learning. Basic problems were immediately corrected through an easy and uncomplicated single-loop learning approach while double-loop learning issues provided an opportunity for the organisation to adapt and adjust some of their information strategies.

Conclusions

Modern businesses are characterised by the increasing reliance on information assets. The protection of these assets depends to a large extend on the employees and users and it is not surprisingly that criminals tend to focus their attacks on humans. Phishing has become one of the most frequently used techniques to obtain personal or private information and to combat it, proper security awareness programs should be in place. To ensure that a security awareness activity does not become a once-off event, organisations may want to consider the use of various organisational learning models to enhance the awareness and educational value of such programs.

In this paper a successful practical phishing exercise was conducted at a large organisation. The aim was not only to record the number of users who are willing to give away personal information, but also to create an opportunity for organisational learning in order to improve the educational value of the phishing experiment. The results have shown that employees are prone to phishing attacks, but more importantly, the phishing exercise created an excellent opportunity for both single and double-loop learning activities. A single-loop learning approach was followed to immediately correct certain shortcomings without changing the status quo, while double-loop learning provided the opportunity to revisit and adapt some of the longer term information security strategies.

One security experiment linked successfully to organisational learning does not necessarily prove that all security exercises will lead to organisational learning. The exercise did, however, provide an insight into exciting possibilities to increase the value of security awareness exercises and that it may ultimately lead to the completion of the learning process described in section 2 of the paper.

Fig. 1 .

 1 Fig. 1. The learning process (adapted from [15])

Fig. 3 .

 3 Fig. 3. Responses related to training completed

Fig. 4 .

 4 Fig. 4. Responses per experience category