Erik Moore

Dan Likarish
email: dlikaris@regis.edu

A Cyber Security Multi Agency Collaboration for Rapid Response That Uses AGILE Methods on an Education Infrastructure

Keywords: cyber, security, multi agency, collaboration, rapid response, agile, interagency, infrastructure, physical exercise, line training, Colorado, simulation, National Guard, state exercise, constraint, defense, operations, education 1

This study provides a summary and analysis of a cyber security multi agency collaboration for rapid response by Regis University (RU), in partnership with the Colorado Army and Air Force National Guard (CONG) and the State of Colorado (SOC), deploying AGILE methods to improve the ability of the CONG and SOC to respond to attacks against Colorado's critical infrastructure.

The summary covers formative discussions and about a yearlong series of physical exercises, lectures and certification exams that advanced the study participants domain knowledge, awareness of SOC policy and communication with industry. Other states and territories can use the model to the benefit of their citizens. Events included multiple simulations, physical exercise scenarios, and table top exercises designed to give realworld substance to more abstract cyber security concepts and integrate physical world consequences to actions performed by the participants.

Introduction

Defense forces operate under highly constrained procedures because of the high impact of their work, their security requirements, and their culture of readiness. This means that training for standard operations is often formulated as standard operating procedure similar to combat LINE Training that is highly structured. Cyber conflict, however, operates across multiple types of government and private infrastructure administered with many types of professional practice. As different agencies move to more collaborative and agile defense postures in support of this heterogeneous cyber infrastructure, training exclusively for an independent formal style of operations might overlook significant complexity when preparing to support less structured civilian cyber infrastructure teams who are working in collaborative environments against malicious actors whose impetus may turn out to have arisen from organized crime, nation state activity, or loose collaborations of hactivists. This multiparty scenario creates a highly complex environment where agile collaboration and mutual awareness of differing operational practices would likely be required to achieve high levels of joint capability while maintaining legal boundaries. In the context of this emerging challenge, Regis University invited the leadership of the Colorado National Guard to the Rocky Mountain Collegiate Cyber Defense Competition (RMCCDC) and perceived the value of a more open collaborative training model the RMCCDC and other Regishosted competitive and collaborative handson events. The Collegiate Cyber Defense Competition (CCDC), developed and organized by the University of San Antonio [START_REF] White | The Collegiate Cyber Defense Competition[END_REF], is a nationwide cyber defense competition at the college level that has steadily grown to encompass all 50 States in the Union. The approximately seven teams from different institutions are composed of eight to ten students who maintain secure digital services while defending against active cyber attacks and addressing business challenges. In 2012 Regis University joined the CCDC as the last organizational unit, the Rocky Mountain CCDC (RMCCDC) [START_REF] Novak | Developing Cyber Competition Infrastructure Using the SCRUM Framework[END_REF]. In consequence of its RMCCDC experience Regis extended the lessons learned from the RMCCDC to its labbased online and classroom courses, professional user groups and local schools. The CONG leadership, in January 2013, observed a big training advantage in that the types of physical exercises (active scenarios using technology with challenges), tabletop events, and formal lecture training models employed at Regis were not exclusive to any sector or agency. Regis initiated the first effort to leverage this multilaterally compatible structure across agencies and civilian sectors through a successful Information Assurance Scholarship Program Capacity Building grant application submitted to the National Security Agency Education Directorate. The grant application mapped out an extensible model of collaborative capability building activities that would be applicable to the 50 United States of America and related territories. In addition, a significant number of individuals present from other national guard units and law enforcement agencies also participated.

As Steven Cooper et al well expressed the historical context of such efforts, [START_REF] Cooper | An Exploration of the Current State of Information Assurance Education[END_REF] cybersecurity training and education evolved through a wealth of sectorspecific training that has matured over the last 40 years, including the CISOstyle training that matured into SANS; the information security education programs that perhaps started as early as Queensland Institute of Technology in Australia offered a Master's level research degree in computer security 1986. Cyber security forces capabilities in departments of defense around the world began to mature as evidenced by the expansion of the U.S. Air Force Mission to incl ude cyberspace in 2005. [4] Alternately, a review of trends in cyber investigation as developed by federal and law enforcement [5] provide strong operational capabilities requirements that support the types of skills that would lead to situational awareness in support of effective response to cyber security incidents.

Regis and the other parties initiating this effort set as the primary goal to offer multiple agencies and civilian entities the opportunity to maintain relationships that enable rapid collaborative response and integrated skill development, raising the level of joint capabilities. In context, the collaborative's work extended an earlier ACM education model that places high value on a student's "working knowledge in actually using their skills to interact with the society at large." [START_REF] Hoffman | Exploring a National Cybersecurity Exercise for U niversities[END_REF] Individually each participating entity in the collaboration identified their own targeted results. The Colorado National Guard (CONG) expected to leverage the differential between commercial education and internal training to provide a broader experience. The CONG leadership also wanted to achieve a stronger socialization with state and civilian participants. The State of Colorado was expecting to extend their existing training programs and have regular skillrefresh events. Regis was hoping to improve their competition engines, and refine their AGILEbased management structure given a new and broader set of collaborative customers. In addition, Regis expected significant individual knowledge and technical capability benefits of faculty and students who participated in the event.

Regis University provided the technology that facilitated simulation, training, and tool walkthrough activities. These include enterprise layer 3 switches, multiterabyte class production storage network, blade server sets, virtual machine infrastructure, and remote console capabilities provided in team rooms. Regis provisioned this network previously for large multiteam competitive events, research sandbox environments, coursework laboratory space, and industry training events. As Regis deployed diverse configurations in support of this set of applications, the team developed an AGILEbased development and management model.

2

Study Methodology

This study uses case study methodology [START_REF] Smith | The case study: a useful research method for information management[END_REF] generally to present and analyze the collaborative efforts from its inception and initial deployment through the current expectations one year after the first event. This study provides background information on the expected needs and describes how the partners managed operation change, developed solutions, and provide mutual review of the outcomes. This study does not provide specific data on the technical outcomes of individual performance, operational details specific to participants, or descriptions of specific technologies used in the specific simulation scenarios as based on interviews, observation, and related references. Also it presents preliminary criteria by which events like this can use for assessment, based on stakeholder objectives.

3

Joint Training Event Design Activities

The leaders of each participating entity contributed specific resources to the effort and exposed specific needs in January of 2013 in order to allow for a collaborative design process based on gap analysis. The Colorado National Guard brought a strong functional cyber security vision and technical security expertise while needing a customized education and a collaborative way to socialize with operations partners in both private and public sectors. The State of Colorado brought a different but overlapping cyber security expertize in addition to their functional requirements. Regis University brought an infrastructure of handling joint physical exercises, a conference facility, and a pool of staff and students to facilitate activities. Regis hoped to use this set of events as an opportunity to improve training facilities and methods, gain faculty and student experience, and further develop the AGILE methodologies that it had begun to use in support of academic computer laboratory infrastructure.

The Xmodel represented in Figure 1 indicates both the joint effort of the three initial participants and the gateway they represent to larger groups. While the State of Colorado is an exclusive entity, it could also provide this model to other states in support of their own collaborative activities. The collaborative used many scenarios to build the events, but the primary scenario portrayed the exhaustion of a single private sector or agency's resources and the need to have rapid and wellprepped teams to provide escalated capabilities for incident response. The "Call in the Guard" scenario breaks down into the following steps:

1) A private entity or group of entities becomes overwhelmed by a cyber security event that has scale and scope significant to the State of Colorado. The entity(s) resources are exhausted and further action is called for.

2) The private entity calls the State of Colorado to provide assistance.

3) The Governor's Office assesses the situation and escalates by calling in the Colorado National Guard to provide cyber security support as "cyber smoke jumpers." In order for this sequence to be successful, wellinterfaced lines of communications need to be preestablished and welltested processes, plans, and roles should be invoked in a collaborative way. During the first collaborative session the leadership team realized that having more active participation from the private sector early on provided greater validation for the simulation and tabletop scenario events. Figure 2 represents the new model for interaction.

Fig. 2. Updated Xmodel for Collaboration and Cooperation Includes Private Sector

To ensure the value of the event for all participants, the formative group performed gap analysis of the sum of the contributed resources in relation to what would be necessary to fulfill the expected needs of each group. The Gap analysis of resources versus needs that took place before the first collaborative event was expected to be speculative at best, and the formative team decided to employ an iterative Agile methodology that provided onthefly tuning during events and significant tuning of activities and resources between events.

Capability Gap Analysis

The joint leadership team focused on the gap between current joint capabilities and desired future capabilities in regards to rapid and agile joint operations as deployed with ad hoc civilian teams that currently protect critical infrastructure. This required that a coordinated response happen rapidly with existing civilian infrastructure teams, and that these teams all be familiar with each other. In Support of this the joint leadership team identified within their agencies a need for mutual tool awareness. Considering a higher layer of coordinated incident management, they also set as a goal the coordinated implementation of disparate policies, such that each organization understands the boundaries and professional practices of their partners.

These expected operational needs led to the development of the multiagency collaboration.

The leadership identified through gap analysis the need for a lowpressure training environment that allowed for those policy sets to be meshed in a series of response scenarios. Each group would be aware of how their agency should respond, but would develop an understanding of other agencies' practices and procedures.

3.2

Logistical Hurdles of Interagency Collaboration

As the event came closer, the collaboration leadership team discovered logistical hurdles that were artifacts of the multisector effort. The national guard schedule required the events be set on the weekend, while state employees needed to get credit for their time off. The technical team at Regis needed to deprovision some of the technical infrastructure from regular education labs and deploy it exclusively for the weekend virtually overnight. This multiparty coordination challenge existed partly because both the National Guard and the State of Colorado staff saw great value in having the event take place live, face to face, in a location with a minimum of distraction. The physical exercise was the hardest thing to accommodate. The facilities needed a lecture hall large enough for all participants and relatively secure rooms for the teams defending active web services.

Regis University uses a pool of volunteer faculty, students, and partners to facilitate these types of events. Therefore, the Regis Staff works hard not to overburden any individual volunteer. To respond to this issue the Regis team uses an AGILEbased method to build into each development and deployment cycle the specific load of requirements that they are able to handle in one cycle.

Collaborative AGILE Development Methodology

Over the past 10 years, Regis has been developing an agile process by which infrastructure is maintained and deployed using a very small staff, each with other responsibilities, and a pool of volunteers from the affiliate faculty, alumni, students, and members of the professional community. This situation required an AGILE [START_REF]Manifesto for Agile Software Development[END_REF] development model that accommodated a significant flux in human resources, mix of donated, loaned, and core university equipment, and manages. Te Regis team named it the Framing Forward Model, and presented it to The Colloquium in 2014. Briefly, it is composed of three layers. In the event layer teams adjust, in realtime, active cyber scenarios to navigate towards successful completion. In the middle Workspace Layer, teams work jointly on a particular project develop scenarios to prepare for events. In the lowest Services layer, the department tracks what human and technical resources are available at any one time to manage demand and allocate resources to project workspaces. Using that model, the Regis staff and volunteers have met needs for a broad range of events like the Rocky Mountain Collegiate Cyber Defense Challenge, the Information Systems Security and Information systems Auditing and Controls Association professional development events, classroomsupporting laboratory environments, etc. This new challenge of providing crosssector handson events required a new level of agility.

Timeline of Events Across First Year of Collaboration

The collaboration included several types of events along the following general timeline. Between these major events the planning team designated intermediate activites where individuals or groups could prepare for upcoming events and focus on skill development in order to enhance their technical contribution to the events. The following bulleted list provides a general timeline of events. The multimonth cycle of learning that this pace incited yielded obvious results in discussions as participants convened at the beginning of each event.

• August 2013 Event 1 This event focused on incident response in a red/blue team format. The big lesson learned from this event was that the exercise needed to include more contextual details of operation and success criteria in addition to a standardized technical configuration While the annual timeline is fairly well structured, it is not ridged in that the Regis team adapted the events in real time and debriefed after every event, in order to feed input in to the AGILE development cycle. As new needs became evident, the joint leadership used the intervening time to tune the upcoming collaborative sessions.

Joint Exercise Collaborative Session Components

The Collaborative Leadership Team formulates each session agenda from separate components listed below. The collaboration leaders compiled this component list from preexisting academic education activities, military exercises, and traditional training methods that each partner brought to the table. However, the content of each component varies from session to session.

• Partner Mutual Introductions: This fostered mutual awareness of command hierarchy, jurisdictional boundaries, key standard practices, working vocabulary, and potential handoff opportunities • Physical Exercise : Participants from each collaborative area looked for handson skill improvement in key areas such as protocol analysis, infrastructure awareness and analysis, technology and configuration control, and live incident response.

Crosstraining of skills and supported introduction of technologies both played significant roles in challenge events. • Tabletop Scenario Walkthrough : Key personnel from each sector led a live walk through of a wide variety of scenarios that involved interagency handoff, joint activities, authority boundary maintenance, and resolution of hierarchy of authority. In addition discussions of escalation options, procedures for accessing resources, and review of role interactions took place as prompted by questions from the participants in the audience. The walkthrough led to a significant set of questions regarding governmental response to private sector events and how privacy, intellectual property protection, and access assistance/controls would be handled during events. • Technology Reviews : Private and governmental partners both brought tools to the collaboration by presenting them in review sessions. The tools covered a broad range of areas including the commonly used tools in the commercial world where joint activities are likely to occur. This set ranges from protocol analyzers and event log aggregators to scanners and detection systems. • Simulation Challenges: The Colorado National Guard provided access to a scenariobased participantvsmachine game that provided challenge events and learning opportunities. • Handson formal Tool Training : Various groups provided specialized forensic, investigation, and defense tools that supported joint operational awareness. The collaborative leadership maintains and supplements this list in preparation for each major event as needed. Various participants then formulate contents between events. Regis faculty advocated for maximizing reuse and efficiency by structuring modularity at each scale of deployment. This modularity facilitated constructing the event, designing each event component, and tuning the component performance on the fly based on immediate feedback or facilitator awareness of participant issues.

Analysis of Outcomes

Generally, the National Guard provided a clear afteraction report for the collaborative events. Results of this report proved useful in triggering event improvements as quickly as the very next event. The first major finding was that National Guard team members were organizing their leadership structures internally on an ad hoc basis rather than having it imposed by collaborative leadership or preexisting rank. The room observers noted that leadership and organization of National Guard teams participating in exercises was not formed with common expectations and in the future this would need to be addressed. The standard organization hierarchy that exists outside the scenarios does not appear to apply in a smallgroup incident response team as much as experience in technical response and knowledge on incident response procedures.

Second, the team learned that providing a clear process and procedure for self evaluation and skill assessment yielded significant value. Initially, without clear direction participation in assessments was erratic. The data from those evaluations were used internally by each collaborative entity for identifying training needs of the groups and as a basis for improving the event modules. While the collaborative leadership team did not track participant technical progress individually, they considered that moving to a personalized tracking model might be good. At this time the collaborative leadership team is reviewing logistical and interagency policy issues that might present challenges in this area. The specific point in the agenda when posttests were administered turned out to be significant. The collaborative leadership discovered in the first collaborative event that testing too close to the event end resulted in less participation and loss of attentiveness to detail.

The AGILE method employed by Regis to achieve successful events was not deployed inside the participants' teams during physical exercises. There was a significant delay in some groups particularly when physical exercises began. After discussion with the teams, the gap here may have been because the Regis facilitators were accustomed to working with greater ambiguity of objectives and rapidly selfforming into working groups, so some details had not been sufficiently addressed or described as the scenario was delivered. This suggests that in the future, the team building scenarios will need to design clearer expectations and more detailed structure to more fairly simulate an operational situation.

Developing common vocabulary regarding tools, policy, and logistics turned out to be a big factor linked to the expression of situational needs. Without clear understanding of the full vocabulary involved in a multiagency situation, miscommunication occurred through assumption or alternative interpretation. With a common vocabulary, efficiency appeared to increase among those groups.

The Colorado Guard had specific desires to cover specific technologies like realtime packet capture analysis, socialize the different groups, ensure all parties understand each sector's leadership hierarchy and roles, and build response team skills and relationships. This went a long way in making the gap analysis effective prior to the collaborative events. Regis' initial requirements were more ambiguous in terms of general improvement of processes, capability, and team knowledge. The challenge for the Regis facilitators was ensuring that the events ran smoothly on a technical and logistical level. The State of Colorado also came with more general goals of collaboration and training, but their early representation in the formative meetings meant that both individual agency and multiagency collaboration goals would be possible.

7

Conclusion

The Guard, the State of Colorado, and Regis have reaffirmed a strong commitment to continue with the Cyber Security Multiagency Collaboration for Rapid Response exercises. The collaborative extended the joint effort to more readily include the private sector. Within less than a month after the initial grant review the collaborative leaders established a publicprivate round table panel linking the public and private sectors. This round table both disseminated results of the grant activities and collected needs from the private sector in order to reset the next round of event goals. The next planned event extends the working relationship and expands the depth and breath of content areas resulting in an advancement of technical, communication and interpersonal skill and knowledge. The collaborative leaders' analysis of the first year's cycle also resulted in refinement of the Xmodel.

The leadership and the technical teams providing training resources continues to successfully use AGILE approaches as part of ongoing collaborative events. The largest adaptation of the AGILE model during the grant period was not in provisioning technical resources for the project, but in controlling a "protocol bleeding" of operational expectations of AGILE methodologies unchecked into assumptions about scenarios. It became clear that because of the need to focus on policy and protocol integration, many functions that the collaborative addressed could not be fulfilled with either AGILE modeling or the Framing Forward Model.

Fig. 1 .

 1 Fig. 1. Initial Xmodel for Collaboration and Cooperation

  • Intermediate time 1 During this time participants worked on security training, with some completing CISSP and Security+ certifications. • February 2014 Event 2 This event focused on cyber forensics and a review of the tools of investigation. Physical exercises included forensic challenges and vendors presented relevant products. • Intermediate time 2 During this time participants continued on certifications including Ethical Hacker training. • August 2014 Event 3 This event contained an array of policy, socialization activities, tabletop scenarios, a physical exercise, and malware response review. By this time, participation had grown to include six state guard units, representatives from two Centers of Academic Excellence, and State of Colorado IT and public safety personnel. • Intermediate time 3 Participants continue external training and certification in preparation for the next collaborative event.


Sagar Raina
email: sraina1@students.towson.edu

Blair Taylor
email: btaylor@towson.edu

Siddharth Kaza
email: skaza@towson.edu

Security Injections 2.0: Increasing Engagement and Faculty Adoption using Enhanced Secure Coding Modules for Lower-level Programming Courses

Keywords: integer overflow, buffer overflow, input validation, cs0, learning sciences, interactive learning modules, instant-feedback, auto-grading

Learning interventions based on modules are common in computer science education. Traditional learning modules that present a large amount of content in a linear format can lead to students skimming and skipping content resulting in lower student engagement and effectiveness. In this paper, we present theoretical support for increasing engagement and effectiveness of learning modules, describe a system that implements these principles, and discuss the results of a study across four sections of CS0. Using the Security Injections @Towson cybersecurity modules, we enhanced select modules by incorporating the e-learning design principles of segmentation and interactivity. The study compares student engagement between the current (1.0) modules and the enhanced (2.0) modules. The use of the enhanced (2.0) modules significantly increased student engagement and these results persisted across gender and race. Feedback from instructors indicates higher student and instructor interest in the enhanced modules; in spring 2015, more than 20 instructors are using the enhanced (2.0) modules.

Introduction

The recent focus on cybersecurity education has led to the development of cybersecurity learning materials across various academic institutions and organizations in the United States [START_REF]Curriculum Resources -Teaching Tools for Educators[END_REF]. Among such initiatives are web-based learning modules developed by the Security Injections @Towson project [START_REF] Kaza | Injecting Security in the Curriculum : Experiences in Effective Dissemination and Assessment Design[END_REF][START_REF] Taylor | Security injections: modules to help students remember, understand, and apply secure coding techniques[END_REF][START_REF] Turner | Security in Computer Literacy-A Model for Design, Dissemination, and Assessment[END_REF]. These learning modules target key secure coding concepts including integer error, buffer overflow, and input validation in various programming languages, for Computer Science 0 (CS0), Computer Science 1 (CS1), and Computer Science 2 (CS2); and general security concepts, such as phishing, passwords, and cryptography for use in Computer Literacy courses. The Security Injections @Towson modules have been found to be highly effective at improving security awareness and the ability to apply secure coding concepts [START_REF] Taylor | Security injections: modules to help students remember, understand, and apply secure coding techniques[END_REF]. In over six years of dissemination to over 150 institutions, the following issues have been observed: 1) instructors noted that students tended to skip content and proceed directly to lab exercises, and 2) large-scale adoption by instructors remains a challenge.

The modules originally developed in the project are web-based learning modules that follow a traditional linear format (we refer to these as 1.0 modules hereafter). This type of module design, that presents a large amount of content at one time, is common in many disciplines, but can lead to issues like skimming and skipping [START_REF] Protopsaltis | Towards a Hypertext Reading/Comprehension Model[END_REF][START_REF] Tseng | The Difficulties That EFL Learners Have with Reading Text on the Web[END_REF]. In this paper, we discuss the key issues that lead to skipping and skimming, including the amount of content presented on a screen and decreased student engagement. Presenting less content allows students to process information more easily and improves learning; increasing student engagement motivates students to learn [START_REF] Al-Samarraie | Can structured representation enhance students' thinking skills for better understanding of E-learning content?[END_REF][START_REF] Clark | e-Learning and the Science of Instruction: Proven Guidelines for Consumers and Designers of Multimedia Learning (Google eBook[END_REF][START_REF] Moreno | Interactive Multimodal Learning Environments[END_REF][START_REF] Protopsaltis | Towards a Hypertext Reading/Comprehension Model[END_REF][START_REF] Tseng | The Difficulties That EFL Learners Have with Reading Text on the Web[END_REF][START_REF] Zhang | Interactive Multimedia-Based E-Learning: A Study of Effectiveness[END_REF][START_REF]Handbook of Online Learning[END_REF]. We describe enhanced modules, Security Injections 2.0, that address these issues by utilizing segmentation and interactivity, and a study that tests the engagement of the new modules. In addition, the enhanced modules include an autograding functionality to facilitate easier instructor adoption.

To assess the effectiveness of enhanced learning modules towards student engagement and instructor adoption, we compare the Security Injections 1.0 modules to 2.0 and explore the following questions:

Q1: How can we improve the traditional web-based learning modules to reduce content skipping, increase engagement and instructor adoption?

Q2: Can the use of 2.0 modules with segmentation and interactivity increase student engagement compared to the 1.0 modules? Q3: Can the use of 2.0 modules with auto-grading increase instructor adoption compared to the 1.0 modules?

2

Literature Review

Web-based learning modules present content using hypertext. Individuals reading hypertext have a tendency to skip or skim the content [START_REF] Taylor | Using the World Wide Web in Undergraduate Geographic Education: Potentials and Pitfalls[END_REF]. In this section, we discuss: 1) why hypertext readers skip or skim the content, and 2) how can we improve interactivity in learning modules to engage learners.

Hypertext reading and content skipping

Hypertext is text that contains links to text, audio, video, or graphics in other documents or media resources, giving flexibility to readers to click any of the links to gain knowledge [START_REF] Destefano | Cognitive load in hypertext reading: A review[END_REF]. Hypertext readers often adopt a reading strategy which determines what to read and what to skim [START_REF] Destefano | Cognitive load in hypertext reading: A review[END_REF][START_REF] Lawless | Knowledge, Interest, Recall and Navigation: A Look at Hypertext Processing[END_REF]. This may lead more selective reading for lengthy documents. Readers who skim have a tendency to read the first half of the paragraph and if they determine that the gain of information is low, they may skip the other half of the paragraph and jump to the next paragraph [START_REF] Duggan | Skim Reading by Satisficing: Evidence from Eye Tracking[END_REF]. This skipping may lead readers to lose important information [START_REF] Duggan | Skim Reading by Satisficing: Evidence from Eye Tracking[END_REF][START_REF] Protopsaltis | Towards a Hypertext Reading/Comprehension Model[END_REF]. Selective reading may also lead to less in-depth reading, less concentration and attention towards the content [START_REF] Liu | Reading behavior in the digital environment: Changes in reading behavior over the past ten years[END_REF]. In addition, skipping of the content leads to poor learning [START_REF]Handbook of Online Learning[END_REF].

Research suggests that instead of presenting large amounts of hypertext content at once, the content should be broken into smaller chunks and presented one idea at a time on a single screen [START_REF] Al-Samarraie | Can structured representation enhance students' thinking skills for better understanding of E-learning content?[END_REF][START_REF] Clark | e-Learning and the Science of Instruction: Proven Guidelines for Consumers and Designers of Multimedia Learning (Google eBook[END_REF][START_REF] Moreno | Interactive Multimodal Learning Environments[END_REF]. This concept is referred to as segmentation. Segmentation improves processing of information in the working memory and makes recalling and retention of concepts easier [START_REF] Moreno | Interactive Multimodal Learning Environments[END_REF].

Improving interactivity in learning modules

Interactivity in e-learning is the -responsiveness to the learner's actions during learning‖ [START_REF] Moreno | Interactive Multimodal Learning Environments[END_REF][START_REF] Zhang | Interactive Multimedia-Based E-Learning: A Study of Effectiveness[END_REF]. Interactivity improves engagement and motivates students to learn [START_REF] Zhang | Interactive Multimedia-Based E-Learning: A Study of Effectiveness[END_REF]. The types of interactivity in e-learning environments include: dialoguing, controlling, manipulating, searching and navigating [START_REF] Moreno | Interactive Multimodal Learning Environments[END_REF]. In this paper, we focus on dialoguing and controlling.

Increasing interactivity with Dialoguing -Dialoguing occurs when the learner answers questions and receives feedback to his/her input. Dialoguing has been considered beneficial, as learners can relate the feedback to the current content [START_REF] Thalheimer | Providing Learners with Feedback-Part 1: Research-based recommendations for training, education, and e-learning[END_REF]. In elearning, dialoguing can be implemented using assessments (like formative questions) with appropriate feedback [START_REF] Thalheimer | Providing Learners with Feedback-Part 1: Research-based recommendations for training, education, and e-learning[END_REF]. This assessment can be conducted using multiplechoice, fill-in-the-blank, short answer and essays formats. The feedback provided on the answers can be either immediate or delayed. The feedback provided can also be classified based on the amount of detail provided on the answers [START_REF] Van Der Kleij | Effects of feedback in a computer-based assessment for learning[END_REF][START_REF] Thalheimer | Providing Learners with Feedback-Part 1: Research-based recommendations for training, education, and e-learning[END_REF] and is categorized as: 1) Knowledge of Results (KR) , 2) Knowledge of Correct Response (KCR) and, 3) Elaborate Feedback (EF).

Knowledge of results (KR) informs the learner if their answer is correct or incorrect; knowledge of correct response (KCR) informs if the answer is correct or incorrect and includes the correct answer; and elaborate feedback (EF) informs if the answer is correct or incorrect, includes the correct answer, and also includes a concise explanation of the correct answer.

We plan to implement dialoguing in 2.0 modules using Multiple Choice Questions (MCQs), true or false and constructed response (CR) type of assessments with immediate knowledge of results (KR) and immediate elaborate feedback (EF).

Increasing interactivity with controlling -Controlling implies that the learner can determine the pace of the presentation. Controlling helps students learn better by allowing them to process information at their own pace [START_REF] Clark | e-Learning and the Science of Instruction: Proven Guidelines for Consumers and Designers of Multimedia Learning (Google eBook[END_REF][START_REF] Moreno | Interactive Multimodal Learning Environments[END_REF]. We plan to implement controlling in 2.0 modules using answer-until-correct [START_REF] Thalheimer | Providing Learners with Feedback-Part 1: Research-based recommendations for training, education, and e-learning[END_REF] with immediate knowledge of results (KR) and immediate elaborate feedback (EF) to MCQs, true or false and constructed response (CR) type of assessments.

Segmentation breaks large content into smaller chunks and presents them one at a time which may result in less reading and less skipping of content. Less skipping of content leads to increased learning [START_REF]Handbook of Online Learning[END_REF]. Interactivity (dialoging and controlling) on segmented chunks leads to engagement and enforces learning (see Fig. 1.). The Security Injection learning modules 1.0 were developed on the cognitive learning principles of Bloom's taxonomy and adopt a uniform structure. Each module begins with a background section to describe the problem with examples, followed by a -Code Responsibly‖ section (that includes methods to avoid security issues), a laboratory assignment with a security checklist, and discussion questions. The module structure is designed to help students to first understand the problem through the background and code responsibly sections, remember it through the laboratory assignments, evaluate it through checklists, and apply the concepts learned through discussion questions.

In the 2.0 modules, we enhanced the 1.0 modules by applying segmentation to reduce skipping of content, and interactivity, using dialoging and controlling, to improve student engagement. We implemented segmentation by breaking up the module content per section (background, code responsibly, laboratory assignment, discussion questions) and presenting each section, one at a time, on the screen (see Fig. 2.). We implemented dialoguing using formative assessment including true or false, multiple choice and constructed response that include immediate knowledge of results (KR) or elaborate feedback (EF). We implemented controlling using answer-until-correct giving the students control over the number of attempts. Students receive immediate knowledge of results (KR) feedback until third attempt and immediate elaborate feedback (EF) thereafter.

Module Design -In the background and code responsibly sections, students are required to go through the content and answer a set of checkpoint questions. Each question provides immediate feedback on submit (see Fig. 3. and Fig. 4.). The student cannot advance to the next section until all questions are answered correctly. In the laboratory assignment and discussion question, students answer text-based, multiple choice questions, and identify vulnerabilities based on a security checklist (see Fig. 5.). These are also auto-graded.

Fig. 2. Modules broken into sections

. Auto-graded security checklist

System Implementation -To implement the enhanced modules, several solutions were considered (including writing the system from scratch) before determining that a modified version of Stanford University's class2go web-based application (https://github.com/Stanford-Online/class2go/) was most appropriate. Class2go is built using the Django framework. Class2go is an open-source framework that provides core functionality, including user registration, course creation, test administration, and some components for auto-grading. We wrote code to auto-grade the security checklist, and regular expressions that match keywords to verify answers for shortanswer questions (constructed response). In addition, we developed an instructor dashboard with a progress grade-book to monitor student progress (see Fig. 6.).

Methodology

A quasi-experimental study was conducted in fall 2014 across four sections of a CS0 course (using C++) at a large public university, using a posttest only control group design. Two of the sections used the 1.0 version (control group) and the other used 2.0 (treatment group). The study was conducted during the laboratory sessions which were at different times for each section. Three modules -integer error, input validation and buffer overflow -were introduced, in that order, with approximately four weeks between the interventions. Both groups were administered a student engagement survey at the end of the semester.

The survey instrument used in this study measured student demographics and student engagement. While questions related to student demographics were derived from previous security injection studies [START_REF] Taylor | Security injections: modules to help students remember, understand, and apply secure coding techniques[END_REF], which measured students' gender, age-group, ethnicity and major, we adapted a set of eight item questions from a well-tested User Engagement Scale (UES) [START_REF] O'brien | The development and evaluation of a survey to measure user engagement[END_REF] to measure student engagement. The eight item student engagement questions were recorded on a five point Likert scale. (See Table 1 for sample survey questions.)

Based on the survey scores, we proposed the following hypothesis to compare Security Injections 1.0 and Security Injections 2.0 (treatment group) on the following dependent variable: student engagement score. H1: The mean of survey scores for student engagement in the treatment group will be significantly higher than the mean of the survey scores for student engagement in the control group.

Table 1. Survey questions

Student engagement

Q1

I felt deeply engrossed while completing security injection modules using this web-based platform.

Q2

I get so involved while completing security injection modules using this webbased platform that I forget everything.

Q3

While completing the security injection modules using this web-based platform, I tend to block out conversations with others around me. Q4 The Security Injection modules presented on this platform hold my attention.

Q5

Using this web-based platform excited my curiosity to learn cybersecurity principles.

Q6

Time seemed to go by very quickly when I use this web-based platform for completing Security Injection module.

Q7

The screen layout of this web-based platform for Security Injection modules was visually pleasing.

Q8

Using this web-based platform for completing Security Injection modules was attractive.

Initial Results

A total of 116 students participated in the study. After filtering missing data and outliers, 80 (42 in the treatment group and 38 in the control group) students including 54 (29 in the treatment group and 25 in the control group) males and 26 (13 in the treatment group and 13 in the control group) females completed the survey. For student engagement, each response was assigned codes from 1 to 5, on a five point Likert scale, 1 representing ‗strongly disagree' and 5 representing ‗strongly agree'. The engagement score for each respondent were calculated as the mean of codes for eight questions. The Cronbach's alpha, for eight-item engagement questions, was found to be 0.74, which suggested good internal consistency. The hypothesis was tested using independent samples t-test. We picked independent samples t-test because Shapiro-Wilk test showed that scores for student engagement in both the groups (treatment n=42, control n=38) satisfied the conditions of normal distribution (treatment p=.593, control p=.187) and homogeneity of variance (F=2.554, p=.114 > 0.05).

Comparison of survey scores for student engagement in treatment and control groups.

In the survey results, the mean score for the treatment group (n=42, mean=3.43) was found to be significantly higher at 95% level (t=-2.265, p=0.026) than the mean score for the control group (n=38, mean=3. [START_REF]Curriculum Resources -Teaching Tools for Educators[END_REF]). This implies that students found Security Injections 2.0 more engaging than Security Injections 1.0 (see Fig. 6a.). This leads us to accept H1 and supports research question Q2. In addition, higher engagement persisted across gender (see Fig. 6b.) and race (see Fig. 6c.). Comparing results for individual survey questions between control and treatment groups.

The student engagement mean score for the treatment group was found to be higher than the control group (see Table 2). In particular, the scores for Q6 and Q8 were found to be statistically significant at 95% level (refer to Table 1 for survey questions).

5

Instructor Feedback Security Injections 2.0 modules were introduced to over 50 faculty members in workshops conducted last year. In the past year, five instructors have used the beta system for both in-class and online instruction to approximately 180 students. Introduction of the modules at the workshops has led to overwhelmingly positive response among instructors who look forward to incorporating the 2.0 modules in their curriculum, primarily due to the auto-grading functionality. One workshop attendee indicated, -I am excited about the interactive modules (injection 2.0). I will be more likely to incorporate these modules.‖ An instructor who used both versions of Security Injections modules in a class of 30 students in summer 2014 commented, -The students were able to access the materials easily, and seemed to enjoy the interactive modules more.

Conclusion

In this study, we designed a system of enhanced secure coding learning modules that implement the principles of segmentation; by breaking content from 1.0 into individual sections per screen with checkpoint questions, and interactivity; by including instant-feedback. We conducted a study to test their effectiveness towards increasing student engagement across four sections of CS0 using the post only control group design. In addition, we collected instructor feedback.

Results from the study showed the segmented and interactive 2.0 modules led to significant increase in student engagement. Additionally, higher engagement persisted across gender and race. Feedback from instructors indicates both higher student and instructor interest. In addition, the increase in use of enhanced modules by instructors indicated higher instructor adoption.

Due to the limited instruments available to assess whether segmentation leads to less skipping and skimming, we plan an observational study in fall 2015 to record student behavior using segmented and interactive modules.

In addition, in future work, we plan a full scale experiment across multiple sections and courses, to examine the effectiveness of the enhanced modules, towards our goal of increasing secure coding knowledge among computer science students.

Fig. 1 .

 1 Fig. 1. Literature suggests that segmentation and interactivity in modules may increase learning

Fig. 3 .Fig. 4 .Fig. 5

 345 Fig. 3. Instant-feedback on incorrect answer

Fig. 6 .

 6 Fig. 6. Instructor grade-book

Fig. 6 .

 6 Fig. 6. (a) Average student engagement score in treatment and control group (b) Average student engagement score between males and females in treatment and control group (c) Average student engagement score between ethnic groups in treatment and control group

Table 2 .

 2 Results of individual survey questions

		Mean Score
		Control (n = 38) Treatment (n = 42)
	Q1	3.39	3.48
	Q2	2.63	2.74
	Q3	2.89	3.07
	Q4	3.37	3.45
	Q5	3.34	3.33
	Q6	3.08	3.79*
	Q7	3.50	3.79
	Q8	3.29	3.79*
	Student Engagement Mean Score	3.19	3.43*
	*p < 0.05 (statistically significant at 95% level)

  They were also able to get feedback faster, and seemed more comfortable with it compared to the other version.‖ In spring 2015, approximately 20 instructors are using 2.0 modules. Security Injections 2.0 modules are developed on the principles of learning theory and with the intention of facilitating wide-spread instructor adoption. Initial feedback indicates that segmented and instant-feedback based modules are likely to increase instructor adoption and student interest, which supports our initial result and third research question.

Acknowledgements

Class2go (https://github.com/Stanford-Online/class2go/). This project is partially supported by NSF DUE-1241738.


Gamze Canova

Melanie Volkamer

Clemens Bergmann

Roland Borza

Benjamin Reinheimer

Simon Stockhardt

Ralf Tenberg

Learn To Spot Phishing URLs with the Android NoPhish App

Phishing is a prevalent issue in today's Internet. It can have financial or personal consequences. Attacks continue to become more and more sophisticated and the advanced ones (including spear phishing) can only be detected if people carefully check URLs -be it in messages or in the address bar of the web browser. We developed a game-based smartphone app -NoPhish -to educate people in accessing, parsing and checking URLs; i.e. enabling them to distinguish between trustworthy and non-trustworthy messages and websites. Throughout several levels of the game information is provided and phishing detection is exercised in a playful manner. Several learning principles were applied and the interfaces and texts were developed in a user-centered design.

Introduction

The financial benefit of phishing [START_REF] Ramzan | Phishing attacks and countermeasures[END_REF] is an incentive for phishers to keep luring victims into disclosing their sensitive information. The anti-phishing working group registered more than 120.000 unique phishing attacks in the first half of 2014, i.e. more than 120.000 impersonated websites [START_REF] Aaron | Global phishing survey: trends and domain name use in 1h2014[END_REF]. Furthermore, they report that the average up-time of phishing websites was 32 hours and 32 minutes. During this time potential victims have to be self-reliant, i.e. they have to check the URL in order to know whether the destination is trustworthy. People could be supported by tools such as the Netcraft Extension [3]. However, such tools can never provide 100% accuracy [START_REF] Sheng | Anti-phishing phil: The design and evaluation of a game that teaches people not to fall for phish[END_REF]. Therefore, the tools' checks need to be complemented by humans checking the URLs. Yet, many people lack the required knowledge to properly check URLs [START_REF] Dhamija | Why phishing works[END_REF][START_REF] Lin | Does domain highlighting help people identify phishing sites?[END_REF] and assess the trustworthiness of a given website. Some people are even not aware of faked messages and websites at all and, thus, are not aware that they should check URLs before providing sensitive information [START_REF] Li | Larx: Large-scale anti-phishing by retrospective data-exploring based on a cloud computing platform[END_REF][START_REF] Dhamija | Why phishing works[END_REF].

Several solutions have been proposed to address the problem of lacking knowledge e.g. tutorials or guides [START_REF]Phishing: Gefährliche umleitung für ihre passwörter[END_REF][START_REF] Onguardonline | Phishing[END_REF], quizzes [START_REF] Online | Race to stay safe[END_REF][START_REF]SonicWALL: Sonicwall phishing iq test[END_REF] and games [START_REF] Sheng | Anti-phishing phil: The design and evaluation of a game that teaches people not to fall for phish[END_REF][START_REF]Anti-phishing phyllis[END_REF]. Tutorials are read-intensive if they cover all the different channels (such as email, SMS, QR codes, instant messaging, and social media) and URL spoofing tricks phishers exploit. The quizzes we are aware of do -if at all -indirectly educate people based on the feedback whether given answers are correct or incorrect; i.e. they do not explain why answers are correct/incorrect. In addition, they do not cover all different channels and tricks phishers exploit. The game Anti-Phishing Phyllis [START_REF]Anti-phishing phyllis[END_REF] only focuses on the email channel. Anti-Phishing Phil 1 and 2 [START_REF] Sheng | Anti-phishing phil: The design and evaluation of a game that teaches people not to fall for phish[END_REF][START_REF] Aleven | Anti-phishing phil v2.0[END_REF] are already rather advanced in terms of different URL spoofing tricks; but can still be improved by including awareness aspects, addressing different channels, explaining the structure of a URL more precisely, addressing more categories of URL spoofing tricks, and providing knowledge about HTTPS.

Our goal 1 was to develop a new game -NoPhish -an anti-phishing education app that addresses these issues to provide more sophisticated knowledge on how to properly check URLs -be it in messages or in the address bar of the web browser. We opted for an Android smartphone app since in particular smartphone users are much more likely to access phishing websites than desktop users [START_REF] Canova | Nophish: An anti-phishing education app[END_REF]. The detection of phishing on mobile browsers is complicated because in many cases address bars disappear and even if shown only parts of the URL are visible. Furthermore, a smartphone app provides the opportunity of casual gameplay and thus, compared to desktop solutions, more flexibility and time-independence. Thanks to the Google Play Game Services it is also possible for users to compare their performance with others. This adds a supplementary challenge and a motivational aspect, both key characteristics of gameplay and therefore successful learning. Furthermore, NoPhish integrates a number of learning principles recommended by literature.

Preliminary Considerations

Required Skills

Earlier phishing attacks could be detected by checking messages and websites for spelling and grammar mistakes or for design flaws. But since phishing attacks get more advanced, the URL is the only reliable indicator for the authenticity and trustworthiness of messages and websites (note, assuming a non-compromised system). Correspondingly, the following skills are required to successfully protect against phishing:

-Being aware that messages, links and websites can be easily impersonated; -Knowing that the URL is a reliable indicator for the authenticity and trustworthiness of a website rather than a website's content; -Knowing how to access and view entire URL; -Knowing how to parse the URL properly; -Knowing different URL spoofing tricks; -Knowing the importance of HTTPS when entering sensitive data.

Target Group

With NoPhish, we address in particular people who lack knowledge regarding all the aspects listed in Section 2.1; but people who use the Internet frequently.

We assume that NoPhish users have a general interest in learning to protect themselves (as they decided to install the app). Furthermore, our target group are Germans2 .

Learning Focus

URLs can either be checked using the URL preview function (cf. Figure 1) provided by several email clients and mobile browsers or directly in the mobile browser's address bar. We decided to educate people how to check URLs in the address bar for the following reasons: First, not all applications provide such a preview function, e.g. Android's (e.g. version 4.2) standard email client (e.g. version 4.4.2) does not provide such a preview function. Second, well-crafted URLs can still deceive users since the preview URLs are cropped in case they are too long (cf. Figure 1). We are aware that this decision comes with the disadvantage that users might fall for an attack by clicking on a link already, e.g. they could download malicious software. This issue is addressed by NoPhish in the final remarks (cf. Section 3.5).

Categorization of URL Spoofing Tricks

Phishers apply several URL spoofing tricks. Different tricks should be explained in different levels of NoPhish. There are several approaches to categorize URL spoofing tricks [START_REF] Sheng | Anti-phishing phil: The design and evaluation of a game that teaches people not to fall for phish[END_REF][START_REF] Lin | Does domain highlighting help people identify phishing sites?[END_REF][START_REF] Boodaei | Mobile users three times more vulnerable to phishing attacks[END_REF]. We propose to use a different categorization; one that is based on the difficulty to detect the corresponding spoofing trick. Such a categorization is most appropriate for the later leveling. Correspondingly, we identified the following categories (note, the examples are taken from PhishTank): a) IP Address URL without Brand: Sometimes phishers do not even bother registering any domain at all. In this spoofing trick, the host area of the URL contains an IP address while the path part does not contain the brand name, e.g. http:// There are some more URL spoofing tricks which either cannot be recognized by the human eye (e.g. homograph attacks [START_REF] Garera | A framework for detection and measurement of phishing attacks[END_REF]) or are irrelevant for our setting because they are redirected URLs such as tiny URLs [START_REF] Gabrilovich | The homograph attack[END_REF] or cloaked URLs [18]. After succefully completing the last level of the game, some general remarks on these issues are provided to the users (cf. Section 3.5).

Learning Principles

This section explains the principles of learning according to [START_REF] Alnajim | Fighting internet fraud: anti-phishing effectiveness for phishing websites detection[END_REF][START_REF] Thorndike | The fundamentals of learning[END_REF] which are essential for increased learning performance:

Readiness: The principle of readiness states that motivation is crucial for effective learning. Note, due to the definition of the target group we assume that NoPhish users entail readiness.

Exercise:

The principle of exercise is composed of two aspects: First, training and repetition help increase learning. Second, feedback is crucial for good learning performance. Ideally, these two aspects are applied in combination.

Effect:

The principle of effect states that people who associate their learning with positive feelings, e.g. early successes, learn more and better while on the other hand, negative feelings can decrease the learning performance. Correspondingly, enabling early success and maintaining people's motivation with positive feedback is crucial for successful learning.

Intensity: The principle of intensity states that learning is encouraged by things that are more intense. E.g. people are likely to learn more from an exciting and enthusiastic teacher than from a boring and monotone one or from a text book.

Primacy: The principle of primacy states that the first thing people learn makes the strongest impression; i.e. it should be started with important content.

Gamification

The following game elements are used in most modern games [START_REF] Murphy | Why games work and the science of learning[END_REF][START_REF]Badgeville: Game mechanics[END_REF] and are important for a good game experience:

Lives: An inherent element of a game is the possibility of losing it. If users are not able to lose a game they have no incentive to win it or play it. At the same time, one does not want the user to lose the game directly as the result of one minor mistake. Therefore, most games have some kind of "you have N tries"-element, which is commonly referred to as lives.

Levels: Leveling serves multiple purposes: First, it is important for the users to get a feeling for the progress they make. Second, it provides fixed points in the game from where they can restart or pause and continue the game later on. Finally, it enables to increase the difficulty of the game with increasing levels.

Achievements: Achievements are special elements of a game that users can unlock if they, e.g. find a special object or if they play a certain level exceptionally well. This is in particular for people who are willing to invest a lot of time in a specific level in order to finish it perfectly or to find every hidden secret in it.

Leaderboards: A leaderboard is an area where a user can compare the own progress in the game with the one of other users. The comparison with others motivates people to improve skills relevant for the game resulting in better performance.

User-Centered Design

The design and implementation of a user friendly and understandable app as described by Abras et al. [START_REF] Siering | Gamification: Using game-like elements to motivate and engage students[END_REF] is achieved by giving extensive attention to the users' needs and wants as early as possible.

This section elaborates on the game design and explains how the identified aspects of Section 2 are addressed.

Initial Survey

Before elaborating on the app design and implementation we inteded to get an idea of the users' preferences with regard to an anti-phishing education app. Thus, we ran an initial user survey where we asked users whether they would prefer a rather neutral game or a comic style education game with e.g. a fish as main character. The results of our survey confirmed previous findings by Volkamer et al. [START_REF] Abras | User-centered design[END_REF] that for a German audience (adults at least) a rather neutral game-based approach would be best accepted.

Learning Content per Part

The app entails two introductory parts, the game with nine levels, and a final remarks part. Table 1 shows the link between the skills to properly judge on the trustworthiness of websites (cf. Section 2.1) and the different parts of NoPhish.

Introduction Parts

After a short introduction to the problem of phishing and its consequences, NoPhish starts with two introductory parts. The awareness of spoofed messages, links, and websites can be best addressed by actually sending corresponding messages and access and view entire URL can be best exercised in the mobile browser. Thus, these two parts are seperated from the actual levels of the game.

Part 1 -Awareness of Spoofed Messages, Links, and Websites: First, users are made aware of how simple it is to spoof messages, e.g. emails [START_REF] Volkamer | Adopting the cmu/apwg anti-phishing landing page idea for germany[END_REF]. This is done by enabling them to send themselves with the NoPhish app an email from a sender address they provide in a corresponding form; and with a content they provide there as well. After submitting the form, NoPhish requests the users to check their email inbox. The received email contains information that this email was sent by NoPhish and that users should check the "from field" to notice that this is actually the email they prepared. Furthermore, the email contains a link with the displayed text "https://www.google.de/" and users are asked to follow the link. Clicking on this link redirects the users back to the app. Thereby, users learn by experience that they should not trust displayed link texts. Back in the app some further information is provided, e.g. that website spoofing is simple as well. Finally, the user is informed that this kind of forgery is not only possible with emails, but also with other forms of communication, such as social networks, SMS or instant messaging systems.

Part 2 -Access Address Bar and View Entire URL: This part teaches the users how to access and view the entire URL in the mobile browser. In detail, the users are told (1) that they need to scroll up the entire website to make the generally hidden address bar reappear; and (2) that they need to tap the text field of the address bar and scroll to the left in order to view the entire URL.

The explanations how to do so are supported with corresponding screenshots. Due to the learning principle of exercise (cf. Section 2.5) after the explanations an exercise follows. Here, users are required to access the URL of a website they are forwarded to by NoPhish. Note, forwarding happens in a way that users first have to scroll up. On top of the page, there is a text field, where they are asked to enter the last four characters of the URL. Then, they are asked to identify the first word of the URL (check one out of four provided possibilities). Once submitted the app checks the users' answers and can thereby ensure that they managed to access and view the entire URL. The users are forwarded to NoPhish as soon as they successfully complete the exercise. At the end of the exercise the users are told that URLs are the only reliable indicator for phishing. They also learn that they should always access the URL just as learned and that all other displayed links might be spoofed (by referring to the previous part). This part closes by explaining that URLs need to be carefully checked and that they learn how to do so throughout the coming parts of the app.

Gaming Part

The gaming part is split into nine levels with increasing difficulty. Each level consists of two parts: an introductory block and the actual exercise. For the introduction of URL spoofing tricks the introductory block consists of a reminder 4 , which provides a summary of previous levels (cf. Figure 2 The exercise is designed in a playful manner, i.e. users start with three lives, represented by hearts, and can collect points for correct answers and lose points and lives for wrong ones.

Users receive direct feedback on their decision. If the given answer is correct the users are rewarded by gaining points and a smiley face. This is relevant for the increase of positive feelings (cf. learning principle of effect in Section 2.5). If the answer is wrong the users lose points and a life. The users are immediately told why their answer was wrong. The next level is achieved if and only if a predefined amount of phishing and legitimate URLs have correctly been identified.

To simulate the "behavior" of the address bar in mobile browsers, the entire URL as such is not displayed but only parts of it. The user needs to scroll to the start of the URL in order to decide about the legitimacy of the displayed URL.

By practicing the learnt content and providing direct feedback we address the principle of exercise (cf. Section 2.5). Note, by making use of levels and lives two of the four gamification elements are already addressed (cf. Section 2.6). The elements achievements and leadersboards are realized by means of Google Play Game Services. Finally, NoPhish by nature provides intensity (cf. learning principle of intensity in Section 2.5) as it is a game.

Level 1 -Structure of a URL: It is essential for people to achieve the capability of parsing a URL properly before learning different URL spoofng tricks. Especially the identification of the domain (first-and second-level domain) in a given URL is a key aspect which needs to be covered extensively. Therefore, the users start learning to identify the domain of a URL in level 1. To explain the different parts, we do not use technical terms such as URL, domain, subdomain, protocol and only provide details users need to know to successfully detect phishing URLs. The focus of this level is the domain, the Who-Section as we refer to it in NoPhish. During the exercise the users are asked to tap on the Who-Section. Levels 2-8 -URL Spoofing Tricks: In levels 2-8, the various URL spoofing tricks (cf. Section 2.4) are addressed. In level 2, we also explain IP addresses by using the analogy of house addresses (with street names and numbers).

During the exercises, URLs together with the name of the website the users are supposed to visit are displayed (cf. Figure 3(a)). Users are asked to decide whether they are legitimate or phishing ones. Note, that in all levels both, HTTP and HTTPS URLs are displayed to the user, i.e. legitimate as well as phishing URLs can use HTTPS. This way, we prevent users from deciding based on the protocol.

When the users correctly identify a phishing URL, NoPhish asks them to tap on the Who-Section (cf. Figure 3(b)). This way, we aspire to ensure that they understood where to look at and did not just guess the answer. Depending on the Note, in level n the number of URLs that need to be properly judged is 6 + 2 * (n + 1). The learning principle of repetition (cf. Section 2.5) is applied as each URL spoofing trick introduced in level n is tested in the exercises of later levels, too. About half of the phishing URLs (more precisesly (6 + 2 * (n + 1))/4 in level n) are repetitions 5 from previous levels. The first level that contains repetitions is level 3 because level 2 introduces the first URL spoofing trick.

Level 9 -HTTPS: In this level, we introduce the difference between HTTP and HTTPS. We explain, that HTTPS represents the higher security level and that this means (1) that the conversation cannot be eavesdropped by someone having access to the network and (2) that the communication partner indicated in the Who-Section proved his/her identity to a trusted authority if no warning is shown in the browser.

Furthermore, we tell users that there exist many legitimate websites without HTTPS (by default). We advice to try to switch to HTTPS. We explain the consequences if sensitive information is entered on a website without HTTPS (while assuming that the domain name is authentic). This level also includes an exercise. However, the question is changed. The users are asked whether they would provide sensitive information on the website. Thus, the users need to check whether a URL provides HTTPS and whether the domain name of the URL is spoofed or not.

Final Remarks Part

When the users reach the final remarks part they are well prepared for the detection of phishing URLs (cf. Section 2.1). Yet, there are some special cases the users are made aware of in this part of NoPhish. E.g. users are informed about further potential URL spoofing tricks that have not been exercised (redirected URLs and homograph attacks). Also, the users are explained that they might encounter URLs which look very phishing-like, but actually are legitimate, e.g. https://www.paypal-community.com. In such a case, NoPhish suggests to directly contact the company and ask for the authenticity of the specific website, i.e. URL, before entering any data. Finally, NoPhish briefly introduces extended validation certificates and provides users with a link to further information on this topic. This part does not include an exercise.

App Development Process

This section gives a brief overview of our approach for the development of a user friendly and understandable app and summarizes the two most important steps regarding the development process -URL generation and user-centered design.

URL Generation

We generate phishing URLs by applying corresponding URL spoofing tricks to legitimate URLs. For the legitimate URLs 30 of the top 100 Alexa ranked domains (for Germany) were collected. The corresponding URLs have no path or subdomain, e.g. google.com. In order to also provide NoPhish users with longer URLs, we visited each of these websites, navigated through them and additionally picked three URLs for each domain, two long and one short URL. Thus, for each domain, a total of four URLs was added to the set of legitimate URLs. This set contains URLs with and without HTTPS.

For each URL to be displayed it is randomly decided which of the legitimate URLs is used. Then, it is randomly decided whether to show a legitimate or a phishing one based on the setting for the corresponding level. For the phishing URLs the corresponding URL spoofing trick is applied.

Applied User-Centered Design Approaches

In addition to the initial survey, we involved potential users of NoPhish in the following ways: we iteratively built, tested, and improved mock ups (while focusing on the first three levels as the app flow does not significantly change from this level on). For instance, according to comments of participants, we simplified and clarified the descriptions on how to access the address bar. We also reduced the text per page in general. The texts for all levels were reviewed by two German language experts (who are no security experts).

The next step was a user study, where we decided to go for the low cost method of guerilla user testing [START_REF] Avoine | Computer system security: Basic concepts and solved exercises[END_REF]; i.e. participants reading the app texts while thinking aloud. We included a little exercise in order to assess whether the users comprehended the texts or not, i.e. for each introduced URL spoofing trick we included a small list of URLs on which the users had to decide whether they were phishing URLs or not. Finally, the users were asked to provide their general impression. The received feedback was integrated.

In addition, we applied a statistical method -once we improved the textin order to assess the legibility of our texts. Corresponding tools [START_REF] Simon | The art of guerilla usability testing[END_REF][START_REF]Stilversprechend: Stilversprechend[END_REF][START_REF] Lesbar | Testen sie ihren text[END_REF]] take a regular text as their input and return a legibility index as their output. The average index value of the three tools for the entire text used in the app is 62. Given a scale from 0 to 100, where an index of up to 30 indicates an academic level and 90 and above is considered easy to understand, an index of 62 is considered as reasonably comprehensive for teenagers [START_REF] Schöll | Flesch-index berechnen[END_REF]. Regarding our target group, this result is reasonable. However, it is something we have in mind for later user tests to evaluate.

Related Work

Section 1 already gives an overview of related work in anti-phishing education.

Here, we especially elaborate on Anti-Phishing Phil 1 and 2 [START_REF] Sheng | Anti-phishing phil: The design and evaluation of a game that teaches people not to fall for phish[END_REF][START_REF] Aleven | Anti-phishing phil v2.0[END_REF], as these games resemble NoPhish the most of all the anti-phishing education approaches.

Anti-Phishing Phil 2 uses a diver as main character (while Anti-Phishing Phil 1 uses a fish). A major improvement compared to Anti-Phishing Phil 1 is that the game emphasizes the importance of the domain. E.g. the users are asked to mark the part of a URL which indicates phishing. This is an aspect we included to NoPhish. Furthermore, the information texts are generally improved and extended to be more precise.

The design of Anti-Phishing Phil 2 as such could not be applied for our context due to the results of our initial survey. However, we carefully analyzed the structure and the educated content to decide what could be adapted and what we wanted to modify or improve: The provided information, is often not precise enough and sometimes even inconsistent. In one information text the host is introduced as the part between "http://" and the first "/" and the domain is right before the first "/". In another information text the domain is addressed by referring to the right hand side of the host area, which could be either only the top level domain or could also include subdomains. In addition, Anti-Phishing Phil 2 covers only four URL spoofing tricks (similar to Anti-Phishing Phil 1): subdomain tricks, IP addresses, domains with hyphens (part of derivated domain names) and replacing character(s).

The users are asked to decide on only three URLs per level which seems not to be enough in order to internalize the learnt content according to the principle of exercise. Also, URL spoofing tricks from previous levels are not repeated which is essential for good learning performance according to the learning principle of exercise. Finally, the difference between HTTP and HTTPS is not addressed as well as the contents of introduction parts 1 and 2 of NoPhish.

Conclusion and Future Work

In the scope of this work, we have designed and implemented an anti-phishing education app -NoPhish -in a user-centered design approach. In a playful manner, users obtain valuable information on how to detect phishing URLs, in particular on a smartphone. The detection of phishing URLs is realized as a game, where the user can win or lose points or lives.

We integrated learning principles and diverse gamification elements recommended in literature. In order to provide levels, with increasing difficulty to detect phishing URLs, we proposed a new categorization of URL spoofing tricks which we then explain in the different levels.

The app is divided into two main parts: the security awareness and the educational part. In the security awareness part the user is shown how simple it is to spoof emails and links by sending themselves a spoofed email with a spoofed link. In the educational part the user is taught how to access the URL and how to detect phishing URLs.

We already conducted a user study including a retention study which showed very promising results [START_REF] Amstad | Wie verständlich sind unsere Zeitungen? Abhandlung: Philosophische Fakultät I[END_REF]. In future, we also plan to assess how such an education app can best be distributed. An idea would be to utilize embedded learning [START_REF] Jansson | Simulating malicious emails to educate end users on-demand[END_REF] where simulated phishing emails are sent to users. Whenever users fall for such an email they could be proposed to download the education app. We also plan to extend the target group and consider kids and youth.

Figure 1 :

 1 Figure 1: Screenshot -URL preview function

  (a)) and the introduction of a new URL spoofing trick (cf. Figure 2(b)).

Figure 2 :

 2 Figure 2: Example introductory block

  (a) Exercise -Phish/No Phish (b) Exercise -Show Who-Section

Figure 3 :

 3 Figure 3: Example Exercise Screenshots

  Derivated Domains: A phisher can register a modification of the original do-

	b) Random/Unrelated/Trustworthy Domain, without Brand: This trick uses
	random/unrelated or trustworthy names or strings as domain name 3 and
	does not include the brand name of the targeted website in any other part of
	the URL. E.g. http://www.szuhsa.fr/login.html, http://www.weather.com/
	login.html or https://secure-payment.com to impersonate PayPal.
	c) Random/Unrelated/Trustworthy Domain, with Brand in Subdomain:
	A phisher can include the brand name into the subdomain of a URL in combi-
	nation with a random/unrelated/trustworthy domain name, e.g. http://paypal.
	mark-chippy.com/account-setup/ or http://www.amazon.account.com/.
	d) Random/Unrelated/Trustworthy/IP Domain, with Brand in Path:
	A phisher can include the brand name into the path part of a URL in combi-
	nation with a random/unrelated/trustworthy domain name, e.g. http://online-
	payment.com/www.paypal.com/. This attack can also happen in combina-
	tion with an IP address URL, e.g. http://5.178.64.164/paypal.
	e)
	5.178.64.164/secure to impersonate PayPal.

main. In this case the modified domain contains the brand name in some form, e.g. facebook-login.com can be registered in order to impersonate facebook.com. f) Introducing Typos: Phishers can register domains which resemble the targeted domain, but have a typo, e.g. the phisher can register micosoft.com to impersonte microsoft.com. One special case of the typo is swapping letters in the original domain name, e.g. mircosoft.com to impersonate microsoft.com. g) Replacing Character(s): A phisher can also exploit character resemblance, i.e. the phisher can register domains where characters are replaced by other similar characters, e.g. https://www.arnazon.com.

Table 1 :

 1 Skills -Levels -Assignment

	Taught Skill

Note, a summary of this paper is availalbe at[START_REF] Amstad | Wie verständlich sind unsere Zeitungen? Abhandlung: Philosophische Fakultät I[END_REF]

Note, this has an impact on the language as well as on the selected URLs and the design of the app.

Note, that we refer to the first-and second-level domain of a URL as domain.

Note, this reminder is not shown if the user immediately starts off the new level after having successfully completed the last one but only if some time passed since the user played previous levels.

Acknowledgements. This work was supported by CASED and EC SPRIDE.