Alaa Tolah
email: alaa.tolah@plymouth.ac.uk

Furnell

Maria Papadaki
email: maria.papadaki@plymouth.ac.uk

A Comprehensive Framework for Understanding Security Culture in Organizations

Keywords: Security Culture, Human Factors, Qualitative study. 1

Organizational security is exposed to internal and external threats, with a greater level of vulnerabilities coming from the former. Drawing upon findings from prior works as a foundation, this study aims to highlight the significant factors that influence the security culture within organizations. Phase one of the study reports upon an interview-based investigation undertaken with thirteen experienced, knowledgeable security specialists from seven organizations. The main findings confirmed the importance of the identified factors from the previous work. The focus to emerge from the interviews concludes that continuously subjecting employees to targeted training and awareness development improves security culture. Indeed, there was a clear lack of awareness and compliance regarding the implementation and clarity of security policies in organizations. Also, the inefficient training program and limit to specific employees in organizations leads to a lack of awareness and compliance.

Introduction

Nowadays, a knowledge-based economy has become more dynamic than at any time in history, which continues to progress. Technology enables all business operations and information technology to develop into a central concept for most aspects of life. However, information technology developments continually initiate new risks to the security of information assets. The use of ICT can make the violation of information security easier, while many technical approaches to security and counter-measures have augmented in organizations; although this development needs to increase, as many researchers perceive security to be both a "people" and "technical" issue [START_REF] Alhogail | A Framework for the Analysis and Implementation of an Effective Information Security Culture Based on Key Human Factor Elements and Change Management Principles[END_REF][START_REF] Connolly | Organisational culture, procedural countermeasures, and employee security behaviour: A qualitative study[END_REF]. Nonetheless, technology is less likely to cause problems than human error, which is a cause of the majority security breaches (75% of organizations suffer security breaches by insiders) [13]. Therefore, organizations need to focus on employees' behavior to achieve information security, as many studies examined the human factors and their relation to information security with social psychology issues that determine reasons for unacceptable behavior that leads to security breaches [START_REF] Parsons | The Human Aspects of Information Security Questionnaire (HAIS-Q): Two further validation studies[END_REF]. One approach the or-ganization can take to manage the security would be developing strategies that enhance security culture [START_REF] Alhogail | A Comprehensive Human Factor Framework for Information Security in Organisations[END_REF][START_REF] Da Veiga | A framework and assessment instrument for information security culture[END_REF].

A positive security culture contributes to support organizations in guiding employees to follow security policies, which lowers the potential risk of harmful information interaction by employees, as they develop knowledge and advance their skills correctly, and behave securely in their working environments [START_REF] Alhogail | A Framework for the Analysis and Implementation of an Effective Information Security Culture Based on Key Human Factor Elements and Change Management Principles[END_REF][START_REF]Growing positive security cultures -National Cyber Security Centre[END_REF]. The culture that promotes secure human behavior through knowledge, values, and assumptions is better than regulations that merely mandate employees' behavior [START_REF] Alhogail | A Comprehensive Human Factor Framework for Information Security in Organisations[END_REF]. Various studies suggest that organizational security culture could lead an employee to act as a "human firewall" [18]; where acting correctly is commonplace [START_REF] Roer | Indepth insight into the Human factor: The Security Culture Report[END_REF]. Consequently, organizations are required to use understandable guidelines to develop a culture of security awareness, which utilizes various approaches to improve comprehension [START_REF] Alhogail | A Framework for the Analysis and Implementation of an Effective Information Security Culture Based on Key Human Factor Elements and Change Management Principles[END_REF][START_REF]Growing positive security cultures -National Cyber Security Centre[END_REF].

The proposed paper extends the proposed framework by [START_REF] Tolah | A Comprehensive Framework for Cultivating and Assessing Information Security Culture[END_REF], which facilitates an understanding of security culture and its elements. It initially assists in comprehending whether the level of security culture enhances the security of information assets, and also assesses the relationship between influential factors and factors that constitute security culture.

The development of an initial framework is based on Alnatheer's model and a review of academic literature in the security culture. In the proposed framework, the security culture comprises several factors, as the components are structured into: factors that influence security culture (top management, security policy, security education and training, security risk assessment and analysis, and ethical conduct); factors that constitute security culture (security awareness, security ownership and security compliance); and factors of organizational behavior that contribute to workplace behaviors and could influence the security culture (personality traits and job satisfaction) (see Fig. 1). These factors appear to be the most influential factors and are considered as part of the security culture's conceptualization. By understanding the influential factors or reflection factors, it is possible to aid in directing the interaction of humans with information security. These factors provide management with a means to implement adequate security management approaches that include the guidance provision and control implementation in the assessment of security culture.

As this study relied on prior literature to develop an initial security culture framework and to explore a detailed perception and understanding of human behavior, it was considered that using an interview method would enable in-depth information from employees involved in an organization's information security, and understand factors affecting security culture with which are more critical from the perspective of participants. The main aims of the interview were to explore whether all the ten factors within the proposed framework are necessary; which factors are the most important, whether new factors should be incorporated, and to identify new issues that had not previously been considered to alter security culture potentially. Fig. 1. The comprehensive security culture framework [START_REF] Tolah | A Comprehensive Framework for Cultivating and Assessing Information Security Culture[END_REF] Therefore, this paper presents the findings from exploratory interviews with IT/security professionals and combines these with the literature review to investigate whether the identified factors are vital for organizations and to determine a base for more comprehensive research through quantitative techniques. The paper initially provides a review of related work for security culture frameworks. Subsequently, the current research method is described with the analytical approaches used to interpret the results. The paper concludes by outlining the research implications and future research.

Literature Review of Existing Security Culture Approaches

Instilling an effective culture is vital to create adequate levels of information security.

Various studies have focused on security culture, while literature reviews offer an overview that focuses on security culture (Alhogail [START_REF] Alhogail | A Framework for the Analysis and Implementation of an Effective Information Security Culture Based on Key Human Factor Elements and Change Management Principles[END_REF]; Glaspie and Karwowski [START_REF] Glaspie | Human Factors in Information Security Culture: A Literature Review[END_REF]).

Their literature analyzes concluded that most investigated issues in security culture relate to: the conceptualization of culture to identify concepts and factors that affect security culture, the creation of security culture, or an assessment of security culture to measure whether it is an adequate level. Many studies provide various approaches and models that highlight security culture's importance, promote its benefits and provide guidelines to create and assess security culture. The literature analysis showed that most studies demonstrated various essential factors that may shape or change security culture [START_REF] Alhogail | A Framework for the Analysis and Implementation of an Effective Information Security Culture Based on Key Human Factor Elements and Change Management Principles[END_REF]. A comprehensive review of security culture was conducted to gain an overview of the current security culture frameworks, which focused on studies that assess security culture, which presented essential knowledge with regards to fac-tors that assist in developing security culture [START_REF] Tolah | A Comprehensive Framework for Cultivating and Assessing Information Security Culture[END_REF]. Thirteen research perspectives relate to the creation of security culture and five studies incorporate an assessment of security culture. The security culture is a product of various factors, such as security policy and security training that affect the individual's behavior in organizations [START_REF] Tolah | A Comprehensive Framework for Cultivating and Assessing Information Security Culture[END_REF]. These studies have developed comprehensive security culture models and contributed to how organizations potentially create and maintain acceptable security culture levels.

Studies by Alnatheer [START_REF] Alnatheer | Understanding and Measuring Information Security Culture[END_REF] and Da Veiga and Eloff [START_REF] Da Veiga | A framework and assessment instrument for information security culture[END_REF] provide an approach that utilizes the same framework to assess security culture, which both provide statistically sound assessment instruments to perform security culture assessment. Few studies have provided reliable and valid security culture assessment instruments; Schlienger and Teufel [START_REF] Roer | Indepth insight into the Human factor: The Security Culture Report[END_REF] designed a questionnaire to detail how proper rules impact upon employees' security behavior, while Da Veiga and Eloff [START_REF] Da Veiga | A framework and assessment instrument for information security culture[END_REF] designed a security culture assessment tool. Moreover, there is minimal coverage of other influential factors, such as individual difference variables and job satisfaction. The positive impacts of these factors on workplace behaviors had proven by studies from D'Arcy and Greene [START_REF] D'arcy | The Multifaceted Nature of Security Culture and Its Influence on End User Behaviour[END_REF] and McCormac [START_REF] Mccormac | Individual Differences and Information Security Awareness[END_REF]. The literature review illustrated that there is a need for more investigation in the area to provide comprehensive frameworks and the best practices of security culture cultivation and assessment. Hence, this study proposed an initial framework that integrates the most important factors that influence security culture. The interview method was adopted to explore and examine the identified factors.

Methodology

This study used the pragmatic philosophical approach with mixed methods of data collection. The initial phase was a qualitative design to acquire sufficient information regarding security culture from IT/security specialists in organizations and to explore influential factors. The second phase will be quantitative data for a survey with a minimum sample size of 250 employees from several organizations to develop an understanding of the relationship between factors and to test the framework's validity. The findings from the qualitative phase will be incorporated with the literature review to identify constructs that influence security culture and related survey items. To enhance theoretical propositions, an exploratory interview was adopted to investigate whether all identified factors are necessary for organizations, which also assist in the survey design.

Sample

It is essential to select organizations from a broad range of sectors and industries, which may require various levels of security. It will help organizations to make individual decisions regarding security culture programs and guide investments in security awareness and training sessions [START_REF] Schlienger | Information Security Culture-from analysis to change[END_REF]. Access to appropriate organizations was difficult, as certain organizations are restricted against discussing security management. The initial choice of organizations' location was in the UK, due to the study's loca-tion. However, the low number of respondents in the UK resulted in an invitation email being sent to international organizations' members that had cooperated on past occasions in Saudi Arabia and the USA. The study managed to interview seven individual organizations: two in the UK, one in the USA, and four in Saudi Arabia, which covered private, public and semi-public sectors, and included various industries: four in education, and one insurance, law, and mining, respectively. These organizations have the security infrastructure in place; have technology adoption, and used security management practices. However, the diversity of organizations' geographical locations would assist in advancing the understanding of security culture from varied backgrounds.

A semi-structured was utilized in an exploratory manner, and there was no restriction in sample size. This study does not aim to demonstrate the representation of security culture; it aims to identify factors of more critical importance within organizations. This exploratory study could provide indications of the validity of factors intended in the framework. The results cannot be generalized, although it is possible that the findings from the quantitative survey in the next phase could be generalizable. Access to people with relevant knowledge and experience of organizations' security was sufficient. Nevertheless, only thirteen participants from the IT/Information security department agreed to participate in the interview; experience and knowledge of respondents yield rich data. Table 1 below lists the demographic features for participant organizations.

Data Collection

The primary method for collecting data for discourse analysis was individual interviews from October 2017-January 2018, conducted face to face or through Skype. An interview guide was designed and utilized during the interview to ensure data consistency and minimize bias. The interview questions consisted of four parts, with the first part on demographics and a general overview of participants. The second part

included questions with open-ended answers related to organizations' security practices, how employees are educated and acquire security regulations' awareness. The third part includes questions with open-ended answers that determine security culture behaviors, knowledge, and practices of employees; and part four enabled interviewees to construct individual interpretations. A pilot study with three respondents was conducted to ensure that the interview guide was appropriate, and participants understood the questions. Subsequently, the language was either in English or Arabic, depending on the preference of each interviewee, although the transcripts were transcribed in English. The average interview time was twenty-five minutes, while certain interviewees refused to have their interviews recorded, due to the perceived sensitive nature of their information, even though the identity of the interviewees and the data were kept confidential.

Data Analysis

A within-case analysis was used for data analysis, which includes detailed interviews. Each interview assists in providing insight into the identified factors and how different constructs are perceived within real life situations. The interviews provided data regarding security culture in the form of a narrative discussion, instead of analyzed interpretation. The understanding was obtained by reviewing the significant findings and reflecting upon personal views following a review of past literature. The data analysis process in the study includes developing a coding scheme to capture the data's critical content; applying the coding scheme to each identified comment made by the interviewee; and exploring the results' frequencies and patterns, where pieces are taken individually from the analysis and compared with other forms of data to determine which parts are similar or different to establish data correlations.

During the analysis, both explicit and implicit results derived from the interviews. By coding the data, the interviews were sufficiently analyzed, as this was a repetitive process that involved determining consistency and providing data links to the framework's constructs. The data from this repetitive process were correlated and coded into constructs during the analysis stage and resulted in data points for further analysis. Furthermore, data categories were identified, which are: top management, policy, education and training, ethical conduct, risk assessment and analysis, awareness, compliance, ownership and job satisfaction. The selected comments focused on various levels of analysis, such as the comments' structure, the overall content of what was said, and the vocabulary.

Results

The interview analysis depicted the perception of employees towards security culture. The findings of the interviews illustrated the relevant signify factors and correlation regarding security culture. The participants responded to the interview questions concerning their organization's practices: the security culture practices, the employee security behavior patterns, and the perceptions for improving security culture. Quota-tions are used to highlight explanations of the findings, followed by a symbol that denotes the organization's name and the participant's number [A1].

Security Culture Practices

This section determines information regarding security practices; how employees educate; and acquire the awareness of related security regulations and risks.

 viewees reported that their organization's employees are not alert to any security risks, such as B and G. Ten interviewees reported that all employees in the organization are alert to security risks and dangers inherited in the work environment.  Information Security Level in the Organization. To attain quantifies data, interviewees rated the level of information security in their organizations. Six participants believed that their organizations have a moderately acceptable level of information security; four participants believed a slightly acceptable level of information security; two participants claimed a very acceptable level; and one believed that it is completely acceptable.

Employee Security Behavior Patterns

This section attempts to uncover information regarding the employee's security culture behavior, knowledge, and practices in organizations.

 Employee's General Security Behaviors. Participants rated the employee's security behavior to obtain a broad picture. Eight participants believed that the security behavior in their organizations is "OK"; whereas four participants thought it to be "poor or very poor"; one believed it to be "good".

 Identification of the Most Effective Security Practices on Employee's Security

Behavior. Interviewees were given several security practices to rank the level of effectiveness, including: top management, IT department initiatives, technical security countermeasure and personal values and beliefs regarding information security. Eight respondents stated top management involvement as a high priority. Following the IT department's initiatives, there was a priority for technical security countermeasures, and less priority for personal values and beliefs.  Perceptions of an Effective Security Culture. Participants made a range of statements that aimed to form an effective security culture. All thirteen participants stated that conducting security training programs would be effective upon acceptable levels of security culture in organizations. Nine participants affirmed this statement: "Educating employees about the information security to increase their knowledge and make the right decision is important to have an adequate level of security culture." [B1]. The comparative analysis demonstrated that participants from the USA and Saudi Arabia organizations agreed and indicated that developing security policies and support from all levels of leadership are essential to develop an effective security culture. Participant C7 highlighted this: "To create or expect an efficient security culture, there should be an active, continuous engagement and endorsement of the information security by all levels of leadership and managers in the company". The participants from the UK and Saudi Arabia organizations stated that increasing security awareness is necessary for security protection that supports security culture. Four participants from Saudi Arabia organizations suggested that enhancing security ownership in employees would help to promote an acceptable level of security culture.  The Main Contributory Factors Establishing Security Culture. There has been an agreement among participants from the UK and Saudi Arabia organizations that a security training program is one of the most important factors in security culture. Nine participants revealed that developing security training sessions for employees in the organization is one of the highest contributory factors in establishing security culture. Participant A2 assured this: "the security training program is one of the key factors for educating employees to adopt security and influence the employee behavior which will lead to establishing the security culture". Seven participants indicated that developing clear security policies as a second top contributing factor; they affirmed the effectiveness impact of security policy in security culture. Participant E3 commented that: "implementing security policies in the company will be efficient because it helps employees to clarify and get a detailed understanding about the security requirements, and the way to comply with the security rules".

Five participants mentioned that increasing security awareness is a third factor that supports security culture; while three participants signified that a periodical risk analysis is another factor that has to be considered. Two participants revealed that security culture could be effective if there is support from top management, as well as employee's compliance with security policy. One participant stated that understanding the ethical obligations of the organization and employees' job satisfaction are also vital in improving security culture.

A comparative analysis illustrated that four participants from the US and Saudi Arabia organizations believed that top management support and employees' security compliance are contributory factors in developing security culture; participant E3 stated that: "the security culture can be established effectively if all members of the company comply with security policy and regulations". Five participants from the Saudi Arabia organizations considered additional factors, such as security risk analysis, ethical conduct and job satisfaction, to contribute to the security culture. As participant A3 stated that: "Understanding the risk involved with information security and more importantly conducting a periodical risk assessment is a vital key for establishing the security culture environment". Participant B4 affirmed the effectiveness of ethical conduct policies in advancing an organization's security culture: "understanding the ethical codes and obligations is essential to improving security culture". Participant E10 suggested that job satisfaction can motivate employees to comply with organizations' security requirements: "one of the issues that should be considered in a company is the employee's job satisfaction; when the employee has a positive feeling, he/she will be more likely to comply with company security policies".  The Main Barriers to Achieving Improved Security Compliance. The findings indicated that nine participants considered that the lack of awareness and training programs to be the first obstacle; the lack of clear direction in security policies was second; followed by a lack of leadership support and ownership. Two participants signified certain obstacles related to faulty human behavior by misunderstanding the ethical obligations and having no consequences to employees who fail to comply with security procedures. There has been some agreement among seven participants' perceptions of organizations in the UK and Saudi Arabia regarding the main obstacles. Seven interviewees believed that the security training sessions tend to achieve security compliance with the security policy of organizations; participant F1 stated that: "we have simple security policies, but most of the employees lack relevant training and hardly follow any security policies; the training program is vital in educating all employees to comply with security rules and guidelines". The comparative analysis illustrated that four participants from the USA and Saudi Arabia organizations expressed that the absences of clear security policies contribute significantly to the lack of compliance with information security. Three participants revealed that some of the organizations' top management lack the appropriate commitment to promote security policies; participant C2 stated that: "in my company, I have not observed any serious commitment from the top management to enforce enhancing security". One participant from Saudi Arabia considered ethi-cal conduct as an additional factor, while one in the USA focused on the lack of security compliance.

Perceptions for Improving the Security Culture

The Participants had different recommendations that aimed to develop a security culture. There was some agreement among participants' perceptions of organizations in the UK, the USA and Saudi Arabia. Four participants agreed that the top management support in enhancing information security in organizations would improve security culture. Eight participants from the UK and Saudi Arabia organizations considered implementing security awareness as useful in developing security culture. Nine participants in the USA and Saudi Arabia organizations signified that conducting security training sessions and developing clear security policies are the most positive factors that help in improving security culture. Four participants from Saudi Arabia suggested additional factors, such as enhancing the sense of security ownership, developing ethical conduct, conducting a periodical risk assessment, and enhancing the employees' job satisfaction. Meanwhile, one USA participant suggested the importance of improving security compliance. Moreover, three participants from the UK and Saudi Arabia organizations stated the need for a tool or a model that could be used as guidance in implementing the required security culture factors that are targeted at the appropriate tiers of employee behavior, and to inculcate acceptable levels of security culture.

Discussion

Internal threats continue to exist within organizations. It was evident that factors identified in previous research continue to be significant. Phase one in this study provides rich data from thirteen experienced and knowledgeable respondents in seven organizations in the UK, the USA, and Saudi Arabia. The interview also revealed an apparent gap in the efficiency of providing training programs and security policies. Participants stated the requirement for periodic security training, as information security is mentioned once on induction day when the employee starts working for an organization. Participant A1 disclosed that: "education and training are not memorized for long; once the new employee finishes the first week of training, information security is forgotten." There was also concern regarding the limitations of training programs to key managers and IT members. They concluded the need for a structured training program aimed at all the organization's members. Moreover, respondents suggested that they were unclear how their policies were implemented and updated, which is important due to the changing nature of threats. There was concern about the possible clarity of organizational policies to follow. Participant C1 suggested that: "the threats are always changing, the environment is always changing, and information security is always changing. Hence, it is vital for a company to have clear policies that are clearly described; improve the security poli-cies by reviewing it continuously and maintain it up to date". Thus, phase two takes these points into consideration when constructing the questionnaire.

However, a comparative analysis suggested some differences in the data sets, including data collected in the UK, the USA and Saudi Arabia. In particular, there are both similarities and differences among the participants' perceptions based on their country regarding the main factors in terms of cultivating an effective security culture. There has been an agreement among the participants' perceptions regarding the important factors in establishing organizations' security culture. Based on the findings, it appears that security education and training programs, as well as security awareness and the security policy are the most significant factors that contribute toward security culture. Security education and training are considered as the most important factors to influence security culture's effectiveness. It is essential to implement and conduct periodic security training sessions in organizations to develop security culture and to improve employee's awareness, which tends to encourage security compliant behavior [START_REF] Da Veiga | A framework and assessment instrument for information security culture[END_REF]. Additionally, it was indicated that in three organizations the security lessons had been learned following specific incidents.

The interviews indicated the importance of security awareness in promoting security culture. Both security education programs and security policy encourage compliant behavior by increasing employees' security awareness. The findings show that some organizations viewed security awareness as important in establishing a common understanding of security culture. It helps structure how employees think about information security and provides the common language and base of useful knowledge when discussing various security-related topics. Participant G1 indicated that: "it is important to create a mindset within employees; you have to develop active awareness programs, and that will give the employee a high level of awareness". Three participants stressed that sometimes security awareness might be inadequate if members are not aware of possible consequences of security breaches and cannot see the value of their security role in the organization's holistic security work. Demonstrating a high level of security awareness would lead to security-cautious behavior, which tends to encourage security compliance, while also improving security culture [START_REF] Da Veiga | The Influence of Information Security Policies on Information Security Culture: Illustrated through a Case Study[END_REF]. The findings also demonstrate that clear and sufficient security policy could promote security-cautious behavior through security awareness and establish an acceptable level of security culture. Four participants stated that security policy is an important measure, although it might be insufficient if members are not well-informed about the existing security policies.

The interviews revealed other factors that should be considered in establishing organizations' security culture: top management support and security ownership. Respondents from the UK, the USA and Saudi Arabia agreed that gaining top management commitment and support is significant in increasing organizations' security effectiveness. The support and commitment from top management help to form organizational security and predict security culture quality [9]. There were concerns regarding the failure of top management in enhancing security culture through developing appropriate training sessions, as an organization can subsequently face major issues in daily operations. Participant E2 indicated that: "employees in the company understand the importance of security; although we do not yet have a robust security policy because of that the top management is still in the process of establishing the company security activities and structures". Also, respondents from the UK and Saudi Arabia revealed that when employees understand security responsibilities and personal ownership, they comprehend security risks and behave more securely, which tends to increase security awareness and compliance; thus, better security culture [START_REF] Alnatheer | Understanding and Measuring Information Security Culture[END_REF]. Participant B1 asserted: "we do not expect to establish the security culture if our employees do not understand the importance of protecting information and it is their responsibility."

The participants from Saudi Arabia and the USA suggested three factors: security risk analysis and assessment; ethical conduct; and security compliance. The findings demonstrated that security risk assessment and analysis tend to assist organizations and employees to become capable of understanding potential damage to security, which helps to increase awareness and knowledge of security culture [START_REF] Alnatheer | Understanding and Measuring Information Security Culture[END_REF][START_REF] Martins | Assessing Information Security Culture[END_REF]. The interview also suggested that ethical conduct is a vital factor that influences security culture, as it supports employees to integrate ethical behavior, ensuring the security of information and what is accepted by the organization [START_REF] Alnatheer | Understanding and Measuring Information Security Culture[END_REF][START_REF] Martins | Assessing Information Security Culture[END_REF]. The findings illustrated the importance of improving security compliance in security culture creation to increase an organization's security and ensure that employee behavior complies with security policy. Participant C1 stated that: "employees are often unaware of the consequences of security breaches caused by their actions; the company should have a method that ensures employees' behavior continues to be monitored to the compliance program's effectiveness". Furthermore, another vital factor that was suggested by one respondent from Saudi Arabia is employee job satisfaction tends to promote security-cautious behavior, which develops security culture [START_REF] D'arcy | The Multifaceted Nature of Security Culture and Its Influence on End User Behaviour[END_REF].

Interview analysis supports other studies and highlights the significance of factors that have an impact upon employees' security behavior and is important to be considered as part of security culture conceptualization. These factors have a positive influence on each other, and thus, have a positive influence on the security culture. The possible relationships between factors will be tested statistically in order to determine whether the proposed framework is valid. However, personality traits have received little attention from researchers, despite indications that personality traits directly affect individual behavior [START_REF] Mccormac | Individual Differences and Information Security Awareness[END_REF]. This is important for future research and will be an essential factor in the construction of the survey items. This paper provides some insights, although cannot be generalizable, as additional investigations are required. The study will be expanded to develop a statistical framework that would identify the correlations between factors. Knowledge management would be integrated to develop a framework that would help organizations to create the culture efficiently and predict how the security culture could be improved. Consequently, the study adopts a quantitative survey on a range of different sizes of organizations with sample size range from 250 to 300 employees.

Conclusion and Future Work

This study has aimed to build further comprehension of various factors that positively assist with organizations' security culture from employees' perspectives. Existing literature states factors regarding security culture adoption, while all top factors should be considered to create an environment that promotes better security culture.

The study conducted an exploratory interview to present important factors that potentially affect organizational security culture and to identify existing gaps in what employees are aware of. The interviews comprised of thirteen experienced and knowledgeable security specialists from different organizations located in the USA, the UK and Saudi Arabia. These interviews were analyzed to highlight the significant factors in security culture stemming from participants' experiences.

The findings from this study contribute to the existing knowledge by providing factors that are significant in affecting human behavior and vital in security culture. The information gained from interviews provided further knowledge of how different factors were viewed regarding reality settings. The findings also revealed an apparent gap in the organizational policies' implementation and ineffective training programs that lead to a lack of awareness and compliance. The result of this study cannot be generalizable but can be viewed to be indicative. In addition, the findings from this study demonstrate certain limitations, such as the how it cannot test the security culture framework's non-logical validity; the initial influence of factors on security culture; or factors that define security culture. However, the findings from the qualitative phase produced variables that are utilized in the quantitative phase to test the development framework's validity. In order to design the survey, the specific operational elements involved were measurable to the literature review, which were incorporated with the findings from the qualitative phase to identify constructs and related survey items that influence security culture.

  The Main Information Security Practices and Rules used in the Organization. Interviewees provided the main security practices used in organizations, which indicated that all seven participating organizations use general security practices, such as security policies, security training, physical and technical measures.  Security Education and Training Courses in the Organization. Interviewees reported whether their organizations provide security training sessions. Eight participants from organizations located in Saudi Arabia and the USA reported an absence of security training programs. Five participants reported security training courses in their organizations: two organizations located in the UK and one in Saudi Arabia. Each organization informed its members regarding information security matters through induction training.  Different Methods of Security Awareness and Training Sessions. The participating organizations use various channels to distribute related security awareness to employees, such as sending e-mails' notification, conducting seminar and training courses, sending text message notification, using posters or displaying information security in the organization website. Organization A usually uses e-mails, text message and training sessions to make the members aware of any security issues. While participants from organizations B and C reported that e-mails and posters are the main security awareness activities utilized in their organizations. In organization D, the main security awareness methods are e-mails, training courses and displaying information security on the organization website. Organizations E and F adopted the same security awareness activity of e-mails. Finally, organization G uses e-mails, text messages, training courses, and displays information security on the organization website as the main awareness methods.  Alerting Regularly about Risks and Dangers in the Organization. Three inter-

Table 1 .

 1 Organizations demographic profile

	Org	Type	Size	Location	Sector	No. of in-	Interviewees
						terviewees	position
	A	Public	4000	SA	Education	3	IT assistance
							director and IT
							specialists
	B	Public	5000	SA	Education	2	IT specialists
	C	Private	6000	USA	Education	2	IT specialists
	D	Public	5000	UK	Education	1	Enterprise securi-
							ty architect
	E	Private	400	SA	Insurance	3	IT supervisor and
							IT specialists
	F	Semi-	4000	SA	Mining	1	Security manager
		Public
	G	Private	1500	UK	Law	1	Security manager
			Total		13


Matt Bishop
email: mabishop@ucdavis.edu

Melissa Dark
email: dark@purdue.edu

Lynn Futcher
email: lynn.futcher@mandela.ac.za

Johan Van Niekerk
email: johan.vanniekerk@mandela.ac.za

Ida Ngambeki
email: ingambek@purdue.edu

Somdutta Bose
email: sombose@ucdavis.edu

Minghua Zhu
email: mhzhu@ucdavis.edu

Learning Principles and the Secure Programming Clinic

Keywords: Secure programming clinic, learning principles, robust programming

Several academic institutions have run a clinic on robust and secure programming. Each time a clinic was run, it was associated with a specific class. Using pre-and post-class evaluation instruments, it is clear that the effect of the secure programming clinic on students' understanding of secure programming was generally positive. However, in some instances the clinic was underutilized, and in other cases it could not be run at other institutions. The goal of this paper is to examine the structure of the clinic in light of five basic learning principles, and provide information about when a clinic will not improve students' understanding, and when it will. We validate this by examining an instance of the secure programming clinic, and show how the learning principles explain the improvement in student grades, or lack thereof. From this, we draw conclusions about ways to make the clinic more effective, and when it will not be effective.

Introduction

The problem of nonsecure code is widely recognized as a major source of security problems. Indeed, of the vulnerabilities in the U.S. National Vulnerability Database in the last 5 years, over 19,000 are identified as injection and buffer overflow vulnerabilities, exemplars of poor programming practices [START_REF]National vulnerability database[END_REF]. Some, such as Heartbleed, have impacts throughout the Internet [START_REF] Durumeric | The matter of heartbleed[END_REF]. Industries, government, and many other organizations want programmers who can write secure, robust code. The problem is how to teach this material.

Computer-related majors, and many other majors, teach programming. Introductory classes typically teach good programming style, error checking, and other aspects of programming summarized by the word "robust". Being introductory, their focus is not on security as such, but simply on producing good programmers who understand both the strengths and limitations of the language they learn. Future classes expect the students to apply what they have been taught, and rather than focusing on style and robustness, the emphasis is upon correctness. Hence style and robustness is a minimal part of the grade, if they are included at all, giving students the impression that security and quality of programming is less important than required functionality. Students have no reason to apply what they learned in introductory programming. Further, advanced security issues such as programming to avoid race conditions is not taught in introductory classes, so the students lack knowledge of all the aspects of secure coding. The atrophying of this practice, and the lack of knowledge, leads to programmers who do not write robust, secure code.

This problem is exacerbated by the richness of the current curricular guidelines for computer science programs. The ACM Computing Curricula [START_REF]The Joint Task Force on Computing Curricula: Computing curricula 2001 computer science[END_REF] is laden with more material than can be taught in a typical major program, so faculty and program developers choose which parts to emphasize. Given this, creating a required class on secure programming would require something else to be dropped from the curriculum. Further, most faculty do not know all the aspects of secure coding, so many institutions would need to find the instructors for these courses. Thus, few schools have any courses on secure programming, let alone required ones.

An alternative to additional classes is the Secure Programming Clinic [START_REF] Bishop | A clinic for 'secure' programming[END_REF]3]. This clinic works analogously to an English writing clinic, and is discussed in detail in Section 3, below. Like any other aspect of education, its success depends upon a host of factors. The contribution of this paper is to use results from running several instances of this clinic, and basic educational principles, to identify characteristics of an environment in which the Secure Programming Clinic will enable students and instructors to ensure that students program in a robust and secure style, and that instructors who are not experts in this can have the clinic provide feedback about submitted programs that the faculty can take into account when assigning grades. The paper is structured as a Case Study according to guidelines for such studies provided by [START_REF] Creswell | Qualitative Inquiry and Research Design: Choosing Among Five Approaches[END_REF]. The next section provides a brief overview of the case study's layout and how it reflects in the structure of the remainder of this paper.

Methodology

The following structure is often used for the presentation of Case Study research [START_REF] Creswell | Qualitative Inquiry and Research Design: Choosing Among Five Approaches[END_REF]:

-Entry vignette -Introduction -Description of the case and its context -Development of issues -Detail about the selected issues -Lessons Learned and Assertions -Closing vignette

In the context of this paper, the abstract and introduction to the paper respectively serves as the case study's entry vignette and introduction. Section 3 describes the Secure Coding Clinics and serves as a description of the case and its context. The context is further supported by Section 4, which provides a review of relevant literature relating to the teaching of secure programming. The issues of concern in this paper are the identified educational principles, and how these were used in past secure programming clinics. These principles are identified in Section 5 and the relevance/adherence of the three variations of past clinics to the learning principles are mapped in Section 6 which elaborates on the selected issues of concern for this case study. These issues will be used to inform the authors' design intention for future secure programming clinics, which will be presented in the form of lessons learned in Section 6.2.

The closing vignette will take the form of our concluding remarks 3 Description of the Secure Programming Clinics

The basic form of a secure programming clinic is a physical or logical space where clinicians are available to students. As students complete homework assignments and other programs, they bring them to the clinic. The clinicians review the programs, and point out examples of code that create security problems or, more generally, are non-robust. They do not identify all such problems, but instead simply point out examples. They do not examine the program for correctness or whether it meets the requirements of an assignment. The students, on their own, correct the problems. The clinicians do not check that the program meets the requirements of the assignment; indeed, the clinic is not associated with any class, and is available on a drop-in basis. In this way, it resembles a writing clinic. Students can bring papers to the clinic, where clinicians will review the grammar and structure of the paper and offer suggestions on how to improve both. They do not examine the content for accuracy or credibility; they simply look at the form. Several variations of the clinic are possible. The first major set of variations comes from associating the clinic with a particular class. Here are possible approaches, any combination of which may meet the goals:

1. The instructor may apportion some part of the program grade to robustness.

The clinicians would then assist by grading that part of the homework, and providing an appropriate score. The class graders would then grade the program with respect to the assignment's requirements. 2. After #1, the instructor can have the students correct the robustness problems, and then regrade that portion, giving the student some percentage of the points they corrected. 3. The clinicians can act as assistant instructors, helping the students develop threat models for how an attacker might use their program to violate desired security properties. As "security" is defined in terms of requirements, the threat model is critical to knowing the types of security problems that might arise. On the other hand, robustness issues are independent of threats, in the sense that they are common to all threats.

The functions of the clinic can be extended beyond simply reviewing programs. It can also provide information to help the students fix the problems. This typically requires collecting examples of poor programming and how to fix, or (better) avoid, them. It can also provide remote assistance, where the clinicians are not at the institutions. There is a salutary effect for this. If some of the clinicians are volunteers who work in the software industry or for government agencies, their presence and activities will convey the importance that future employers place in high-quality code. This provides incentives for students to learn the material.

The clinic can also be shared among universities. One implementation of the clinic provides a common shared appointment calendar, so students from any of the academic institutions could sign up for appointments even when the local clinicians were not available. The clinicians from the institutions coordinated their times so that one was always available during the day. Were this to be extended internationally, clinicians would probably be available for most of the evening and night (when many students of computer science and related disciplines develop their programs).

The above discussion provides insight into the specific secure programming clinic format of concern to this paper. However, for the sake of comprehensiveness, the next section will briefly highlight other such approaches, and challenges, relating to the teaching of secure programming.

Teaching Secure Programming

Secure programming is about writing secure code. The focus of many programming courses, however, is to write code that works with a lack of focus on writing code securely. A developers unintentional ignorance of known vulnerabilities and insecure coding practices can generate insecure software. Besides the potential financial loss, the successful exploitation of insecure software can impact the confidentiality, integrity and availability (CIA) of critical information. Undetected exploitation can also lead to the embedding of malicious software within an organization, giving the malicious attacker the ability and potential to attack any time [START_REF] Paul | The need for secure software[END_REF]. Secure programming should therefore include the basic principles of robust coding to guard against unexpected inputs and events [START_REF] Ngambeki | Teach the hands, train the mind . . . a secure programming clinic[END_REF].

The challenges of teaching and integrating secure programming into computing curricula have been around for many years, and some of these challenges which are still evident today [START_REF] Nance | Secure coding education: Are we making progress?[END_REF]. These include:

-Lack of faculty buy-in -Competition with other topics for inclusion into the curriculum -Computing curricula already full -Failure of students to grasp other important programming concepts -Lack of secure programming expertise of faculty members Much research has been conducted to address some of these challenges. A recent study [START_REF] Tabassum | Comparing educational approaches to secure programming: Tool vs. ta[END_REF] investigates a Java proof-of-concept plug-in for Eclipse, ES-IDE (Educational Security in the IDE), that provides vulnerability warnings and secure programming education in the IDE while students write code. It works by scanning a selected project for code patterns that match predefined heuristic rules of security vulnerabilities. In this way, secure programming knowledge can be introduced early and reinforced throughout a students education. Generally, ESIDE was found to increase students awareness and knowledge of secure programming. However, almost no students actually modified their code to mitigate the detected vulnerabilities as they were most concerned with completing functionality and did not want to impact that functionality with additional security-oriented code. In addition, carefully timing the introduction of concepts and skills as well as incentivising such learning is important [START_REF] Tabassum | Comparing educational approaches to secure programming: Tool vs. ta[END_REF].

ESIDE was compared to the Secure Programming Clinic by running each approach with two separate groups of students, one group assigned to ESIDE and the other to the clinic [START_REF] Tabassum | Comparing educational approaches to secure programming: Tool vs. ta[END_REF]. Each group of students were asked to report on how likely they would use the recommended changes in their code during the session. The likelihood results for the Secure Programming Clinic were significantly better than for ESIDE. However, the clearest difference between the clinic and ESIDE were the number of specific vulnerabilities covered. Where ESIDE marked on average 42 lines of code per participant, the technical assistants running the clinic pointed out approximately two specific lines of code per participant.

One response to the need to teach students to program more securely was to introduce a serious game for teaching secure coding practices and principles to novice programmers [START_REF] Adamo-Villaani | Using a serious game approach to teach secure coding in introductory programming: Development and initial findings[END_REF]. Initial findings showed the game to be usable and engaging, with the majority of students being able to make clear correlations between the game levels and corresponding security concepts. Similarly, constructing secure coding duels [START_REF] Xie | Gamifying software security education and training via secure coding duels in code hunt[END_REF] in Code Hunt, a high-impact serious gaming platform released by Microsoft Research, was proposed to instill gaming aspects into the education and training of secure coding. Secure coding duels proposed in this work are coding duels that are carefully designed to train players secure coding skills, such as sufficient input validation and access control. Using serious games for teaching secure coding could alleviate some of the challenges faced by faculty members in this regard.

Furthermore, scorecards and checklists provide a consistent means of evaluation and assessment [START_REF] Taylor | Using security checklists and scorecards in cs curriculum[END_REF] . They describe the use of security checklists and scorecards which provide a quantifiable list of security criteria to aid in writing secure code and further reinforce security principles. Checklists distributed to students included:

-Sample code of errors to look for; -Examples of correct ways of writing code; and -Security mantras including a list of principles that form the basis for the checklist, for example: All Input is Evil! Regardless of the approach used to teach secure programming, such approaches should take into account recognized learning principles, as discussed in Section 5, to ensure that learning takes place.

Learning Principles

Systematic studies of human behaviour, including studies of how people learn, is a relatively new field of scientific enquiry [START_REF] Ormrod | Human Learning[END_REF]. However, despite the youth of this field, many studies have already been dedicated to investigating how learning takes place. During such studies, researchers strive to identify recurring patterns in the data and to make generalizations based on these patterns. Such generalizations lead to the formulation of learning principles and learning theories. Learning principles identify the factors that influences learning. For example, the principle that a behaviour which is rewarded in some way is more likely to re-occur in future than one which is not followed by a reward. A learning theory on the other hand aims to provide an explanation of the underlying mechanisms that are involved in learning. Thus, whilst a learning principle presents what factors are important, a learning theory would explain why those factors are important [START_REF] Ormrod | Human Learning[END_REF].

Learning principles do not change much over time, however, learning theories have continually changed as understanding of human behaviour evolved [START_REF] Ormrod | Human Learning[END_REF]. Due to the fact that learning principles are less changeable, and thus more 'future proof' than learning theories, this research will seek to identify learning principles that could be useful in the secure coding clinics, but will avoid subscribing to any specific learning theory.

Educational literature provides many such learning principles. These principles have been identified, and their impact verified, in a variety of ways. One such approach is the field of brain compatible education. This educational approach stems from a combination of neuroscience and educational psychology and was first made possible by advances in brain imaging during the 1990s [START_REF] Mcgeehan | Brian-compatible learning[END_REF].

Brain-compatible, or brain-based, learning is not a formalised education approach or 'recipe for teachers', instead it provides a "set of principles and a base of knowledge and skills upon which we can make better decisions about the learning process" [9, p xiii]. Brain research has shown that humans literally grow new dendrites and neural connections every time they learn something. Knowing which educational activities are the most effective in stimulating such growth allows educational practitioners to create material that leverages the way the brain naturally learns [START_REF] Lombardi | Beyond learning styles: Brain-based research and english language learners[END_REF]. For the purpose of this research, it is not necessary to understand how these natural learning processes work. One only needs to understand that these principles were verified as being effective in promoting real learning.

No single complete list of such principles exists. However, many principles are presented and discussed in the literature [START_REF] Caine | Making Connections: Teaching and the Human Brain. Association for Supervision and Curriculum Development[END_REF][START_REF] Craig | Brain-compatible learning: Principles and applications in athletic training[END_REF][START_REF] Fogarty | Brain-Compatible Classrooms[END_REF][START_REF] Jensen | Brain-Based Learning: The New Paradigm of Teaching[END_REF][START_REF] Materna | Jump-Start the Adult Learner: How to Engage and Motivate Adults Using Brain-Compatible Strategies[END_REF][START_REF] Van Niekerk | The effectiveness of brain-compatibkew blended learning material in the teaching of programming logic[END_REF][START_REF] Smilkstein | We're Born to Learn: Using the Brain's Natural Learning Process to Create Today's Curriculum[END_REF][START_REF] Sousa | How the Brain Learns[END_REF]. The list presented in Table 1 contains a subset of principles from those used in literature. The principles included in Table 1 were restricted to those the authors specifically deemed most relevant to the context of the Secure Programming Clinic. Relevant principles from literature were reworded and consolidated in cases where there was significant overlap in meaning between those used in the literature and the context for use in this study. Table 1 thus presents the authors' adaption of these principles. The following discussion briefly elaborates on each of the listed principles:

-LP1 -According to [START_REF] Fogarty | Brain-Compatible Classrooms[END_REF][START_REF] Jensen | Brain-Based Learning: The New Paradigm of Teaching[END_REF][START_REF] Materna | Jump-Start the Adult Learner: How to Engage and Motivate Adults Using Brain-Compatible Strategies[END_REF] there is no long term retention without rehearsal.

The brain would prune new neural growth if it is not reinforced by being used. It is vital to repeat lessons taught more than once, otherwise students would be likely to forget these lessons. One should also allow enough time for students to assimilate any new concepts. Several studies [START_REF] Caine | Making Connections: Teaching and the Human Brain. Association for Supervision and Curriculum Development[END_REF][START_REF] Fogarty | Brain-Compatible Classrooms[END_REF][START_REF] Materna | Jump-Start the Adult Learner: How to Engage and Motivate Adults Using Brain-Compatible Strategies[END_REF] explains that the brain will reconsolidate new neural growth for several weeks after learning using both conscious and unconscious (sleep) processes to decide how to incorporate knowledge into existing neural structures. -LP2 -If the new knowledge is too advanced for the target audience, learning might be inhibited because the learners feel threatened instead of challenged by the content [START_REF] Caine | Making Connections: Teaching and the Human Brain. Association for Supervision and Curriculum Development[END_REF][START_REF] Craig | Brain-compatible learning: Principles and applications in athletic training[END_REF][START_REF] Fogarty | Brain-Compatible Classrooms[END_REF][START_REF] Jensen | Brain-Based Learning: The New Paradigm of Teaching[END_REF]. Furthermore, new knowledge can only be assimilated if it builds upon prior knowledge, since novel patterns can only form as extensions of existing patterns [START_REF] Craig | Brain-compatible learning: Principles and applications in athletic training[END_REF][START_REF] Materna | Jump-Start the Adult Learner: How to Engage and Motivate Adults Using Brain-Compatible Strategies[END_REF][START_REF] Smilkstein | We're Born to Learn: Using the Brain's Natural Learning Process to Create Today's Curriculum[END_REF][START_REF] Sousa | How the Brain Learns[END_REF]. -LP3 -The process of learning consists of the brain recognizing patterns [START_REF] Caine | Making Connections: Teaching and the Human Brain. Association for Supervision and Curriculum Development[END_REF][START_REF] Craig | Brain-compatible learning: Principles and applications in athletic training[END_REF][START_REF] Fogarty | Brain-Compatible Classrooms[END_REF][START_REF] Jensen | Brain-Based Learning: The New Paradigm of Teaching[END_REF]. For these patterns to form the learners need to recognize and connect patterns by themselves [START_REF] Craig | Brain-compatible learning: Principles and applications in athletic training[END_REF][START_REF] Jensen | Brain-Based Learning: The New Paradigm of Teaching[END_REF][START_REF] Materna | Jump-Start the Adult Learner: How to Engage and Motivate Adults Using Brain-Compatible Strategies[END_REF][START_REF] Smilkstein | We're Born to Learn: Using the Brain's Natural Learning Process to Create Today's Curriculum[END_REF]. This process works best if the learners experience these patterns in contexts that are relevant to themselves [START_REF] Craig | Brain-compatible learning: Principles and applications in athletic training[END_REF][START_REF] Jensen | Brain-Based Learning: The New Paradigm of Teaching[END_REF] and their real-life experiences [START_REF] Materna | Jump-Start the Adult Learner: How to Engage and Motivate Adults Using Brain-Compatible Strategies[END_REF]. -LP4 -Humans naturally learn in social settings and through interaction with others [START_REF] Caine | Making Connections: Teaching and the Human Brain. Association for Supervision and Curriculum Development[END_REF][START_REF] Fogarty | Brain-Compatible Classrooms[END_REF]. Collaboration with others enhances learning [START_REF] Materna | Jump-Start the Adult Learner: How to Engage and Motivate Adults Using Brain-Compatible Strategies[END_REF]. -LP5 -Rehearsal will make learning permanent, however, this does not guarantee the rehearsed learning is in fact correct. Practice should be accompanied by feedback that is constant, consistent, and specific to ensure that practice that is permanent is also correct [START_REF] Fogarty | Brain-Compatible Classrooms[END_REF][START_REF] Van Niekerk | The effectiveness of brain-compatibkew blended learning material in the teaching of programming logic[END_REF]. The effect of feedback is also amplified if it is immediate [START_REF] Craig | Brain-compatible learning: Principles and applications in athletic training[END_REF][START_REF] Jensen | Brain-Based Learning: The New Paradigm of Teaching[END_REF].

Mapping of Clinics to Selected Learning Principles

We begin by examining the instances of the secure programming clinic that have been run, and how they reflect the learning principles. We then discuss how the clinic might be improved by mapping the principles into various forms of the clinic.

Experimental Secure Programming Clinics

The University of California at Davis, the California Polytechnic State University at San Luis Obispo, the California State University at Sacramento, and Purdue University Northwest have run the Secure Programming Clinic over the past 4 years. These instances of the clinic were tied to particular classes such as networking, operating systems, computer security, and introductory programming classes.The methodology was the same in all instances. At the beginning of the term, students were asked to fill out an evaluation form that tested how much they knew about secure programming. They received class credit for beginning the questionnaire, and could indicate if they declined to proceed after giving their name and student ID (so they could get credit). At the end of the class, they filled out a similar questionnaire. The results of the two questionnaires were compared to see how their knowledge of, and ability to practice, secure programming changed. During the class, for each programming assignment, students could go to the clinic before submitting the assignment, and modify their programs based on the clinician's feedback. When assignments were submitted, they were given to the clinicians to check for the robustness and quality of the programming; graders assigned to the class graded the submitted programs for correctness. Then the instructor combined the results to give the program a grade. When the assignments were returned, students were told they could correct the robustness and security problems, and get back a large percentage of the points deducted for this (usually 75% or 80% of the points). They had a week to do this and resubmit the assignment. The clinicians would then review the changes, compare them with the original programs, and inform the instructor about the changes. The instructor would change the grade accordingly.

Institutional Review Boards (IRBs) at all institutions examined the experimental protocols to ensure they complied with federal and state law, and with the institutions' own rules about gathering and retaining student data. At the University of California at Davis, the principle investigator of the project was also the instructor, which the UC Davis IRB saw as a conflict of interest. To keep students anonymous to the instructor, student information (name and student identification number) was coded with a 4-digit number, and all work relating to that student's interaction with the clinic was recorded using that 4-digit number. During the term, the clinicians kept track of each student's 4-digit num-ber, and any analyses had the student names and other identifying information redacted and replaced by the 4-digit number. When assignments were graded, the clinicians were given everyone's grade. This way, the instructor had no access to a particular student's pre-and post-questionnaires, nor to any information recorded about grade improvement.

Data on students' clinic usage and secure programming scores were collected in 2017 in one class (see Table 2). Of 42 students enrolled, 36 visited the clinic and 6 did not. Of the 36 students who visited the clinic, 14 students visited the clinic before the assignment was due, which were called "proactive" clinic users. Five visited the clinic before and after the assignment was due; they were called "consistent" users. Seventeen only visited the clinic after the initial secure programming assignment was submitted and graded; they were called "reactive" users.

The average score for 12 of these 14 students on the secure programming assignment was 77% and the standard deviation was 4%. These students did not submit their assignment for a regrade. Two of the 12 proactive students (students who visited the clinic before the assignment was due) submitted their original assignment, received a grade, and then made changes to their assignment and submitted their assignments for regrade. These two students differ notably. The first student received an initial score of 9% and did not gain any additional points when he/she submitted the assignment for regrade. The second student scored 78% on the initial assignment and 91% on the assignment that was submitted for regrade. When the learning results of the Secure Programming Clinic are interpreted in the context of the 5 learning principles, there are several interesting findings. In the context of individual student learning, the principles are intersectional. For example, while lessons must be repeated at suitable intervals, what makes an interval "suitable" is partly contingent upon the pre-existing knowledge of the target audience, which is comprised of every student. Thus, pre-existing knowledge levels and learning trajectories vary. Another intersection is between LP1 and LP5. Immediate feedback (LP5) signals to learners the need for additional practice (LP1). When the learning setting provides for additional practice (LP1), at an appropriate level of difficulty (LP2) and supported through interaction with others (LP4), learning is supported.

Interpreted in the context of the principles, the 12 proactive clinic users who did not submit their assignments for regrade, visited the clinic. They then used the specific and timely feedback they were given to practice more secure programming. They submitted the assignment for feedback, and received timely positive feedback via their scores. This signals to the student that they are on track. In the case of the proactive student who received a high score and chose to make modifications to his/her assignment and submit for regrade, the clinic afforded the use of specific and timely feedback (LP5) in support of rehearsal and mastery (LP1 and LP2). In the case of the proactive student who received an initial score of 9% and did NOT improve, there are two considerations. If the student does not care about learning, then most instructional attempts at intervention will fail. However, if the student wants to learn, but is failing, then it would be important to incentivize the student to visit the clinic again. Provided there is desire to learn, the deficiency is likely due to prior knowledge (LP2) and/or lack of active, personally relevant experiences that afford pattern formation (LP3). In this case, the clinic should provide suitable diagnostics.

The 5 students who visited the clinic before the assignment was due and then after receiving an initial grade and before the reworked assignment was due are also interesting. Three of the five received a score of 23% on the robustness of their code for the assignment. It would seem as if the clinic was marginally useful for these three students before they did the assignment, especially if you compare them to the 12 proactive clinic users who scored 77% on the first assignment submission after visiting the clinic. Upon receiving their grade and then revisiting the clinic, these three students raised their grade to 77%. For these students, the value of the clinic demonstrates all of the learning principles. The clinic provided a venue for these students to repeat the exercises. The students seem to have acquired useful knowledge through this rehearsal, even though the new knowledge had to be built on wrong performance. Unlike their 12 peers in the proactive group who were able to attend the clinic, get useful information and submit the homework and earn a high grade, these students attended the clinic, got the proactive information, applied it incorrectly (LP2), received a poor grade (LP5), reattended the clinic (LP2, 3 and 4), learned about the same secure programming practices in the context of marginal performance (LP1, 2), and then succeeded. This is very encouraging.

The 17 students who were reactive users of the clinic are also interesting. Upon receiving an initial grade (mean score for this group was 21%, standard deviation 17%), learners had knowledge of what they did not know. The specific and timely feedback (LP5) acted as a spur to motivate students to use the clinic. The feedback was diagnostic in that it showed students exactly where their thinking was deficient, thereby allowing practice and rehearsal in correcting their insecure programming practices (LP1). In order for these students to continue expanding their secure programming practices, the clinic should support multiple classes, so that lessons are repeated (LP1), in a manner that subsequent knowledge builds on prior knowledge (LP2), culminating in the formation of patterns (LP3). When this is the case, learners will have robust knowledge of robust programming practices.

Discussion and Lessons Learned

The preceding section shows how important the principles are to the clinic's success. Here we discuss more general lessons learned.

Principle LP2 suggests that students build on pre-existing knowledge to learn, and LP3 says students learn by recognizing patterns. In the instance of the secure programming clinic for an introductory programming class in C, few students availed themselves of the clinic even though they were offered the chance to resubmit work, as described above. The problem was that the students were struggling to learn the language and techniques. Although all were supposed to have programmed before, none had used C, C++, or Java, and so concepts like pointers were new to them. They also had not mastered recursion and other basics. Hence they had no pre-existing knowledge of many features of C, and so they were unable to build on that pre-existing knowledge. Further, concepts of security were too difficult, so the clinic focused on robustness (avoiding buffer overflows, checking input, and so forth). Even these basics required the students to recognize patterns in programming, and they simply did not have the background for this.

The clinics discussed above were tied to specific classes. The students who went consistently (that is, before the assignment was due and again after the grading) showed the greatest improvement. This accords with learning principle LP1, that learning must be repeated at suitable intervals. Were the clinic not tied to specific classes, and were faculty to have their students use the clinic, this finding suggests that students would improve their robust and secure programming skills.

The structure of the clinic had the clinicians interacting directly with the students, sometimes in person privately, sometimes in a group, and sometimes using remote technology such as Skype. They gave immediate feedback by identifying examples of robustness and security problems in the students' programs, and discussing them, and approaches to possible solutions, with the students. The students then had not only to figure out the best solution and implement it, but they also had to analyze their program looking for other instances of that type of problem, because the clinicians did not point out all the problems. The goal was to give the students the skills to find the problems themselves, and learn how to avoid programming them in the future. This is an application of LP5, immediate feedback amplifying learning. Its success is shown by the marked increase in scores for the consistent and reactive students' programs.

The clinic provided students interaction with others (the clinicians). Although no formal statistics were gathered, the clinicians thought the group sessions were better because the students would explain problems to one another and therefore learn from their peers as well as from the clinician. This is an application of LP4, that collaboration and interaction with others enhances learning.

Several lessons from this mapping are apparent:

-The secure programming clinic will not work well with beginners or those who have little programming experience. -The secure programming clinic will benefit students the most if it is available throughout the students' educational career, and they are required to use it in all classes that require programming (except introductory programming classes). -The secure programming clinic must be tuned to the experience and background of the students, individually when possible, or taking into account the educational environment of the students when not. -The secure programming clinic should hold group meetings as well as individual consulting sessions.

Conclusion

The secure programming clinic was developed to assist students in learning to write robust, secure programs, and to reinforce this throughout their schooling.

A key question is to determine the conditions under which it will work well, and under which it will not work well. The research presented above, and its interpretation in light of the educational principles, provide general answers.

An interesting question would be to determine how to tailor the secure programming clinic to take advantage of specific educational environments. As an example, the clinic would probably be instantiated differently in South Africa than in the U.S. due to the differences in the structure of the academic programs. How to take those differences into account, to make the clinic maximally effective, is ripe for study. Similarly, could the clinic be adapted to a short, weeklong course designed to teach programming intensively, and if so, how would the clinic operate to provide support after the course? Any analysis or development of these clinics must examine how to apply the principles in light of the environmental constraints.

Table 1 .

 1 Learning Principles

	Principle# Description
	LP1	Lessons must be repeated at suitable intervals
	LP2	Lessons must build upon the pre-existing knowledge of the target audience
		and must be of an appropriate level of difficulty
	LP3	Learning happens through the recognition of patterns. To recognize new
		patterns, learning must be actively, personally, and specifically experienced
		in a context the learner can relate to
	LP4	Learning is enhanced through collaboration and interaction with others
	LP5	Immediate feedback amplifies learning

Table 2 .

 2 Statistics from the Secure Programming Clinic.

			Initial clinic Grade	Clinic Regrade
		n	visit	mean stdev revisit mean stdev
	Proactive; no regrade 12	yes	77% 22% no N/A N/A
	Proactive; regrade	2	yes	44% 28% no 50% 5 41%
	Consistent; no regrade 2	yes	34% 22% yes N/A N/A
	Consistent; regrade	3	yes	23% 10% yes 77% 19%
	Reactive; no regrade 0	no	N/A N/A no N/A N/A
	Reactive; regrade	17	no	21% 17% yes 70% 10%
	Never; no regrade	0	no	N/A N/A no N/A N/A
	Never; regrade	6	no	23% 98% no 48% 16%

These data are anomalous. There were only two students in the group. One student scored 91% on the regrade and the other student remained at a score of 9%.

Acknowledgements. This material is based upon work supported by the U.S. National Science Foundation under Grant Number DGE-1303211 to the University of California at Davis and Grant Number DGE-1303048 to Purdue University. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the U.S. National Science Foundation, the University of California at Davis, and Purdue University.