Pablo Hernández-Bolaños

Jorge Rodríguez-Díaz

Business Process Management (BPM) and e-Government: An Experience at University of Las Palmas de Gran Canaria (ULPGC)

Keywords: e-Government, business process management (BPM), success critical factors

The development of business process management (BPM) is a key factor to launch e-Government in a public organization. This development requires first identifying the components that make up the business process management and, second, implementing them. It is just this implementation which has been presented as one of the most complex milestones in the development of BPM. In this paper the authors show how to implement the components of BPM successfully based on construction that deploys an organizational initiative that addresses directly a problem expressing the organization and indirectly the implementation of BPM. This methodology is justified by a case study carried out in the ULPGC.

Introduction: Need for BMP to Implement the ULPGC e-Government Project

The University of Las Palmas de Gran Canaria (ULPGC) promoted in 2010 the e-Government project, whose purpose was the implementation and promotion of e-Government within this public organization.

The different actions carried out within this project represent a significant evidence of the deployment of e-Government in the ULPGC, as this university has an electronic office system for the identification and authentication of both, citizens and administrative bodies, in the exercise of its powers [1], electronic register, electronic notifications, ability to recognize the validity of an electronic document, electronic management of procedures, availability of electronic information for citizens about the status of procedures and possibility of cooperation between administrations for the promotion of e-Government [2]. All this provides evidence that the ULPGC is capable of supporting an electronic service delivery.

However, the objective of this work has not been directly the implementation of e-Government, but the implementation of the Business Process Management (BPM) as an essential element to promote e-Government in public organizations [START_REF] Janssen | E-government business models for public service networks[END_REF][START_REF] Corradini | Inter-organizational business process verification in public administration[END_REF]. This article highlights a methodology to implement the BMP in these organizations consisting of starting an initiative to solve a concrete organizational problem of the institution itself, while the BPM components are developed. This paper also describes the experience carried out in the ULPGC between 2003 and 2007 to evaluate the methodology. This experience was called Management Memorandum Model (MMM) -see Fig. 1. BPM is a management system based on the use and development of organizational capabilities to manage processes and improve results thereof according to their strategies [START_REF] Benedict | A Guide to the Business Process Management Common Body of Knowledge[END_REF]. This management system has many benefits as shown in Table 1, but stresses the advantage of implementation for the deployment of electronic services and process automation. However, in most cases, BPM projects carried out in organizations have had a high failure rate [START_REF] Trkman | The critical success factors of business process management[END_REF]; this task may be even more complex in the public sector due to the functional culture and departmental thought, an aspect contrary to a process approach [START_REF] Da Silva | Process management tasks and barriers: functional to process approach[END_REF]. To ensure the implementation of BPM in an organization, it is necessary to ensure the proper development of a number of critical success factors (CSF), that is, those aspects that should be strengthened to successfully complete the project [START_REF] Trkman | The critical success factors of business process management[END_REF], [START_REF] Ariyachandra | Critical success factors in business performance management -Striving for success[END_REF][START_REF] Bai | A grey-based DEMATEL model for evaluating business process management critical success factors[END_REF][START_REF] Bandara | Means of achieving business process management success factors[END_REF][START_REF] Karim | A methodology for effective implementation of lean strategies and its performance evaluation in manufacturing organizations[END_REF]. Rosemann and vom Brocke [START_REF] Rosemann | Six Core Elements of Business Process Management[END_REF] state six CSF for BPM: strategic alignment, governance, methods, ICT, people and culture. The work of these authors disaggregates these factors in other components called capabilities, which are referred to as sub-factors (s-CSF) in this work and are shown in Table 2. Several authors, such as Fettke et al. [START_REF] Fettke | Business Process Maturity in Public Administrations[END_REF]; Niehaves et al. [START_REF] Niehaves | Business process management capabilities in local governments: Amulti-method study[END_REF]; Santos et al. [START_REF] Santos | Strategies for Managing Critical Success Factors of BPM Initiatives in Brazilian Public Organizations: A Qualitative Empirical Study[END_REF] and Lönn et al. [START_REF] Lönn | Process Management Challenges in Swedish Public Sector: a Bottom Up Initiative[END_REF], support these sub-factors as valid for the public sector. Nevertheless, the deployment of CSF and s-CSF in an organization is not an automatic task, but on the contrary, it is very complex since it requires an adaptation of these components to the characteristics of the organization [START_REF] Trkman | The critical success factors of business process management[END_REF]. Either way, there are no described models that enable this adaptation, and even less in the public sector [START_REF] Škrinjar | Increasing process orientation with business process management: Critical practices[END_REF]. This being precisely what the study identified as the research problem.

The Research Method

The research methodology applied in this work has been deducted qualitatively [START_REF] Hyde | Recognising deductive processes in qualitative research[END_REF] since, from the literature review, it was proposed a construction or model that was evaluated putting it into practice through the case of study [START_REF] Yin | Case Study Research, Design and Methods[END_REF]. Considering Fettke et al. [START_REF] Fettke | Business Process Maturity in Public Administrations[END_REF], indicating the need for indirect ways to implement the critical success factors in BPM, and Santos et al. [START_REF] Santos | Strategies for Managing Critical Success Factors of BPM Initiatives in Brazilian Public Organizations: A Qualitative Empirical Study[END_REF], pointing to the desirability of developing strategies with the same objective namely indirect methods, it was decided to design a model to launch a project or initiative that, including the implementation of the CSF and s-CSF as indirect finality or purpose, had the utility to solve a particular organizational problem as direct finality. The model was designed considering the CSF and s-CSF [START_REF] Rosemann | Six Core Elements of Business Process Management[END_REF]. Likewise, to ensure effective implementation of the s-CSF in the organization, BPM maturity models were taken into account, such that each s-CSF improves its sophistication cyclically [START_REF] Trkman | The critical success factors of business process management[END_REF]. This was achieved by requiring the instrument to solve the direct purpose incorporating the criterion of continuous improvement cycle, closely associated with planning systems. Finally, a set of criteria (design criteria) were defined, matching the ten principles of BPM established by vom Brocke [START_REF] Vom Brocke | Ten principles of good business process management[END_REF], which would ensure the successful implementation of BPM in an organization. Hence, to develop the CSF and s-CSF in a public organization, and therefore a successful implementation of BPM, this research proposes as a solution for the research problem the development of an initiative in the organization that solves a major problem and that relies on ten criteria or principles of vom Brocke.

RESULTS

Organizational problem solution

Development CSF and s-CSF

Therefore, the organizational initiative is twofold: direct and indirect. Direct refers to an organizational problem solving and the second to implement BPM. The method used for the first purpose will depend on the problem to be solved; to ensure that the initiative answers the two purposes, the model states that the proposed tool to solve the organizational problem should be supported by BPM; at the same time, to achieve this support, the initiative must promote the development of the CSF. On the other hand, the method used for the second one will be the application of design criteria based on the ten principles of BPM. Table 3 shows the characteristics of the dual purpose of the organizational initiative that arises as a solution to successfully implement BPM.

To evaluate the proposed model, this one should be applied to a specific public organization launching an organizational initiative. The level of implementation of the BPM will show the level of success of the method, so the expected result would be the development of different CSF and s-CSF, shown in Table 2. The results of the evaluation were qualitative conclusions about the validity of the initiative [START_REF] Keen | Relevance and rigor in information systems research: improving quality, confidence, cohesion and impact[END_REF]. The data collection was done by direct observation techniques [START_REF] Bisquerra | Metodología de la investigación educativa[END_REF][START_REF] Flick | Introducción a la investigación cualitativa[END_REF], and based on experience because the researchers themselves were part of the senior management team that drove the innovation initiative.

The Management Memorandum Model: an Indirect Model to Implement BPM

To evaluate the designed model, the case of study was applied on an initiative called Management Memorandum Model (MMM), developed between 2003 and 2007 by the ULPGC, an institution with an annual budget of more than 130 million Euros.

Direct Purpose

The main purpose of MMM was to continuously improve their academic and administrative services. This was done to make its own staff identify, implement and evaluate performances to improve those services. The MMM model consisted mainly in implementing the following stages in the recurring cycle of one year: a) Identification of potential needs for improvement. Through different instruments for data collection, it pointed to service units with possible needs for improvement, which should be validated by the heads of the units with their teams and endorsed with the senior management of the University in a working group session during two days, that was engaged exclusively to this task. b) Solutions design. Multidisciplinary teams, made up of the unit heads, designed solutions to identify needs improvement. This work was done during the working group session and solutions measurements constituted the annual improvement plan. c) Implementation and monitoring. The unit heads coordinated the implementation of the measurements. They were put into operation in a collaborative working platform based on Moodle allowing tracking of the implementation of the plan and an adequate transfer of knowledge. d) End of cycle evaluation. At the end of the planning period a compliance report was produced from the received reporters. This report served as a working tool in the study days for the planning of the next period or cycle. The implementation of the MMM in ULPGC has provided the following information:

• Model execution duration. Since the improvement cycle referred to the model should be repeated several times to assess its validity, the project duration was three years. • Improvement measures. During the three cycles, the MMM allowed to carry out 150 measures.

• The units involved. All administrative units of the organization were involved in the project, mainly by the exercise of leadership of senior management. In total, there were twenty units participating.

• Enhanced Services. The academic and administrative services that were subject to the improvements were about fifty.

• Implemented procedures. There were three hundred administrative procedures identified and homogenized.

• People involved. It should be noted that there are two types of people involved: the personnel who manage enhanced services and persons belonging to receptor group improvements. They were 400 working directly, 2,600 staff and 25,000 teachers and students as users of services.

To develop s-CSF, MMM required several conditions that made possible s-CSF during its four life cycles (therefore these conditions were needed to undertake the improvement of academic and administrative services). Thus, the MMM enabled the development of the s-CSF, and therefore the implementation of the BPM (see Table 4).

Indirect Purpose

The MMM incorporated the design criteria set out in the strategy as follows:

• Criterion 1. Critical success sub-factors (see Table 4) should be applied only when they could be assumed by the characteristics of the organization.

Table 4. Equivalence between conditions and s-CSF (Source: Authors' own).

CONDITIONS s-CSF

The services which should be improved were formulated in terms of processes, so that the annual plan referred to the MMM was a process improvement plan.

A.1. Plan process improvement.

The services were improved in concordance to the business strategy set out in the strategic plans of the organization.

A.2. Alignment between business strategy and process.

The criteria established to standardize processes and services were formulated according to these processes.

A.3. Business process architecture.

The persons responsible for each service were designated, and therefore, for each process.

B.2. Roles and responsibilities of processes.

Criteria homogenization process served to model the services.

C.1. Design and process modelling.

The services were executed according to the life cycle processes.

C.2. Implementation and execution of processes.

Own tool was used to model the processes. D.1. ICT for design and process modelling.

Technologic workflow was launched to mechanize electronically the processes. D.2. ICT enabler for the implementation and execution of processes.

Tool based on Moodle was pushed for collaborative work.

E.3. Teaching and learning processes.

• Criterion 2. Sub-factors should be incorporated gradually, as the organization could take over. To ensure the gradual incorporation of sub-factors, we worked at two levels. On the one hand, every sub-factor was launched the first year, but a deal for each cycle repeating pattern was scheduled. On the other hand, in each cycle, sub-factors that have already been built were increasing their level of development, as the organization was becoming more mature [START_REF] Gottschalk | Maturity levels for interoperability in digital government[END_REF][START_REF] Kazanjian | An empirical test of a stage of growth progression model[END_REF][START_REF] Van De Ven | Explaining Developments change in organizations[END_REF], applying to this the maturity model of Melenovsky and Gartner [START_REF] Melenovsky | BPM Maturity Model Identifies Six Phases for Successful BPM Adoption[END_REF]. • Criterion 3. Organization personnel led the management model and therefore the sub-factors. Own staff, the management team ahead, not only managed the model, designing, planning and implementing the improvement actions, but established the coordination protocols that made the model work.

• Criterion 4. The scope of the model should cover every aspect of academic and administrative organization. The model was extended to all academic and administrative services and all the units in the organization.

• Criterion 5. The model should be integrated into the regular development of the organization and led by the management team. The unit heads for the operational management of academic and administrative services would manage improvement measurements, which are planned and executed following the budget cycle, marking the pace of activity in the public organization. Thus, the MMM was part of the ordinary activities of the organization • Criterion 6. Every stakeholder in the organization should be involved in developing the model or perceive its effects. Over 50% of the staff participated in improving measurements arising from the application of the model and 100% appreciated the effect of these improvements in the performance of their activities on development.

• Criterion 7. The model should be formed by elements of the organization for the purpose it was perceived as friendly by all staff. The terminology used for the business process management was the same as the activity of the organization; in this way, nobody perceives the model strangely but as a different way to carry out their responsibilities.

• Criterion 8. The model must answer an important purpose and resolve any organizational problem. The model was launched to solve two problems: a) the lack of adaptation of service units to the needs of receptor groups thereof and b) the existence of multiple plans poorly aligned and with little effect on the improvement of services.

• Criterion 9. The model should be simple. The model consisted of a methodology incorporating developments in the management of the units, but did not involve an increase in the complexity of its business because these developments were improved in each cycle model.

• Criterion 10. The information and communications technology (ICT) plays an important role in the model. The improvements should have a technology component; in this way, everybody perceived improvements more palpable. In addition to the successful implementation of BPM through the development of s-CSF, the MMM was really useful in order to incorporate the culture of continuous improvement in the teams that manage the academic and administrative services and the technological vision of such services, thus promoting the other two elements necessary for a successful implementation of e-Government [START_REF] Weerakkody | E-Government: The Need for Effective Process Management in the Public Sector[END_REF].

Results

The direct evidences that corroborated the successful implementation of BPM and matched the critical success sub-factors were:

• The catalog of processes • The improvement plans • The planning system

• The tool for knowledge management • The Information Systems Plan Anyway, the main evidence of the effective implementation of BPM is the proper e-Government project, as it would not work without a deployed BPM.

Conclusion

The BPM is a needed element for the development of a variety of management systems, including the e-Government. Although many authors have highlighted the difficulties in implementing the BPM, especially in the public sector, this work has exposed that BMP solutions may be developed if it is used as a method consisting of launching an initiative that is to solve organizational problems through organizational tools that are based on the components of BPM, as the sub-factors (see Table 2). This method has been proven by its application in three public organizations: a university, case presented in this paper with the initiative called Management Memorandum Model, and two more departments of the Canary Islands Government that have been studied in a doctoral thesis.

The exposed model implies that the organizational tools have the direct purpose of solving a problem posed by the organization and, indirectly, implementing the BPM. It stands out from this that it has been able to compile practically powerful contributions made so far in terms of BPM as critical success factors, maturity models and fundamental principles to implement BPM. However, it arises for future works delving into different ways of assessing the degree of development of every subfactor in any moment, so that the implementation of the components of BPM is more controllable.

Fig. 1 .

 1 Fig. 1. Relationship among Management Memorandum Model (MM), Business Process Management (BPM) and e-Government (Source: Authors' own)

Fig. 2 .

 2 Fig. 2. Direct and indirect purposes of the Management Memorandum Model (Source: Authors' own)

Table 1 .

 1 Advantages of BPM (Source: Authors' own, using information from several authors).

Table 2 .

 2 Critical success factors and sub-factors (Source: Rosemann and vom Brocke[START_REF] Rosemann | Six Core Elements of Business Process Management[END_REF]).

	STRATEGIC ALIGNMENT	ICT
	• A.1. Process improvement plan	• D.1. ICT for design and process modelling
	• A.2. Alignment between business	• D.2. ICT enabler for the implementation and
	strategy and processes	execution of processes
	• A.3. Business process architecture	• D.3. ICT for measurement and process control
	• A.4. Key processes outputs and	• D.4. Tools for innovation and process
	performance indicators (KPI) • A.5. Priority of stakeholders	improvement • D.5. ICT project management and program
		management
	GOVERNANCE	PEOPLE
	• B.1. Decision-making processes of	• E.1. Skills and expertise in processes
	BPM • B.2. Roles and responsibilities process • B.3. Processes data collected • B.4. Standardized management	• E.2. Knowledge of BPM • E.3. Teaching and learning processes • E.4. Communication and collaboration in the processes
	processes • B.5. Control of process management	• E.5. Leadership in management processes
	METHOD	CULTURE
	• C.1. Design and process modelling	• F.1. Responsiveness to shift to the process
	• C.2. Processes implementation and execution	approach • F.2. Values and beliefs about the processes
	• C.3. Process measurement and control	• F.3. Activities and behaviour to processes
	• C.4. Innovation and process	• F.4. Senior management leadership in process
	improvement • C.5. Project and program management	management • F.5. Social networking process management
	processes

Table 3 .

 3 Dual purpose of the organizational initiative (Source: Authors' own).

		ORGANIZATIONAL INITIATIVE
	TYPES OF PURPOSE	Direct	Indirect
	OBJECTIVES	problem To solve an organizational	To implement BMP
	METHOD	Organizational instrument (It must rely on BPM)	Criteria based on the Ten Principles of BPM.


Yoshiaki Matsuzawa
email: matsuzawa@si.aoyama.ac.jp

Yoshiki Tanaka

Sanshiro Sakai

Measuring an impact of block-based language in introductory programming

Keywords: Programming education, block-based language, learning analytics, working time analysis, compile error analysis

The use of block-based visual language in introductory programming is a popular method in education. However, there is little research which provides evidence showing advantages of block-based language. This paper presents the results of learning data analysis with fine grain logs recorded by students' development environment where the students can select their language in block-based or Java. A total of 400+ students' logs collected each of four years were analyzed. The results show that migration from Block to Java can be consistently seen each year, although the whole block-editing rate was influenced by the method of the instructor's introduction. Though block-editing did not affect working time and Lines of Code (LOC), it could reduce the compile error correction time, whereas using Java requires approximately 20% of compile error correction time for students. We concluded that block-based language worked to encourage students to focus high-level algorithm creation, as well as it provides an advantage to understanding text-based language.

Introduction

The body of introductory programming is not to develop an understanding of the grammar of particular programming languages, but it should develop the problem-solving skills with computing that is called "computational thinking" [START_REF] Wing | Computational Thinking[END_REF]. The similar concept has been proposed by the United Nations Educational, Scientific and Cultural Organization (UNESCO), as "designing a task-oriented algorithm" [START_REF]UNESCO: ICT Curriculum for School / Program of Teacher Development[END_REF]. Both statements include common sense. One is to focus on the thinking and creation of algorithms. Another is that computing is not only dependent upon the use of actual computers, but logical modeling for the required problem-solving.

A use of visual language, especially "building-block approach" [START_REF] Maloney | Scratch: a sneak preview[END_REF] is the most popular way to form the learning environment for the purpose of education. Many block-based languages for education have been proposed over more than two decades, and yet developers continue by trial and error to improve the language using modern software technology. The first workshop specifically blockbased language focused was held in the last year (Blocks and Beyond in Visual Languages and Human-Centric Computing(VL/HCC 2015) [START_REF] Turbak | Blocks and beyond: Lessons and directions for first programming environments[END_REF]). The workshop was able to collect a remarkable number of submissions, the participants discussed the design of the next generation of block-based language, including the topic of how to design the tools as a bridge to text-based language (e.g. [START_REF] Bart | Position paper: From interest to usefulness with blockpy, a block-based, educational environment[END_REF][START_REF] Monig | Blocks at your fingertips: Blurring the line between blocks and text in gp[END_REF]).

However, as the workshop stated "Despite their popularity, there has been remarkably little research on the usability, effectiveness, or generalizability of affordances from these environments" [START_REF] Turbak | Blocks and beyond: Lessons and directions for first programming environments[END_REF], there is little research which provides evidence showing advantages of block-based language. Practitioners using visual programming languages believe that the visual programming approach is an effective way in developing computational thinking because learners can focus on their problem-solving tasks [START_REF] Fal | How Scratch Programming May Enrich Engineering Education[END_REF]. The block-based language should be used as scaffolding to text-based language. However, the belief is not verified, and the percentage of degree is not clear yet.

Hence, we tried learning data analysis with fine grain logs recorded by students' development environment. We conducted the introductory education using the tools we developed, where the learners can switch their language between visual-block language (Block) and one for a text-based language (Java) by bidirectional translation technology. The data analysis was conducted mainly using the data of time students spent. These include amount of total working time, the time using block-editor, the time of compile error correction. The tile representation of each student shows gradual migration to Java on their own schedule, and reducing compile error correction time shows the success by focusing on high-level algorithm creation.

Related Work

There have been many reports of using block languages for introductory programming education [START_REF] Lewis | What do Students Learn About Programming From Game, Music Video, And Storytelling Projects?[END_REF], [START_REF] Ozoran | Using Scratch In Introduction to Programming Course for Engineering Students[END_REF], as well as reports on the development of new block languages [START_REF] Ingalls | Back to the future: The story of squeak, a practical smalltalk writtern in itself[END_REF], [START_REF]Scratch Team Lifelong Kindergarten Group MIT Media Lab: Scratchimagine.program[END_REF], [START_REF] Cooper | Teaching objects-first in introductory computer science[END_REF], [START_REF] Cheung | Filling the gap in programming instruction: a text-enhanced graphical programming environment for junior high students[END_REF], [START_REF]Blockly:a visual programming editor[END_REF]. Researchers agree that block-based languages feel familiar to beginners, however they are not a standard for use in introductory programming. Not a few researchers/practitioners claim disadvantages of blockbased language, particularly that block-based language is different from common practical languages used in industry, so that students have difficulty in moving to text-based programming after their introductory programming course.

Hence, which text-based or block-based language we should use in introductory programming is still an unsolved theme in this field. Lewis et al., conducted direct comparison using Scratch and Logo [START_REF] Lewis | How programming environment shapes perception, learning and goals: logo vs. scratch[END_REF], however, their results were very limited to illustrate the advantage of block-based language. Matsuzawa et al. demonstrated gradual migration from block-based to text-based language using a bidirectional translation system [START_REF] Matsuzawa | Language migration in non-cs introductory programming through mutual language translation environment[END_REF]. Although the limitation of the study is that they showed only trajectory of their working time, they demonstrated the capability of transition to text-based language from block-based language.

From the viewpoint of tool proposition, a rich number of tools have been proposed on migration from block-based languages or text-based languages, and it is still growing. Pasternak [START_REF] Pasternak | Visual programming pedagogies and integrating current visual programming language features[END_REF] proposed offering the ability to translate a program written in Block to a program written in Java. Google Blockly [START_REF]Blockly:a visual programming editor[END_REF] can be translated into multiple languages (e.g., into Java-script or Python) from its block representation. In 2012, Dann et al. tackled the problem of figuring out how students can transfer their knowledge of Alice3 (a block-based language) to Java programming, and they reported that experience with a block language behaved as a helpful scaffolding for students learning to program in Java [START_REF] Dann | Mediated transfer: Alice 3 to java[END_REF].

The present research projects are attempts to raise the "ceiling of programming" up to practical programming levels. One approach is to raise the ceiling of block-based language. Harvey et al. [START_REF] Harvey | Bringing no ceiling to scratch: Can one language serve kids and computer scientists? Constructionism[END_REF] proposed a functionality they called BYOB ("Build Your Own Block") in Scratch. This functionality expands the descriptive capabilities of the block language. The purpose of the project is to support a block language suitable for a wide range of users, from beginners to professionals. The developers of Scratch and Squeak are currently working on a GP project [START_REF] Monig | Blocks at your fingertips: Blurring the line between blocks and text in gp[END_REF]. GP will be the first block-based language written by itself, where learners can explore the entire software system by the level of learners' interest. It means the tool will provide an open-ended, no ceiling environment, where either text-based or block-based is merely one of the representations.

Another approach is to promote migration from block-based to text-based language by making educational scaffolding on the tools. The most popular approach is considered to develop language translation systems. Warth et al. proposed a bidirectional translation system, "TileScript," between a block language and JavaScript [START_REF] Warth | Toward a more scalable enduser scripting language[END_REF], although that research verified only that the technical requirements could be met to implement the prototype system. Matsuzawa et al. demonstrated a mutual translation between Block and Java [START_REF] Matsuzawa | Language migration in non-cs introductory programming through mutual language translation environment[END_REF]. Similar attempts are growing a variety of languages. BlockPy [START_REF] Bart | Position paper: From interest to usefulness with blockpy, a block-based, educational environment[END_REF] is designed for education in Python. PencilCode [START_REF] Bau | Pencil code: Block code for a text world[END_REF] is specially designed for migration to text-based language using modern Javascript technology.

Improvement of the editing environment of text-based language is another approach. One distinction of the BBC micro:bit project described above is providing an intermediate layer of editing system called TouchDevelop, which provides some scaffolding for text editing and, is designed for the migration to text-based language. Homer et al. proposes Tiled Grace, which is a tiled representation of Grace and, was an originally designed text-based language for education [START_REF] Homer | Combining tiled and textual views of code[END_REF]. Kolling et al., who is the founder of GreenFoot, proposes the "Frame-Based Editing" [START_REF] Kölling | Frame-based editing: Easing the transition from blocks to text-based programming[END_REF] where the "frame" is added to support novices with Java.

Method

Tool: Bidirectional Translation System

We conducted empirical studies in our introductory programming class over the span of four years. The goal of the study was to evaluate the impact of introducing block-based language. In each year, we used a bidirectional translation system between Block and Java proposed in the previous study [START_REF] Matsuzawa | Language migration in non-cs introductory programming through mutual language translation environment[END_REF]. The resulting environment has two interfaces, one for the (visual) Block language and one for the textual Java language, as well as a system for bidirectional translation between the two languages. Figure 1 shows the user interface for the environment. All the students in our class were given the opportunity to select the language they used to solve their programming assignments.

Research Question

In the previous study, Matsuzawa et al. demonstrated that learners would choose to use Block first, which will act as scaffolding for learning programming, and then learners will gradually migrate to Java on their own schedule. We expanded the research to explore the further impact of the environment. The original two research questions for this paper are as follows:

RQ1 Can the gradual migration nature be seen consistently every year, even if some variables change (students, teachers, method of teaching, or tool functionality)? RQ2 Does the tool(BlockEditor) successfully encourage students to focus their algorithm creation by removing compile error correction opportunity? If so, how much wasted time can we save?

Educational Environment Descriptions

The introductory programming course was designed for art students, rather than for computer science students. Therefore, the objective of the course was to develop an understanding of task-oriented programming. The objective was independent from any programming language, although Java language was used in the actual environment. Approximately 100 students participated in this course each year; two lecturers and six teaching assistants conducted the class.

Because of RQ1, we conducted an action research. Actions taken each year in our experimental course were listed as follows:

1st year (2012) The first year we started to use BlockEditor with the Block-Java translation system. 2nd year (2013) Major improvements in BlockEditor were made, especially the functionality of the variable scope indicator and the design of the block for method creation. The improvement was made using the method calling navigation system MeRV [START_REF] Ohata | Merv: A scaffold to promote creating 2d map of method call structure in block-based programming language[END_REF]. 3rd year (2014) Main lecturer changed from the author of Block Editor to a teacher who was from outside the research. The lecturer used Java using sample codes, whereas the former lecturer used Block for explanation. 4th year (2015) Compulsory assignments using Block was reduced from 4 times to once. The opportunity for everyone to use Block was reduced.

Metrics

All the students' activities in the development environment were recorded by PPV (Programming Process Visualizer) [START_REF] Matsuzawa | Programming Process Visualizer: A Proposal of the Tool for Students to Observe Their Programming Process[END_REF], and the logs used to calculate metrics to analyze the learning process. We calculated five kinds of metrics as follows for both RQ1 and RQ2.

Working Time (WoT) The working time for each assignment. It was calculated by summing up the time and excluding periods of longer than 5 minutes with no user operation in the development environment. Block Editing Working Time Ratio (BWT%) The ratio of working time to block out of the total working time. We can assume students used Java outside of BWT.

BW T % = BW T (W orkingT imewithBlock) W oT (1)
Compile Error Correction Time (ECT) Compile Error Correction time calculated by activity logs for each compilation error occurrence. Further description will be provided below.

Compile Error Correction Time Ratio (ECT%)

The ratio of compile error correction time out of the total working time.

ECT % = ECT W oT (2)
Lines of Code (LOC) Lines of Code for finally submitted assignment.

The technical description of the calculation method for ECT was provided by this paper [START_REF] Hirao | Compile error collection viewer: Visualization of learning curve for compile error correction[END_REF]. The difference in time between the error occurred and resolved, will be calculated in a general case. However, sometimes multiple errors occur or are resolved at the same time. For such cases, the system calculates by the amount of time spent in error collection BY the number of errors. An assumption of the method is the difficulty of corrections is equal in every case.

Results

Results of Statistics

Descriptive statistics for all calculated metrics are summarized in Table 1. For each metric, the number shows the average of the value, and the number in parenthesis shows the standard deviation of the distribution. The unit of value for BWT% and ECT% is the percentage amount per hundred, for WoT and ECT is minutes, for LOC is the number of lines, respectively. The average of BWT% was 32.7 for all year, although the data for each year indicates the variations in value. The total average of ECT% was 14.1. If we assume that block can swipe ECT, and the fact of BWT%, we estimate the ECT% using Java as approximately 20%. We calculated Pearson Correlation for all combinations of the 5 metrics, the results are shown in Figure 2. The most significant result is BET% has a negative correlation with ECT%, which clearly indicates the Block-editing was able to reduce the compile error correction time. The results also indicate Blockediting did not affect working time and LOC, which means that the quality of the outcome by Block-editing is equivalent to the outcome by text-editing.

Grid Representations

We created a grid representation of the rate of BlockEditor using (BET%) to illustrate the nature of the seamless migration from Block to Java. The representation is shown in Figure 3. Each column represents one task assigned to students, arranged in chronological order from left to right. Each row represents one student; the rows are sorted by course-average BET% with higher rows indicating higher average use. Each cell represents the value of BET% for a particular student completing a particular task. High-intensity color indicates a high BET%, and low-intensity indicates an BET% of 0 (i.e., using only Java). We selected 36 tasks which were mostly common for all four years, however three of the tasks selected were lacking in 2014. The tasks can be seen as vertical white belts in 2014.

We can observe that the seamless migration both overall (by the number of students who used Java in the class) and at the individual level for all 4 years were consistent. A sudden drop of the whole rate could be seen in the middle of 2012; however, it improved to a milder migration in 2013. As the class content involved introducing method/function, the improvement was caused by the action of improvement of tool functionality, as we described in section 3.3. Although the pattern of gradation of migration can be seen each year, the entire density is different; 2014 and 2015 are low-density. The density corresponds to the average ratio of the year as shown in Table 1. In the classes of 2014 and 2015, we could not observe any other difference with the exception of the actions described in section 3.3; the cause of the difference is influenced by the method of the instruction by the teacher, and opportunities to use Block.

We created another grid representation using Compile Error Correction Time Rate (ECT%). The representation is shown in Figure 4. When we compare Figure 3 and Figure 4, we can observe that inverted illustration. This means the Block reduces the compile error correction rate, the result corresponds with the result of statistic correlation (Figure 2) and our intuitive sense. Cells with high ratio focus on the latter part of the class (right-side on charts), because more advanced grammar content (e.g. Method/Function, collection) was introduced. Another significant result we can observe is that the factor of task was bigger than the BET% in the past. For example, the latter part of 2012 clearly illustrates that density of ECT% shows no difference between the high BET% group and the low BET% group.

Discussion

RQ1 asked "Can the gradual migration nature be seen consistently each year, even with some variable changes?" The results were generally positive: we can consistently see the migration nature. These results reinforced evidence that a block language can successfully act as scaffolding for students learning text-based programming. However, the factor of the method of instruction, especially which language is used in the explanation in the lecture affects the selection of language. The impact can be calculated by comparing the lowest 19% of BET%(2015) to the highest 46%(2013). The result indicate that not only the tools, but also the educational design including the use of tools is important.

RQ2 asked "Does Block encourage students to focus their algorithm creation by removing compile error correction opportunity?". Both statistics and microanalysis using grid representations indicate that Block clearly eliminates compile error correction time. The result itself is not surprising; however, it is not a small thing that the research revealed the amount (approx. 20%) of overhead which was wasted for compile error correction. We can eliminate 20% of overhead by Block, also can reduce to 10% in mixture usage. Additional observation shows that some students select Block-editing after being plagued with compile error correction. Hence, we concluded that block-based language worked to encourage students to focus on high-level algorithm creation.

The lack of direct evidence of the learners' understanding is still a limitation and requires further consideration. We believe that the results could reinforce existing evidence to encourage teachers who use a block language in introductory programming education.

Fig. 1 .

 1 Fig. 1. Bidirectional translation system between Block and Java.

Fig. 2 .

 2 Fig. 2. Correlation table between 5 metrics

Fig. 3 .Fig. 4 .

 34 Fig. 3. Grid representation of BWT%(Block Editing Rate) for each student.

Table 1 .

 1 Descriptive statistics for the calculated metrics.

	year	#student#task BWT%	WoT	ECT	ECT%	LOC
	2012 102	36	36.7(17.5) 47.9(15.8) 6.8(2.9)	14.3(4.7) 57.9(14.4)
	2013 96	38	45.9(23.3) 51.7(25.3) 5.3(2.6)	12.2(4.6) 66.3(36.6)
	2014 106	34	29.6(21.8) 44.6(14.4) 5.9(3.5)	14.6(5.9) 65.5(59.1)
	2015 100	48	19.2(23.7) 43.4(15.5) 5.8(3.0)	15.1(5.5) 65.7(49.2)
	total 404	156	32.7(23.7) 46.8(18.4) 6.0(3.1)	14.1(5.3) 63.8(43.4)
				0	100 200
				BWT%
					0.10 -0.17

Acknowledgments. This work was supported by JSPS KAKENHI Grant Numbers JP25730203, JP26280129.


Matthias Kramer
email: matthias.kramer@uni-due.de

David Tobinski
email: david.tobinski@uni-due.de

Torsten Brinda
email: torsten.brinda@uni-due.de

Modelling Competency in the Field of OOP: From Investigating Computer Science Curricula to Developing Test Items

Keywords: Competency modelling, item development, competency measurement, object-oriented programming, K12 education

In this paper, we describe the results of a thorough analysis of 44 K12 computer science curricula and standards documents conducted as part of an ongoing research project aiming at the development of a competency structure model and measurement instruments in the field of object-oriented programming (OOP). The curricula analysis builds upon a first model draft derived theoretically from a literature analysis in prior work. The model draft is 4dimensional and consists of the four competency dimensions 1) OOP knowledge and skills, 2) Mastering representation, 3) Cognitive processes and 4) Metacognitive processes. We used these dimensions and the belonging subdimensions as a coding scheme and coded competency facets concerning OOP contained in the curricula and standards documents using the method of qualitative content analysis according to Mayring. This way, we could firstly successfully prove the curricular validity of our model draft and secondly, after a step of paraphrasing the identified competency facets, use these descriptions to initiate the process of item development to operationalize our competency model draft.

Introduction

One of the current trends in educational research is the defining and measuring of learning outcomes in terms of competencies [START_REF] Klieme | The concept of competence in educational contexts[END_REF][START_REF]Assessment of competencies in educational contexts[END_REF]. Besides various competency modelling approaches in different disciplines in Germany (e.g. [START_REF] Jordan | The development of competency models: An IRT-based approach to competency assessment in general music education[END_REF][START_REF] Linck | Competence model for informatics modelling and system comprehension[END_REF][START_REF] Schecker | Modellierung naturwissenschaftlicher Kompetenz[END_REF]), on an international level a lot of research has been performed in the context of the Programme for International Student Assessment (PISA) by the Organisation for Economic Cooperation and Development (OECD) which -since the year 2000 -focusses every three years on competencies in the areas of mathematics, science and language. Formerly defined post-hoc (e.g. [START_REF] Neumann | Die Modellierung physikalischer Kompetenz und ihrer Entwicklung[END_REF]), the assessed competencies now rely partly on empirically founded competency models [START_REF] Neumann | From PISA to educational standards: The impact of large-scale assessments on science education in Germany[END_REF]. The results of these regular assessments are used to compare the educational levels of different countries or federal states. An expertise which analysed the "shocking" results (at least from the German perspective) of the first PISA studies under consideration of the different national educational systems came to the key conclusion that those countries performed best which oriented their educational systems at the intended learning outcomes [START_REF] Klieme | The development of national educational standards. An expertise[END_REF] (and not at detailed curricula as before) and recommended (besides other aspects) the development of empirically founded competency models for all educational fields.

Concerning computer science education (CSE), the demand for competency models is still up to date. Although different normative models have been developed as educational standards in the past years (e.g. [START_REF] Brinda | Bridging ICT and CS: educational standards for computer science in lower secondary education[END_REF][START_REF] Seehorn | CSTA K-12 Computer Science Standards: Revised 2011[END_REF]), empirically founded competency models in the CSE field are still rare (e.g. [START_REF] Linck | Competence model for informatics modelling and system comprehension[END_REF]). So, we have started a project to develop such a competency structure model and appropriate measuring instruments for the field of object-oriented programming (OOP), which is widely included in many K12 computer science curricula worldwide. The long-term goal of this research is to develop instruments to compare the learning outcomes in the CSE field nationally and internationally between different student populations.

Background and Related Work

On the Concept of Competency

In the context of competency modelling and assessment the term competency is often used according to the definition of Weinert. In his expertise for the OECD [START_REF] Weinert | Concept of competence: A conceptual clarification[END_REF] he defined competencies 2001 as "the cognitive abilities and skills possessed by or able to be learned by individuals that enable them to solve particular problems, as well as the motivational, volitional and social readiness and capacity to use the solutions successfully and responsibly in variable situations". Despite the awareness that metacognitive factors can have major impact on learning results, these aspects of competencies are quite difficult to assess. Hence, Klieme et al. used a more economic definition in a priority programme of the German research foundation on competency modelling by defining competencies as "context-specific, cognitive dispositions of achievement which are functionally related to situations and requirements of certain domains in the sense of specific learning and action fields" [START_REF] Klieme | Kompetenzbegriff und Bedeutung von Kompetenzen im Bildungswesen[END_REF]. In this context, competency models are used to describe the cognitive dispositions that learning individuals need to solve tasks and problems in a specific content or requirement area. Klieme et al. distinguish between modelling the structure, the level and the development of competencies in corresponding competency models [START_REF] Klieme | The concept of competence in educational contexts[END_REF]. Schecker and Parchmann [START_REF] Schecker | Modellierung naturwissenschaftlicher Kompetenz[END_REF] distinguish between normative (which describe the behaviour learners should show after a specific period of education, usually found in educational standards) and descriptive competency structure models (which describe the cognitive processes that occur in specific learning situations).

The COMMOOP Project

Our project COMMOOP aims to develop a normative competency structure model [START_REF] Klieme | The concept of competence in educational contexts[END_REF][START_REF] Schecker | Modellierung naturwissenschaftlicher Kompetenz[END_REF] and appropriate measurement instruments for beginners, such as secondary school or undergraduate students, in the field of object-oriented programming (OOP). We started by reviewing existing literature on competency modelling in other subject areas (such as mathematics, science, humanities, arts, computer science and problem solving) regarding the development methodology as well as the model structures (for a detailed description of this process, see [START_REF] Kramer | A Competency Structure Model of Object-Oriented Programming[END_REF]). We identified the areas of knowledge structures, content representation, cognitive processes as well as metacognitive influences as common, recurring structural elements, and verified, expanded and refined these based on an extensive literature analysis on theoretical and empirical studies on teaching and learning as well as on psychological aspects in the field of OOP. As theoretically derived candidates for potential competency dimensions we identified: The proposed dimensions span a multidimensional competency space. Each competency candidate that is contained therein covers a certain subspace that is limited by fixed values on some dimensions that can be assigned to this competency. For example, the competency candidate "be able to program arrays in Java" might cover a subspace that is limited by the following fixed values: 1.1 Array, 2 Java, 3.1 Translating the solution in a computer program and 3.2. Understand and Apply. The remaining dimensions might be covered in total by this competency. It has to be noted that, due to national regulations of Java being one of the predominant languages in schools, our research will first focus on this language.

The theoretically derived model draft was validated based on various competency descriptions in terms of applicability and completeness. For this purpose, we identified competency descriptions related to OOP in 44 computer science curricula and standards from several countries and compared these with our model (see section 3). In 2015, an international working group at the ACM ITiCSE conference analysed 14 case studies on K12 computer science education in 12 different countries, which resulted in 247 extracted competency descriptions. Out of these, 119 were related to programming and could be fully integrated in our model. Finally, the structure model was aligned with the results of a survey among 59 computer science teachers and teacher students on learning difficulties. Their views on which skills and abilities someone needs, to become a competent programmer as well as their point of view on typical problems during that learning process could be also fully integrated in our model.

During this process, we had to adjust our theoretically derived model only in minor subsections (see section 3).

Learning Metrics, Psychometric Models and Item Development

The discussion around coding to be the new literacy is vivid and the way to a proficiency scale in computer literacy is in its first stages. In PISA the development of scales followed six steps: (1) Identifying possible scales, (2) Assigning items to scales, (3) Skills audit, (4) Analysing field trial data, (5) Defining the dimensions and ( 6) Revising and refining with main survey data.

In a first step, we identified OOP as an important subscale [START_REF] Kramer | A Competency Structure Model of Object-Oriented Programming[END_REF]. Currently we're dealing with the construction of items as well as bringing them on their way to first field trials. For this stage of development, a more functional concept of competency is followed. This perspective is more interested in abilities to cope with challenges in specific situations rather than in the generative and cognitive systems behind [START_REF] Mcclelland | Testing for competence rather than for intelligence[END_REF]. From this background items have a single achievement threshold; the derived information only distinguishes dichotomously between correct and incorrect answers. Thus, the Rasch model can be applied to the raw data, in PISA a generalized form of the Rasch model has been used [START_REF]OECD: PISA 2012[END_REF].

The Rasch model is "the only model to date that provides the tools for approximating objective reproducible additive measures in the human sciences." [17, p. 8]. It follows the concept that data conforms a reasonable hierarchy on a single continuum of interest. The most important fact is the measuring of a single variable with a map of persons (ξ v ) and items (σ i ) on the same scale, therefore the relationship between the participant and the item is probabilistic. The resulting model can be regarded as a model of competency levels. While those levels have already been regarded in the step of item construction, they will be precised by the step of skills audit and the analysis of the field trial data. At this stage numerical scores and terms of content, which describe what persons who achieve a specific level know and can do, can be reported. The final assessment of COMMOOP will be a computerized adaptive test (CAT), wherefore the Rasch model is essential [START_REF] Frey | Multidimensional Adaptive Measurement of Competences[END_REF].

Generative and structure models are more interested in the components behind the specific ability in a concrete item [START_REF] Klieme | The concept of competence in educational contexts[END_REF]. Modelling latent variables, like working memory or intelligence, comes into the focus, which is mostly driven by cognitivism. These models are rather the result of small experimental designs than of large-scale assessments. Modelling the cognition behind competencies is nowadays a challenge. One of its fruits will be the improvement of intelligent tutorial systems (ITS) and the rise of cognitive tutors. A pioneer of this type of modelling is John R. Anderson and his ACT-R Theory [START_REF] Anderson | An integrated theory of the mind[END_REF][START_REF] Anderson | Cognitive tutors: Lessons learned[END_REF]. Anderson and his colleagues have realized different ITS considering the cognitive limitations of the learner, one specific ITS is about learning to code in LISP. At a later stage COMMOOP will regard cognitive models for code trainings. The prior testing will combine the most promising items with different working memory, intelligence and planning scales.

A key step in our model development process (see section 2.2) was a thorough analysis of K12 computer science curricula and educational standards. Altogether, we analysed 44 documents (German and international ones) and extracted competency facet descriptions concerning OOP from them. We performed this step because of two main reasons: first, we wanted to check the applicability of our theoretically derived normative competency model draft by trying to assign the competency facet descriptions to the proposed model categories to check and to ensure "curricular validity". This was an important step also in other competency modelling projects (cf. e.g. [START_REF] Jordan | The development of competency models: An IRT-based approach to competency assessment in general music education[END_REF][START_REF] Linck | Competence model for informatics modelling and system comprehension[END_REF]). Second, we wanted to prepare the operationalization of our model draft by starting to develop test items suitable to measure the competency facets found in the curricula.

Since we plan to measure OOP competency first in German schools, we included all publically available computer science curricula of Germany's 16 federal states as well as national standards documents of the Board of Federal Ministers of Education and Cultural Affairs and the German Informatics Association in the analysis. If a federal state had different curricula e.g. for lower and upper secondary computer science education, we included both documents in the analysis. To ensure the applicability of our model also in the international context, we also investigated important documents published in English language by other nations, such as the "Computing at school" curriculum (UK), as well as recommendations of international organizations, such as the ACM/IEEE curriculum and the CSTA K12 standards. We included documents from secondary and higher education, because we aim to model and to measure competencies of programming novices regardless of the educational level.

Curricula and standards documents are usually highly structured documents, in which intended learning outcomes are ordered e.g. by age level or by fields of the discipline (such as "algorithms" or "impact on society"). We used scientific definitions of the term "programming" taken from the literature (e.g. [START_REF] Blackwell | What is programming?[END_REF][START_REF] Böszörményi | What is programming?[END_REF][START_REF] Williamson | What is Programming?[END_REF]), extracted possible search terms from them, such as "writing" and "implementing", and searched each of the documents for the sections, in which relevant OOP competency facets were located. Within these sections we performed a qualitative content analysis (QCA) according to Mayring [START_REF] Mayring | Qualitative content analysis-research instrument or mode of interpretation[END_REF] using the tool MAXQDA. We used the dimensions and subdimensions of our theoretically derived normative competency model draft as our category system and coded each OOP competency facet found in the documents with its belonging category. The QCA was performed deductively and inductively (cf. [START_REF] Bortz | Forschungsmethoden und Evaluation für Human-und Sozialwissenschaftler[END_REF]), so the theoretically derived category system was extended inductively whenever a new category candidate was found in the documents.

Since we cannot present the whole coding process and its results in detail here, we give two examples. The text fragment "By the end of this course, students will […] use constants and variables, including integers, floating points, strings, and Boolean values, correctly in computer programs" (taken from the Ontario curriculum for Computer Studies in grades 10 to 12, Canada) was coded with the category OOP knowledge & skills, respectively with the subcategory data structure. The expected outcome that students should "know the basics of object-oriented programming, e.g. inheritance, polymorphism and encapsulation" (taken from the computer science curriculum of Saxony, Germany) was assigned to the subdimension class and object structure.

Overall, our model draft proved to be well applicable for the classification of most of the OOP competency facets found in the documents, which contributes to the curricular validity of our approach. Still there are three places, in which we modified it during the coding process: First, we found competency descriptions relating to the ability of being able to evaluate a different solution approach for a problem, so we added a subdimension "Evaluating different strategies" in the proposed competency dimension Problem solving stage. Second, it became obvious that syntax and semantics are almost inseparable and that third, there was a need to code competency facets relating to "Documenting and maintenance" (both: top-level dimension Mastering representation) to cover text fragments relating to the use of comments in the code as well as to writing clear and maintainable code. Especially because of the inseparability of syntax and semantics we decided to combine the subcategories belonging to Mastering representation in the top-level category and to analyse its inner structure in future work. Table 1 shows the distribution of codings at the end of the coding process. The next important step of the QCA according to Mayring [START_REF] Mayring | Qualitative content analysis-research instrument or mode of interpretation[END_REF] is the paraphrasing and summarizing of specific categories. The aim of this step is to combine similar competency facet formulations in fewer distinct joint formulations. Table 2 shows the result of this process within the dimension "OOP knowledge and skills".

It must be noted here that composite competency facet formulations like "Students are able to assign values to variables of primitive and compound data types as well as more complex data structures" were coded with more than one category.

It is not surprising, that fundamental algorithmic structures can be found most often in the documents, since neither does every country or state follow an object-oriented paradigm nor, in case they do, is the objects-first approach the most dominant one.

Developing Test Items

The next step on the way to an empirically validated competency structure model for OOP (see 2.2) is the operationalization of the competency facets of our theoretically derived model by describing specific situations that require a certain behaviour that persons show who are competent in this area. While competency formulations describe a subspace of skills and abilities limited by certain parameters (see 2.2), the items should only test for a distinct competency dimension in the model to prove that the assumed multidimensionality of the model can be shown empirically. Moreover, the items have to be constructed in a way that solving them correctly will indicate the underlying competency. Hence, many test items of varying difficulty are required. Furthermore, it is necessary to take into account the probands' previous knowledge to ensure the validity of the results. In Germany, students at the end of grade 12 should have the necessary competencies when they have gone through upper secondary computer science education or when they completed an "Introduction to OOP" course at university level.

As a result of the curricula analysis we have collected numerous formulations describing OOP competency facets and abstracted those in the paraphrasing step (see section 3). These descriptions are now used to construct test items, respectively to gain access to programming competencies and hence to validate the assumed theoretical competency structure on an empirical level. In the following, we give two examples of test items to be used at the end of an introduction to OOP. Item 1 is a considerably easy item addressing the understanding of classes as templates, objects as instances, attributes as characteristics and methods as modifiers of attribute values:

Table 1 .

 1 Coding results of the curricula analysis.

	Competency dimensions Subdimension	Number of codings
	OOP knowledge & skills	Data structure	104
		Class & object structure	74
		Algorithmic structure	164
		Notional machine	15
	Mastering representation	Semantics	36
		Syntax	43
		Programming language	37
		Documenting & maintenance 27
	Cognitive processes	Understanding the problem	67
		Determining a plan	103
		Translating the problem	116
		solution into a program
		Testing and debugging	108
		Evaluating different strategies 11
	Metacognitive processes		4

Table 2 .

 2 Categories formed in the subdimensions of the dimension OOP knowledge & skills by paraphrasing and summarizing the competency facet formulations

	Subdimensions	Categories	Number of codings
	Data structure	Data representation/structure in	52
		general
		Primitive data types	29
		Composite data types	36
		Dynamic data structures	27
	Class & object structure OO in general	20
		Fundamental OO-concepts	48
		(class, object, attributes,
		methods)
		Higher OO-concepts	19
		(polymorphism, encapsulation,
		inheritance)
		Using predefined structures	14
	Algorithmic structure	Algorithms in general	45
		Fundamental algorithmic	91
		structures (sequence, selection,
		repetition)
		Subprograms (methods,	21
		procedures, functions)
		Recursion	24
		Searching/Sorting	22
		Algorithmic optimization	19
	Notional machine		15

Mark all appearing class identifiers in red.

Item 1 can also be extended by various other tasks, such as "If objects are created, please write down the respective line number" or "Please write down all the attributes and methods a House-object can access". To raise complexity, it is reasonable to choose more advanced concepts, to differ in the respective task, to use more abstraction, etc. Item 2 is a more complex task addressing the understanding of inheritance, polymorphism and encapsulation. The correct solution of item 2 can indicate a correct understanding of inheritance along with a corresponding concept for polymorphism. As well as item 1 item 2 can be varied, e.g. by one doThis method in class C and asking to predict the correct output, by writing a new subclass D, using super to refer to attributes and methods in super classes and so on.

The items will finally be evaluated by means of Item Response Theory. Hartig and Frey [START_REF] Hartig | Sind Modelle der Item-Response-Theorie (IRT) das "Mittel der Wahl" für die Modellierung von Kompetenzen?[END_REF] already showed that for a multidimensional competency modelling this is a valid approach.

Summary and Outlook

In this paper, we gave an overview of the current state of our project COMMOOP, in which we aim to develop an empirically founded competency structure model for the field of object-oriented programming. We focussed on a curricula analysis, which we conducted to identify competency descriptions concerning OOP. We used these descriptions in a first step to check the applicability of our theoretically derived competency model draft for the classification of these descriptions. Second, we used the competency descriptions assigned to the subdimension "OOP knowledge and skills" to start the operationalization of our model by developing first test items to measure the proposed competencies. The process of item development is continuing under consideration of aspects, such as item difficulty and variety, suitability for adaptive testing and test length restrictions. Several iterations of pretests with the items will be conducted next on the way to develop a test instrument to validate the proposed competency structure.


Andreas Breiter
email: abreiter@uni-bremen.de

Datafication in Education: a Multi-Level Challenge for IT in Educational Management

Keywords: Educational governance, ICT management, datafication, information technology governance

While data-driven decision-making has become a new paradigm for school development and accountability, research on the underlying ICT infrastructures and the ICT management processes have been less prominent. With the trend of datafication, educational management gains new options but also requires adequate controlling mechanisms to take care of the data and to account for privacy and security. The information management cycle can be used to define all relevant aspects of the management process. This is adopted to the specific situation of educational institutions and furthermore extended to account for the different levels of educational governance. This leads to a new concept of educational technology governance as a necessary frame for supporting datafication processes.

stakes), which started in the U.S., Australia and the UK and has a growing importance all over the world [START_REF] Martens | Transformation of Education Policy[END_REF]. This output-orientation goes hand in hand with increased control (governance) of school systems [START_REF] Behn | Rethinking Accountability in Education. How Should Who Hold Whom Accountable for What?[END_REF], also based on neo-liberal reforms for 'new public management' [START_REF] Pollitt | Public Management Reform: A Comparative Analysis[END_REF]. Selwyn [START_REF] Selwyn | Data entry: towards the critical study of digital data and education[END_REF] points out that data are being generated and processed in increased volume, velocity and variety, resulting in what is known as large-scale data or big data. This trend has increasingly led to discussions on data mining and analytics in business, science and government. Williamson [START_REF] Williamson | Governing software: networks, databases and algorithmic power in the digital governance of public education[END_REF] speaks of 'governing software' and the 'emergence of "digital governance" in public education' (p.83). He argues that educational decision-making is increasingly being delegated to database-driven analytics software and states that 'software has now become a significant social actor that can govern and shape people's lives' [START_REF] Williamson | Governing software: networks, databases and algorithmic power in the digital governance of public education[END_REF].

Within the educational context, more data and more heterogeneous data are being generated-deliberately-for monitoring, surveillance or evaluation purposes but also-automatically-through routine operations of digital devices and systems [START_REF] Selwyn | Data entry: towards the critical study of digital data and education[END_REF]. They range from computer-based tests to learning analytics on large-scale data in complex information systems. They allow the 'recording, storage, manipulation and distribution of data in digital form' [17, emphasis in original]. Digital data are distinct from pre-digital forms as they may be exhaustive in scope, highly detailed and can be combined in a flexible manner. Datafication has to be understood in different dimensions. As more and more media are based on computer devices and software, we generate 'digital traces' ( [START_REF] Karanasios | Making sense of digital traces: An activity theory driven ontological approach[END_REF]) by using them. This can be aggregated and processed automatically. A rather new trend is learning analytics and educational data mining. While some embrace the new opportunities for improving teaching and learning [START_REF] Berland | Educational Data Mining and Learning Analytics: Applications to Constructionist Research[END_REF][START_REF] Papamitsiou | Learning Analytics and Educational Data Mining in Practice: A Systematic Literature Review of Empirical Evidence[END_REF][START_REF] Greller | Translating Learning into Numbers: A Generic Framework for Learning Analytics[END_REF], others refer to the challenges of an education system which is defined algorithmically [START_REF] Reimann | e-Research and learning theory: What do sequence and process mining methods contribute?[END_REF][START_REF] Schulz | Monitoring User Patterns in School Information Systems Using Logfile Analysis[END_REF][START_REF] Ferguson | Learning analytics: drivers, developments and challenges[END_REF].

Datafication and Educational Governance

Eynon [START_REF] Eynon | The rise of Big Data: what does it mean for education, technology, and media research?[END_REF] points out that the use of big data observed in the educational sector is not a new phenomenon, as it has been widely applied in the commercial sectors, where data sets are combined to understand markets, as well as in the field of natural sciences. In the business sector, customer satisfaction is usually utilized to measure how well a company performs. The more satisfied the customers are with the service provided by a company, the more profits are likely to be gained. Likewise, in the education sector, students' performance is usually used to measure the quality of education, i.e., the better the students perform, the better the quality of education is usually perceived to be. Education itself brings profit to a country from the perspective of improvement of human capital. In this context, educational government bodies can be seen as the main organizations that provide services to society in the form of education. Especially in education, the governance structure of the education system has a strong explanatory power to understand changes from the classroom level (micro) to the organizational level (meso) to the policy level (macro). ICT has to be embedded in each of these levels and has its impact across the levels. We follow the framework model of educational governance [START_REF] Sergiovanni | Educational governance and administration[END_REF][START_REF] Altrichter | Theory and Evidence on Governance: conceptual and empirical strategies of research on governance in education[END_REF] and contextualize it to the role of ICT in schooling [START_REF] Kozma | Comparative analyses of policies for ICT in education[END_REF]. The three-tier model of educational technology governance allows a broader view on the intertwined processes on the macro, meso and micro level.

On the macro level, in most countries, public pressure on changing education policy enforced by international non-governmental organizations (like OECD or IEA) can been observed since the publication of PISA results in the 1990s. Martens et al. [START_REF] Martens | Transformation of Education Policy[END_REF] explained different reactions of nation states to these pressures -from adoption of achievement tests in the national education policy, to ignoring it. They even identified a convergence of educational systems to follow the logic of the test systems [START_REF] Bieber | Towards a Global Model in Education? Interdisciplinary Perspectives on International Student Assessments and their Impact on Policies and Institutions[END_REF].

Other studies in education policy highlight numbers and evidence-based policy as the mode of educational governance in Australia [START_REF] Lingard | Education policy as numbers: data categories and two Australian cases of misrecognition[END_REF]. They point out that from a sociological point of view, it is necessary to be aware of the social construction behind the numbers, which come to constitute education policy, as an understanding of the socially constructed nature of data leads to the identification of its weaknesses and strategies to improve. On the meso level of the district (or region), comprehensive information systems have been built in order to cope with increasing data [START_REF] Wayman | Organizational considerations in establishing the Data-Informed District[END_REF][START_REF] Cho | Districts' Efforts for Data Use and Computer Data Systems: The Role of Sensemaking in System Use and Implementation[END_REF][START_REF] Coburn | Conceptions of evidence-based practice in school districts: Mapping the terrain[END_REF]. This level is rather under-researched as intermediaries are less in focus but also the differences of educational governance across countries are particularly strong. Depending on the level of autonomy of the schools, the centralization or decentralization of the national school system as well as the size of the school system to be attractive for commercial software companies varies. Hence, the micro level of the school according to data practices and school information systems has been studied extensively. This ranges from technology adoption of school information systems [START_REF] Passey | Next Generation of IT in Educational Management[END_REF][START_REF] Tatnall | Information Technology and Managing Quality Education[END_REF] by administrators and teachers to different uses of information systems for performance feedback [START_REF] Breiter | Assessment Information Systems for Decision Support in Schools[END_REF][START_REF] Verhaeghe | Using School Performance Feedback: Perceptions of Primary School Principals[END_REF][START_REF] Visscher | School Improvement Through Performance Feedback[END_REF] to learning analytics [START_REF] Berland | Educational Data Mining and Learning Analytics: Applications to Constructionist Research[END_REF][START_REF] Ferguson | Learning analytics: drivers, developments and challenges[END_REF]. In this paper, we argue that all information systems and the underlying infrastructures need to be managed, i.e. planned, organized and controlled.

Educational Technology Governance for a Datafied Education System

Educational Information Management

Data are a carrier of information that can be turned into knowledge when they are interpreted in a particular context by humans. This knowledge may lead to action by individuals or organizations when it is perceived as referring to a certain context [START_REF] Breiter | Data for school improvement: Factors for designing effective information systems to support decision-making in schools[END_REF]. Data, information and knowledge can be seen as a continuum where information management is rooted in an efficient and reliable infrastructure for data processing [START_REF] Choo | Information Management for an Intelligent Organization: The Art of Environmental Scanning. 3rd Edition[END_REF]. In the enterprise world, the purpose is to make an enterprise as intelligent as possible by valuing its knowledge assets for success. Choo [START_REF] Choo | Information Management for an Intelligent Organization: The Art of Environmental Scanning. 3rd Edition[END_REF] defines an intelligent organization as one, which is able to learn from its internal and external environments, by detecting and correcting errors between outcome and expectation. Internal environment in this context refers to any available data or information that originates from within the organization. Applying this to the educational sector, student performance can be improved by learning from data or information that resides in the information system the educational institution operates.

While in general information systems can be understood as technologies and associated people and processes, Visscher [START_REF] Visscher | Information technology in educational management as an emerging discipline[END_REF] defines an information system in the context of education as any computer or set of computers with data bases that enable users to record, process, retrieve and distribute data. Breiter et al. [START_REF] Breiter | Educational Management Information Systems -Case Studies from 8 Countries[END_REF] make a clear distinction of the following types of school information systems: (1) Assessment information system, (2) Learning management system, (3) Administrative school information system, and (4) Electronic school directory. An assessment information system is aimed at presenting the evaluation results of students and schools. Drawing on self-evaluation tests, classroom-based assessments or standardized tests, this information system is accessed by students, teachers, parents, and decision-makers. Learning management systems are about teaching and learning and are addressed at teachers and students, such as e-learning or learning platforms. Administrative school information systems are used for administrative or management purposes of school staff. Electronic school directories cover school master data, such as school profiles or addresses, which are usually made available to the public.

Information systems are means to the end for better decision-making, based on an ICT infrastructure (networks, databases etc.) which itself needs to be managed (fig. 1). Anagnostopoulos et al. [START_REF] Anagnostopoulos | The infrastructure of accountability: Data use and the transformation of American education[END_REF] define the underlying information infrastructure as an assemblage of people, technology and policy. This infrastructure is described as being similar to the physical infrastructure of cities or states, which includes canals, traffic and street intersections. For an efficient transportation across countries or regions, this infrastructure needs people to design, build and maintain it continuously, including its rules, norms, and underlying assumptions about its use. The term infrastructure itself is defined as something big, layered, and complex, where adjustment involves aspects dependent on the context applied [START_REF] Star | The Ethnography of Infrastructure[END_REF][START_REF] Star | Steps Toward an Ecology of Infrastructure: Design and Access for Large Information Spaces[END_REF]. Information infrastructures can be regarded as institutions or formative contexts on the basis that they 'constitute the background condition for action, enforcing constraints, giving direction and meaning, and setting the range of opportunities for undertaking action' [START_REF] Ciborra | From tool to Gestell[END_REF]. The implication is that as they are 'infused with value', they become more taken for granted and less expendable. When it comes to decision support systems in the education area, Anagnostopoulos et al. [START_REF] Anagnostopoulos | The infrastructure of accountability: Data use and the transformation of American education[END_REF] define the underlying information infrastructure as an assemblage of people, technology and policy, using the famous term of Latour [START_REF] Latour | Reassembling the social: An introduction to Actor-Network-Theory[END_REF]. This infrastructure is described as being similar to the physical infrastructure of cities or states, which includes canals, traffic and street intersections. For an efficient transportation across countries or regions, this infrastructure needs people to design, build and maintain it continuously, including its rules, norms, and underlying assumptions about its use. The information infrastructure itself is pointed out to have economic, social, political and cultural impacts. It is more than just technology, as it is (re)shaped through the interaction of individual, social, political and technological forces.

As in business, we can find additional cross-sectional educational management processes on the leadership level such as strategic development, human resources and staff development, organizational issues and controlling. On the top level, the availability of data to make informed decisions as well as the demand for this data has to be managed. Further research suggests that this is not a static concept but has to be regarded as an ongoing process across the levels. As we know from generic technology acceptance models [START_REF] Davis | Perceived usefulness, perceived ease of use, and user acceptance of information technology[END_REF][START_REF] Venkatesh | User acceptance of information technology: Toward a unified view[END_REF], as well as from research on school information systems [START_REF] Visscher | Information technology in educational management as an emerging discipline[END_REF], perceived usefulness and ease of use are predictors for integrating technology in everyday activities.

Fig. 1. Information management in education (adaptation from [49])

According to Choo [START_REF] Choo | Information Management for an Intelligent Organization: The Art of Environmental Scanning. 3rd Edition[END_REF], the information management cycle consists of a continuous loop with six distinct steps. It starts with (1) the identification of information needs, which basically consists of answering the questions of what kind of data are needed for which purposes. This is then followed by (2) information acquisition (data collection), (3) information organization and storage (data storage), (4) development of information products and services and (5) data distribution, which is executed by data organizers with the help of ICTs, and then (6) information use.

From an educational management perspective, input data in this system are not limited to student achievement data only, but the system can also be fed with other kinds of related data. Student achievement assessments have been widely accepted as the standard to evaluate student performance for class, school or international levels of comparison. Additionally, budget data, process data, students' e-portfolios or computer-generated data like logfiles [START_REF] Schulz | Monitoring User Patterns in School Information Systems Using Logfile Analysis[END_REF] or from learning analytics ( [START_REF] Berland | Educational Data Mining and Learning Analytics: Applications to Constructionist Research[END_REF][START_REF] Ferguson | Learning analytics: drivers, developments and challenges[END_REF]) can be added to build a heterogeneous data set with different formats, time span, from structured to non-structured and in different aggregation levels. These are typically all characteristics of "big data" ([50]).

Educational Technology Governance

The process character of the management cycle and the relevance of data for decision making on the different levels of educational governance illustrates the necessity for an overall plan and control mechanism. As this is already in place following the Deming-cycle of plan, do, check and act (PDCA) in IT Governance, the next step is to account for the special aspects of IT in educational management. Usually, the term information technology governance is used in two different directions: First, as a description of how decisions on ICT are made in complex organizations in order to align IT and business [START_REF] Wulf | Tailoring groupware for different scopes of validity[END_REF]. Weill and Ross [START_REF] Weill | IT Governance. How Top Performers Manage IT Decision Rights for Superior Results[END_REF] define information technology governance as follows: 'Specifying the decision rights and accountability framework to encourage desirable behavior in the use of IT' (p.8). They make a distinction between different forms of organizational structures: from IT or business monopoly in which one defines the action of the other (i.e. IT drives business or vice versa) to duopolies and to anarchy. Empirical research based on quantitative studies research shows a positive and significant linkage between information technology governance mechanisms and strategic alignment and, further, between strategic alignment and organizational performance [START_REF] Wu | How Information Technology Governance Mechanisms and Strategic Alignment Influence Organizational Performance: Insights from a Matched Survey of Business and IT Managers[END_REF]. In educational governance research, there is no correspondent research, yet.

Second, IT governance is used as a definition of meta-processes for 'good governance' to support strategic business development. The currently most accepted model is COBIT in its version 5 [START_REF]ISACA: COBIT 5 -A Business Framework for the Governance and Management of Enterprise IT[END_REF]. In COBIT 5, information technology governance is '… the system by which the current and future use of ICT is directed and controlled' (p.3). The standard identifies three core elements: (1) Evaluate: checking for current and future use of IT; (2) Direct: defining directives to prepare and implement strategies, plans and guidelines to secure IT business alignment; (3) Monitor: controlling the implementation and the capabilities. COBIT encompasses altogether 37 sub-processes, which are defined with activities, metrics and inputoutput relations. Furthermore, a maturity level can be assigned to each process in order to monitor quality for continuous improvement.

Transferred to educational technologies, this would mean to implement governance processes on both the strategic and the operational level, focusing on teaching and learning as well as on educational management.

Conclusions

As we have highlighted, the rise of data for decision-making on all levels of educational governance has not only implications for preventing unintended consequences but also for processes of ICT management. The information management cycle needs to be reflected on all three levels of educational governance. It is part of educational policy making, requiring adequate data provision, which allow national and international comparisons. This demand is built into information systems, which are based on ICT infrastructures (e.g. national indicators, ranking systems, databases by state and non-governmental organizations, state-wide information systems). On the meso level of the school district, district-wide information systems are in place, serving for the needs of administrators and policy makers. On the micro level of the school, building management needs data, implemented in school information systems and based on ICT infrastructure. Additionally, there are cross-sectional processes on each level and, as a distinct characteristic of the school system, also across levels. Not only the data provision is bottom-up from the school to the policy level (e.g. as reports and achievement test results) but also from school to district and from district to school, and vice versa. Furthermore, specific requirements to process sensible educational data such as grades, performance levels or attendance need to be embedded into an overall policy of privacy and security.

As we concluded, the strategic processes need to be implemented, managed and monitored. In order to cope with the challenge of datafication on a large scale, strategic processes of governance for educational technologies are necessary. This is relevant for both the administrative data practices and teaching and learning. Education systems and policy makers could save time and energy by adopting existing models from business. This would lead education and educational technology to the next level of professional management.