Stefan Pasterk
email: stefan.pasterk@aau.at

Max Kesselbacher
email: max.kesselbacher@aau.at

Andreas Bollin
email: andreas.bollin@aau.at

A Semi-Automated Approach to Categorise Learning Outcomes into Digital Literacy or Computer Science

Keywords: curriculum, computer science, digital literacy, natural language processing, primary education

Computer science related curricula, standards and frameworks are designed and implemented in many countries to incorporate informatics education in schools, already starting with kindergarten and primary education. A recurring point of discussion addresses the focus of those educational models concerning the different fields of computer science -the topics related to the scientific subject of computer science, or digital literacy (the set of skills and competencies needed in everyday life in the digital age). In this paper, we present a semi-automated approach to categorise learning outcomes of computer science related curricula into one of those two categories. Categorisation is performed with linguistic metrics computed for nouns and verbs of representative curricula of each category. The categorisation is compared against classifications of nine experts of computer science teaching and research. The results show a matching categorisation for 70% of all learning outcomes and 90% of learning outcomes uniformly classified by the experts.

Introduction

The incorporation of topics, skills and competencies related to computer science and computer literacy in primary education is currently in focus worldwide [START_REF] Barendsen | Concepts in K-9 computer science education[END_REF]. Curricula, standards and frameworks related to computer science are designed and implemented in many countries. The developed curricula differ in many aspects. A possible distinguishing factor of comparison is the focus between computer science and digital literacy categories. The former typically includes topics related to the scientific subject of computer science, while the latter includes skills and competencies needed in everyday life in the digital age. Experts are still in discussion about how the two terms should be correctly classified, which of the two should be focused, and where to draw the distinguishing line between the two categories regarding the formulation of learning outcomes. One of the problems that arises from those open discussion points is that the number of formulations and learning outcomes overwhelms researchers and curricula developers who seek to determine the focus of a curriculum. Curricula often include between dozens and hundreds of formulations and learning outcomes, and manually classifying them is a tedious work [START_REF] Pasterk | Digital literacy or computer science: Where do information technology related primary education models focus on?[END_REF]. This paper describes a semi-automated approach to categorise learning outcome formulations into computer science or digital literacy categories. Natural language processing (NLP) techniques [START_REF] Han | Data Mining: Concepts and Techniques[END_REF][START_REF] Bird | Natural Language Processing with Python[END_REF] are applied to analyse learning outcomes and extract categorisation features of representative curricula for both categories, building dictionaries of verbs and nouns with their respective fraction of occurrence in learning outcome descriptions. The approach is evaluated by categorising four computer science related curricula and comparing the results to a classification of experts of computer science research and teaching. The results show that it is possible to determine the focus of a curriculum with the NLP-based categorisation approach.

The remainder of the paper is structured as follows. Section 2 presents related work and contrasts the presented approach. Section 3 covers the educational models used for analysis and evaluation. Section 4 presents the experimental setup and the results. Section 5 discusses the results, an application and possible implications. Section 6 summarises the contribution of the paper.

Related work

With the recency of digital technology emergence, the concepts of computer science are in the focus of researchers worldwide, especially for primary education. Most of the resulting articles concerning computer science related curricula focus on one single curriculum and describe this, possibly new, approach in a detailed way. A few other publications analyse and compare different curricula for either primary or secondary education, although most curricula combine those two levels. The article from Barendsen et al. [START_REF] Barendsen | Concepts in K-9 computer science education[END_REF] focuses on computer science concepts in K-9 education (from kindergarten to school level 9) and considers curricula from England, Italy and the United States (US). To analyse the curricula, the learning outcomes of the documents are grouped into knowledge categories with the help of open coding. The occurrences and distribution of the codes within the knowledge categories are calculated and presented to compare the curricula. With the goal of designing a primary school curriculum for computer science and programming, Duncan and Bell [START_REF] Duncan | A pilot computer science and programming course for primary school students[END_REF], in a first step, compared different related curricula. For this purpose, they chose the main Englishlanguage curricula for the primary school level, the Computer Science Teachers Association (CSTA) K-12 Computer Science Standards [START_REF] Seehorn | CSTA K-12 Computer Science Standards: Revised 2011[END_REF], the England computing curriculum, and the Australian Digital Technologies curriculum [START_REF] Falkner | The Australian digital technologies curriculum: Challenge and opportunity[END_REF]. To identify possible key ideas and to show similarities as well as differences, the elements of the curricula were categorised into six content themes [START_REF] Duncan | A pilot computer science and programming course for primary school students[END_REF].

An overview of the global situation of K-12 education is given by Hubwieser et al. [START_REF] Hubwieser | A Global Snapshot of Computer Science Education in K-12 Schools[END_REF]. They use articles that discuss the situations in different countries as a corpus. Following the steps for qualitative text analysis, the corpus is categorised using the tool MaxQDA. They collected 249 competence statements and analysed knowledge elements like 'Algorithm' regarding the verbs used in combinations with them; as we will see later, this was a step that was also relevant for our work. The statements of the 'Goals' category were manually preprocessed and collected into content categories. Afterwards, they compared those new categories and showed which were covered in which countries [START_REF] Hubwieser | A Global Snapshot of Computer Science Education in K-12 Schools[END_REF]. The authors used a manual qualitative analysis approach to extract, categorise and summarise text passages with different topic foci from research texts. In this paper, we present an approach for semi-automatic extraction and categorisation of learning outcome descriptions from curricula documents. Instead of a categorisation considering computer science topics, we focus on the comparison of learning outcomes regarding computer science and digital literacy categories.

International educational models

Different educational models vary in organisational circumstances, learning goals, topics and teaching methods [START_REF] Hubwieser | Computer science/informatics in secondary education[END_REF]. With a high number of educational models, the number of basic pedagogical approaches used also rises. Some of them are based on learning objectives or statements. Most of them differ in formulation, details and volume. In this contribution, the umbrella term 'learning outcome' is used to collect all the statements, and the following definition is used: "Learning outcomes are statements of what the individual knows, understands and is able to do on completion of a learning process [START_REF]ECTS: Users' Guide[END_REF]." This definition suffices for the purpose of this contribution as the focus is on the used words and word combinations, not the structure or the volume.

Selected curricula, educational standards, and competency models

Following the related work, two of the main English-language educational models for computer science in primary education, the CSTA Computer Science Standards from 2011 and the Australian national curriculum for Digital Technology [START_REF] Duncan | A pilot computer science and programming course for primary school students[END_REF] are selected for this contribution. As a recent update, the new CSTA Computer Science Standards from 2017 [START_REF]CSTA: Standards[END_REF] and due tolocality, the curriculum 21 from Switzerland [12] were added. The selected curricula, educational standards and competency models are briefly described.

CSTA K-12 Computer Science Standards (2011).

The CSTA K-12 Computer Science Standards from 2011 [START_REF] Seehorn | CSTA K-12 Computer Science Standards: Revised 2011[END_REF] are well known and often referenced in relevant literature [START_REF] Barendsen | Concepts in K-9 computer science education[END_REF][START_REF] Duncan | A pilot computer science and programming course for primary school students[END_REF][START_REF] Hubwieser | Computer science/informatics in secondary education[END_REF]. They start with the kindergarten and last until the twelfth grade. A combination of the levels K-3 and 3-6 covers an age range comparable to primary education. These levels include 45 standards, 16 for levels K-3 and 29 for levels 3-6. Australian Curriculum (AC). As part of the learning area 'Technologies', the subject Digital Technologies was presented in Australia in 2013 [START_REF] Falkner | The Australian digital technologies curriculum: Challenge and opportunity[END_REF]. It is an obligatory subject from the first school year called Foundation (F) until the eighth year. The ninth and tenth year is elective. The learning outcomes are described for each level, representing two school years. That means levels F-2, 3-4 and 5-6 cover the age range of primary education. For this range, 22 learning outcomes can be found, six of them belong to level F-2, seven to 3-4, and nine to 5-6 [START_REF]Australian Curriculum: Digital technologies in focus[END_REF]. Curriculum from Switzerland (21). In Switzerland, the new curriculum for primary and lower secondary education called 'Lehrplan 21 (curriculum 21)' was presented and established in 2014 by 21 of 26 cantons with the possibility of individual adaptations [12]. It includes the subject 'Medien und Informatik (Media and Informatics)' from the first school year on. The levels of this curriculum are represented by 'cycles' containing three to four school grades. For primary education, it contains overall 44 competence levels, 14 for cycle 1 and 30 for cycle 2.

CSTA K-12 Computer Science Standards (2017). The reworked CSTA K-12

Computer Science Standards were presented in 2016 as a draft version and published in 2017. They differ from the older version in a lot of aspects, such as the levelling system and the strands. Considering primary education, the level 1A (age range from five to seven years), and level 1B (age range from eight to eleven years), are of interest. It contains 39 standards for primary education, 18 in level 1A and 21 in level 1B [START_REF]CSTA: Standards[END_REF].

Categorisation of learning outcomes

The categorisation of the learning outcomes is an often-applied method to compare educational models [START_REF] Barendsen | Concepts in K-9 computer science education[END_REF][START_REF] Duncan | A pilot computer science and programming course for primary school students[END_REF]. In most cases, the categories represent areas of interest, like 'Algorithms'. This contribution looks at two more general categories to identify the focus of the selected educational models: 'computer science' and 'digital literacy'.

Considering the different terminology used in computer science related educational models, it is necessary to clarify and define the term 'computer science' (CS) for this contribution. In English-language countries 'computer science' is a common term, especially in the US and Australia. In Europe, the term 'informatics' is frequently used. For this contribution, we use these terms synonymously, following the definition from the UNESCO/IFIP Curriculum 2000 [START_REF] Duchâteau | Information and Communication Technology in Secondary Education: A Curriculum for Schools and Programme of Teacher Development[END_REF]: "The science dealing with the design, realization, evaluation, use, and maintenance of information processing systems, including hardware, software, organizational and human aspects, and the industrial, commercial, governmental and political implications of these." This contribution builds on this definition of computer science and uses the abbreviation CS.

The terms 'digital literacy' and 'digital competence' can be used synonymously. In the 'DIGCOMP Framework for Developing and Understanding Digital Competence' in Europe [START_REF] Vuorikari | DigComp 2.0: The Digital Competence Framework for Citizens. Update Phase 1: The Conceptual Reference Model[END_REF], 'digital competence' is defined as "the confident, critical and creative use of [information and communication technologies] ICT to achieve goals related to work, employability, learning, leisure, inclusion and/or participation in society. Digital Competence is a transversal key competence which enables acquiring other key competences (e.g. language, mathematics, learning to learn, cultural awareness)". In the following sections, this contribution will use this definition of digital literacy and refer to it as DL.

The experiment

This contribution presents a semi-automated approach to categorise learning outcomes of different educational models with the aim of gaining information about their foci. To evaluate our approach, in a first step, experts were asked to categorise the learning outcomes into CS and DL using a questionnaire. The process and first results of this step have already been described by Pasterk and Bollin [START_REF] Pasterk | Digital literacy or computer science: Where do information technology related primary education models focus on?[END_REF] and are summarised and extended in section 5.1. In a second step, a categorisation with the help of natural language processing based on linguistic features is applied on the same learning outcomes. This process, the results and a comparison to the results from the experts' categorisation are presented in section 5.2.

Learning outcomes classified by experts

As described by Pasterk and Bollin [START_REF] Pasterk | Digital literacy or computer science: Where do information technology related primary education models focus on?[END_REF], a group of nine experts, consisting of four computer science teachers and five researchers in the field of computer science education, participated in a survey to categorise the learning outcomes of three selected educational models. To get a larger basis for the evaluation of the semi-automated approach, the survey was repeated with the same group of experts and with two additional educational models, the CSTA computer science standards from 2011 and from 2017. Every expert completed a questionnaire including all learning outcomes of the selected models for primary education in a order and had to choose one of the following categories: 'CS', 'DL', 'Both' or 'None'. Further, they were asked to describe their strategy for the categorisation process. Experts' Strategy. Considering the answers of the experts regarding their strategy, seven out of the nine experts referred to the definitions of CS or DL. Six experts used keywords that they assigned to either CS or DL. Finding keywords or key terms was the way to categorise for two experts. Two other experts focused on the topics of the learning outcomes and the combined objectives, which were often defined by keywords. Eight out of nine experts took keywords into account during categorisation. Results of Classification. First results have already been presented by Pasterk and Bollin [START_REF] Pasterk | Digital literacy or computer science: Where do information technology related primary education models focus on?[END_REF] and are summarised and extended in Table 1. The added results for the CSTA curriculum from 2011 and from 2017 can also be found in Table 1. The general categories CS and DL were determined by majority votes. Because of the possibility to choose 'Both', this method can lead to undecided learning outcomes. However, this concerned only a few learning outcomes, as can be seen in Table 1. Additionally, the learning outcomes where there was a strong agreement between the experts are included in Table 1. For those, at most, a single expert disagrees with the common classification. Overall, the inter-rater agreement value (Fleiss's kappa) is 0.43 and shows a 'fair to good' agreement, following the interpretation guidelines of Fleiss [START_REF] Fleiss | Statistical methods for rates and proportions[END_REF]. Discussion. The results of the experts' categorisation show that the selected educational models can be grouped into the two types 'focus on digital literacy' and 'balanced orientation'. As can be seen in Table 1, the Australian curriculum and the CSTA standards from 2017 have nearly a uniform distribution between CS and DL. Whereas, more than two-thirds of the learning outcomes from the curriculum 21 from Switzerland and the CSTA standards from 2011 were categorised into DL. Following the majority of the experts, those two educational models focus on DL.

Categorisation by linguistic features

We now present an automated categorisation approach based on linguistic features to assign a category, either CS or DL, to each learning outcome of the four analysed curricula. The categorisation results are evaluated against the expert classification. Linguistic Processing for Analysis. The analysed curricula are available as portable document format (PDF) documents. Manual preprocessing was done by extracting the texts of the learning outcomes. The extraction is implemented in Python. The process of extraction of the linguistic features included the following basic techniques: normalisation of words to improve comparability (lowercase, lemmatising); stop word removal with a list of English stop words; and word tokenising to produce term lists. Each learning outcome text constitutes a single element, called document, in the analysis. Tagging is applied with a trained part-of-speech tagger [START_REF] Toutanova | Feature-Rich Part-of-Speech Tagging with a Cyclic Dependency Network[END_REF] to extend the words with part of speech categories. The learning outcome text is tagged in full sentence form -'the students will be able to' is added at the beginning. Tags are grouped; one of the following categories is assigned to each word: noun, verb, adjective, adverb, and other. After tagging, the sentence start is removed, and the tag category and lemmatised words are stored as term list for each learning outcome. Categorisation Process. For the categorisation, linguistic frequency measures of curricula representative for CS and DL education are computed, following the same linguistic processing, and stored in two dictionaries. The 'Computer Science Curricula 2013' (AIE) [START_REF]Curriculum Guidelines for Undergraduate Degree Programs in Computer Science[END_REF], created by a cooperation of the Association for Computing Machinery (ACM) and the Institute of Electrical and Electronic Engineers (IEEE) members, contains a set of curriculum guidelines for undergraduate CS programmes and is used to build the dictionary for CS. For DL, the dictionary is built from three curricula. 'The Digital Competence Framework for Citizens' (Dig) [START_REF] Ferrari | DIGCOMP: A Framework for Developing and Understanding Digital Competence in Europe[END_REF] designed by the European Union in 2017, 'British Columbia Digital Literacy Framework' (BC) [20] from the Province of British Columbia, and 'Digi.Komp' (DK) [START_REF] Digi | Die Initiative[END_REF] from the Austrian initiative for digital competencies and informatics education. The considered linguistic features include term frequency (TF), term frequency over inverse document frequency (TF-IDF) and document frequency (DF) [START_REF] Han | Data Mining: Concepts and Techniques[END_REF][START_REF] Bird | Natural Language Processing with Python[END_REF]. The metrics TF and TF-IDF performed poorly for the categorisation because of size differences in the dictionaries. For categorisation, the DF value is used. In context, this value describes in which fraction of learning outcomes a term occurs.

For each learning outcome to be categorised, the sum of the DF values of the occurring tagged terms in the two dictionaries was computed. Insights from the experts' strategies suggested that content terms (nouns), cognitive activity terms (verbs) and their combinations should be considered. In this contribution, individual terms tagged as nouns and verbs are counted. The highest value determines the category. When both sums are within 10% of the highest value, a third category 'undecided' is assigned. Figure 1 shows an example of this process. Comparing Categorisation Results to Experts' Classification. The automated categorisation results are compared to the expert classifications in two ways. First, the categorisation is compared against the expert classification regarding all learning outcomes of the curricula. The results show the categorisation performance for a wide range of learning topics. Second, the categorisation is separately compared against the classification of learning outcomes for which the experts showed a strong agreement.

"The students will be able to recognize that software is created to control computer operations."

[recognize, software, create, control, computer, operation] CS value: 0.0054 + 0.0928 + 0.0018 + 0.0180 + 0.0243 + 0.0180 = 0.1603 DL value: 0.0000 + 0.0294 + 0.0441 + 0.0059 + 0.0765 + 0.0088 = 0.1647

Automatic categorization: undecided Expert categorization: 4 CS, 4 DL, 1 Both

Figure 1. Example for learning outcome processing from CSTA standards from 2011 [START_REF] Seehorn | CSTA K-12 Computer Science Standards: Revised 2011[END_REF] Table 2 summarises results of all possible combinations of different sets of the curricula used for building the dictionary for DL categorisation. The results show the fraction of matching categorisations. E.g., row five shows the approach achieves a match with expert ratings for 74% of all learning outcomes of CSTA 11 using BC and DK for the DL dictionary. For the categorisation of uniformly classified learning outcomes, this configuration matches in 94% of the DL learning outcomes, and 94% of all those learning outcomes. CSTA 11 does not contain uniformly classified CS learning outcomes, indicated with '---'. No single dictionary performs best for the categorisation of all analysed curricula. The best overall categorisation scores are achieved by DK for a single curriculum dictionary, with scores in the range .64 to .75 and a mean score of .70, and by the combination of BC and DK for a multi-curriculum dictionary, with scores in the range .68 to .74 and a mean score of .70. Notably, these two dictionaries perform best for two different sets of analysed curricula. Regarding categorisation of uniformly classified learning outcomes, again no single dictionary performs best. Measured with the sum of matching categorisation scores, the dictionary built with BC performs best for categorising CS learning outcomes and overall uniformly classified learning outcomes, with mean scores of .81 and .90, respectively. The dictionary built with all three curricula performs best for categorising DL learning outcomes, with a mean score of .99, mismatching one learning outcome.

Discussion

With the help of the experts' categorisation, it was possible to identify the foci of the selected educational models. For automated categorisation, the best performing sets of dictionaries with an accuracy of 70% are AIE for CS and DK for DL or the combination of DK and BC for DL. To identify the foci of the educational models, the numbers and fractions of categorised learning outcomes are presented in Table 3.

As can be seen for the Australian curriculum (AUS), the dictionary based on AIE/DK tends to identify a focus in DL (.68 compared to .32 in CS). Following the results of the dictionary based on AIE/BC, DK this curriculum is balanced (.41 for DL and .36 for CS). This balanced view corresponds to the results from the experts' choices. For the curriculum 21 from Switzerland, a clear focus on DL can be identified with both dictionaries having similar results (.77-.80 in favour of DL). This result corresponds to the experts' categorisation. A similar situation can be seen for the CSTA standards from 2011 where a focus for DL is visible (.72 in favour of DL). Here again, the results from the experts also indicate a focus on DL. Following the experts' results, the CSTA standards from 2017 tend to be balanced, which is also reflected by the semiautomated generated results (.39-.49 for CS and .44 for DL). Summarising, the semiautomated categorisation matches the experts' opinions in the identification of the focus for the majority of the analysed educational models. In three cases, both dictionaries of the semi-automated approach identified the same foci as the experts did. In one case, only the results from the dictionary based on AIE/BC, DK corresponded with those of the experts. An important conclusion is that the quality of categorisation is highly dependent on the curricula used for building the categorisation dictionary.

The semi-automated approach presented in this contribution shows a few threats to validity. The translation of the learning outcomes from the German-language curriculum can lead to the use of different terms. This can result in a lower frequency of important terms. Because all of the experts were from Austria, it can also be the case that they are biased by the local, well-known 'digikomp (DK)' competency model which was chosen to build the dictionaries. This could be a factor resulting in the higher performance of the dictionary based on DK. Another threat can be that expert categorisation can arise from misinterpretation, invalidating the comparison.

Conclusion and future work

Educational models are designed and implemented on different levels in many countries. These models include national curricula, workgroup recommendations, competency frameworks and guidelines. There is ongoing discussion about whether this newly implemented education trend should focus on topics related to the scientific subject of computer science or the development of skills and competencies needed in everyday life in the digital age. In this paper, we present an approach to semiautomatically categorise learning outcomes of computer science related curricula into computer science or digital literacy categories. For each of the categories, a dictionary of noun and verb terms of curricula representative for the category was built. The value of relative frequency of each term in all learning outcomes of the dictionary was used as a categorisation metric. The categorisation was applied to four computer science related curricula, and results were compared against classifications from nine experts of computer science teaching and research. The best performing dictionaries achieved a matching categorisation of 70% of all learning outcomes of the analysed curricula. Furthermore, for learning outcomes which were uniformly classified by the experts (at most one expert disagreed), the best performing dictionary achieved a matching categorisation of 90% of those learning outcomes. The results suggested that the focus of a curriculum regarding the two categories, computer science and digital literacy, can be identified with the application of the approach. Our goal for future work is an automatic classification of computer science related curricula and the individual learning outcomes regarding different categories. Going forward, we intend to take into account the verb and noun phrases for categorisation, following the general strategy of the experts. Additionally, we want to compare our approach of categorisation with machine learning classification and with sets of additional linguistic features. We also plan to evaluate them against a larger set of expert ratings, including additional experts.

Table 1 .

 1 Summary of the results from the experts' categorisation

		AUS	21	CSTA 11	CSTA 17
	Number of LOs	22	44	43	39
	CS	10	10	10	20
	CS (Strong Agreement)	7	3	0	9
	DL	11	32	32	18
	DL (Strong Agreement)	2	19	16	5
	Undecided	1	2	1	1

Table 2 .

 2 Comparison of results

	1 Curricula for	AUS, all	21, all	CSTA 11, all	CSTA 17, all
	DL dict.	Agreement	Agreement	Agreement	Agreement
	BC	.73	.70	.65	.64
		1.0 1.0 1.0	.86 .67 .89	.88 ---.88	.86 .78 1.0
	Dig	.73	.57	.67	.67
		.89 1.0 .50	.64 .67 .63	1.0 ---1.0	.71 .56 1.0
	DK	.73	.75	.70	.64
		.89 .86 1.0	.91 .67 .95	.94 ---.94	.79 .78 .80
	BC, Dig	.73	.64	.72	.64
		.89 1.0 .50	.73 .34 .79	1.0 ---1.0	.71 .56 1.0
	BC, DK	.68	.70	.74	.69
		.89 .86 1.0	.81 .67 .84	.94 ---.94	.86 .78 1.0
	Dig, DK	.73	.70	.67	.64
		.89 .86 1.0	.77 .67 .79	1.0 ---1.0	.71 .56 1.0
	Dig, BC, DK	.68	.75	.72	.67
		.89 .86 1.0	.91 .67 .95	1.0 ---1.0	.79 .67 1.0

Table 3 .

 3 Application of categorisation with two different dictionaries

			AUS		21		CSTA 11	CSTA 17
	CS	Experts	10	.45	10	.23	10	.23	20	.51
		AIE/DK	7	.32	7	.16	8	.19	19	.49
		AIE/BC, DK	8	.36	7	.16	9	.21	17	.44
	DL	Experts	11	.50	32	.73	32	.75	18	.46
		AIE/DK	15	.68	35	.80	31	.72	15	.39
		AIE/BC, DK	9	.41	34	.77	31	.72	17	.44
	Unde-	Experts	1	.05	2	.04	1	.02	1	.03
	cided	AIE/DK	0	.00	10	.23	4	.09	5	.12
		AIE/BC, DK	5	.23	7	.16	3	.07	5	.12

The header denotes analysed curricula. Results rounded to two decimal places use the format [All CS DL] to show overall relative match of categorisation and relative match for competencies with strong expert agreement. Best scores per column and category are marked bold

Acknowledgments. The authors would like to thank Günther Fliedl for his feedback on the natural language processing methods and on preliminary studies.


Paolo Tosato
email: ptosato@unive.it

Monica Banzato
email: banzato@unive.it

Exploratory Study on the Effort Perceived by In-service K-12 Teachers from Subject Areas not Specialising in Computer Science who are Complete CS Novices

Keywords: Teacher training, computer science education, self-efficacy, intrinsic motivation, perceived effort

Due to the shortage of IT teachers in Italian schools, the teaching of computational thinking is carried out by in-service K-12 teachers from scientific areas not specialised and by novices in computer science (CS). It is crucial to investigate not only the training of teachers in digital skills, but also how their beliefs, attitudes and behaviours can affect, in detail, their implementation in the classroom. From these premises, the present exploratory study investigates the self-efficacy beliefs, intrinsic motivation and perceived effort of a group of 46 teachers who, on a voluntary basis, engaged in a 20-hour workshop on CS teaching. The results show a significant improvement in self-efficacy, despite their perception of strong effort to master the subject.

Introduction

The educational policies of the last few years have been decisive in promoting the inclusion of computational thinking (hereafter CT) in the school curriculum in several European countries and the rest of the world. The aim is to prepare students for a world strongly influenced by information technology [START_REF]Committee for the Workshops on Computational Thinking: Report of a workshop of pedagogical aspects of computational thinking[END_REF][START_REF] Yadav | Expanding computer science education in schools: understanding teacher experiences and challenges[END_REF], encouraging in the new generations a culture of creation and production rather than the mere consumption of technology.

In Italy, CT has been introduced in primary and secondary schools since 2015 [START_REF]MIUR: Riforma del sistema nazionale di istruzione e formazione e delega per il riordino delle disposizioni legislative vigenti[END_REF]. This educational reform, albeit accompanied by a certain initial enthusiasm, nevertheless presents considerable difficulties for primary and secondary schools (hereafter K-12), among which are: the lack of teachers with sufficient knowledge of the subject as well as complete novices; educational activity in computer science (hereafter CS) carried out by non-specialist teachers or, in some cases, replaced by external IT experts; and insecurities on the part of teachers studying this new subject. A similar situation has been reported in other countries, including: the United States of America (USA) [START_REF] Yadav | Expanding computer science education in schools: understanding teacher experiences and challenges[END_REF], the United Kingdom (UK) [START_REF]The Royal Society: Shut down or restart? The way forward for computing in UK schools[END_REF], Israel [START_REF] Gal-Ezer | Computer science teacher preparation is critical[END_REF], and others (see section 2).

The reforms in Italy (as in other countries in the world) have to deal with the lack of CS teachers capable of covering the needs of the entire national school territory. The situation in Italian primary schools, which appears to be the most delicate, has the following peculiarities: 69% of teachers have only a high school diploma; the remaining ones have a higher degree mainly in the science of primary education; and 96.5% are women with an average age of 54.3 years [START_REF]OECD: Education at a Glance 2016: OECD Indicators[END_REF]. Nearly all of these teachers are complete novices in CS. Turning to Italian middle schools, we find that there are practically no teachers of CS due to a normative problem: information technology can be taught by engineers or architects; while science is taught by graduates in biological, geological, natural, environmental, agricultural, mathematical and physical sciences (also by some engineers). The average age of teachers in these schools is 53 years, of which 22% are male [START_REF]OECD: Education at a Glance 2016: OECD Indicators[END_REF]. The presence of CS teachers in the middle schools is not common (these teachers are specialist subject teachers but not specialised in CS); while CS graduates can teach mathematics, applied mathematics and CS in some high schools. As a result, almost all teachers who are involved in training in CS in middle schools are non-CS scientific area teachers. Even if there were to be a change in the current legislation, it seems a remote possibility that the few graduates in computer science in Italy would be attracted to work as teachers in K-12 schools, both because of the low salary remuneration compared to other school roles, and the high job insecurity in the teaching profession.

Unfortunately, at the moment, there is little research on this precise group of K-12 teachers working in scientific areas not specialised in CS (mainly present in middle schools) or novices at CS (present in primary schools). From research carried out in other countries (USA [START_REF] Yadav | Expanding computer science education in schools: understanding teacher experiences and challenges[END_REF], UK [START_REF]The Royal Society: Shut down or restart? The way forward for computing in UK schools[END_REF], Israel [START_REF] Gal-Ezer | Computer science teacher preparation is critical[END_REF]) there are numerous critical issues related to the training of K-12 teachers, also shared in Italy. These include fragmentation and discontinuity of training courses throughout the year, to which is added the heterogeneity of the background of teachers who take training for these courses (both the starting degree and the level of the school where they work). There are CS certification programmes for teachers, recognised by the Italian Ministry of Education, but they have "no tangible relationship with what you need to teach in a computer room" [START_REF] Gal-Ezer | Computer science teacher preparation is critical[END_REF]. Unfortunately, there is very little research literature on the beliefs about difficulties, obstacles and perceived efforts that teachers meet in this initial update phase, which requires shifting from information and communication technologies (ICT) to computer science education (see section 2). In Italy, as well, this type of analysis is still an unexplored territory. For this reason, the present exploratory survey aims to investigate the beliefs of self-efficacy, intrinsic motivation and perceived effort required for K-12 teachers involved in CT training courses. In particular, the research questions which we will try to answer are: 1) what are the self-efficacy beliefs of K-12 teachers in-service involved in programming workshop activities?; 2) what perception do K-12 teachers have regarding the intrinsic motivation and effort required and the skills involved in programming activities?; and 3) based on their training-workshop experience, would teachers be able to imagine themselves confidently performing similar classroom activities with their students? The objectives of the research are: [START_REF]Committee for the Workshops on Computational Thinking: Report of a workshop of pedagogical aspects of computational thinking[END_REF] to establish the factors that influence the self-efficacy and intrinsic motivation beliefs of K-12 teachers who are studying how to teach programming in the classroom; and (2) to determine the impact of the beliefs of K-12 teachers regarding their ability to teach programming in the classroom.

The exploratory survey was carried out in the course of a 20-hour programming laboratory for 46 K-12 teachers. Teachers' participation in the workshop was free and purely on a voluntary basis (no credits or scores were expected).

The Royal Society [START_REF]The Royal Society: Shut down or restart? The way forward for computing in UK schools[END_REF] has reported that "There are simply not enough teachers with sufficient subject knowledge and understanding to deliver a rigorous Computer Science and Information Technology curriculum in every school at present". In parallel, the European Commission has expressed serious concern that "digital literacy is taught mainly by specialist teachers at secondary level but in approximately 50% of countries it is also taught by other specialist teachers such as mathematics or science teachers" [START_REF]Eurydice: Key data on learning and innovation through ICT at school in Europe[END_REF]. "Information Technology is not taught by specialist teachers in Ireland, France, Italy, the Netherlands, Sweden, Liechtenstein and Norway -even at secondary level" [START_REF]The Royal Society: Shut down or restart? The way forward for computing in UK schools[END_REF]. Yadav et al. [START_REF] Yadav | Expanding computer science education in schools: understanding teacher experiences and challenges[END_REF] record a similar situation in the United States: "efforts to increase the number of CS teachers have predominantly focused on training teachers from other content areas". There have been several research inquiries that document the experience of computer scientists who teach other scientific subjects in schools. By contrast, there has been little study of the experiences of scientists not specialised in computer science who are asked to teach CS in K-12 schools. Also as pointed out in a number of studies [START_REF] Yadav | Expanding computer science education in schools: understanding teacher experiences and challenges[END_REF][START_REF]The Royal Society: Shut down or restart? The way forward for computing in UK schools[END_REF][START_REF] Gal-Ezer | Computer science teacher preparation is critical[END_REF], little is known about the difficulties of teachers with no scientific background who must be trained in CS in order to teach it in the classroom. Some studies [START_REF] Feiman-Nemser | A conceptual review of literature on new teacher induction: National partnership of excellence and accountability in teaching[END_REF][START_REF] Yadav | Expanding computer science education in schools: understanding teacher experiences and challenges[END_REF][START_REF] Veenman | Perceived problems of beginning teachers[END_REF] identify several obstacles which are faced by teachers in CS who teach outside their discipline, such as feeling isolated, teaching in multiple disciplinary areas, class management and insufficient planning time. Veenman [START_REF] Veenman | Perceived problems of beginning teachers[END_REF] identifies 24 crucial problems by means of a meta-analysis work of 83 studies. Among these are difficulties in evaluating student work, a heavy teaching load that leads to reduced planning time, problems in lesson planning, inadequate knowledge of the subject matter and insufficient assistance and support. Other studies report that novice teachers are faced with other challenges such as: loneliness, isolation [START_REF] Corley | First-year teachers: Strangers in strange lands[END_REF][START_REF] Yadav | Expanding computer science education in schools: understanding teacher experiences and challenges[END_REF] the lack of adequate IT background and limited resources for professional development [START_REF] Yadav | Expanding computer science education in schools: understanding teacher experiences and challenges[END_REF]. "The researchers argued that these teachers need support during the first years of teaching to increase their content and pedagogical knowledge, self-efficacy and beliefs about what it means to be a successful teacher" [START_REF] Yadav | Expanding computer science education in schools: understanding teacher experiences and challenges[END_REF]. This support should be extended as well to novices and scientists non-specialised in CS K-12 teachers, in both pedagogy and content, taking into account the different educational backgrounds (scientific and not) and the students' education level, as precisely these teachers are facing the first impact of the introduction of CS in K-12. Unfortunately, there is a lack of research documenting what is needed to respond to the needs of these teachers. For this reason, the objective of this exploratory study is to begin an investigation of the critical difficulties faced by K-12 teachers of CS with regard to their self-efficacy beliefs and their perceived effort. "The efficacy beliefs of teachers are related to their instructional practices and their students' achievement" [START_REF] Tschannen-Moran | Teacher efficacy: Its meaning and measure[END_REF].

Methods and instruments: self-efficacy, intrinsic motivation and perceived effort

Based on the research questions and objectives presented in section 1, the exploratory survey analysed the results of self-efficacy, intrinsic motivation and perceived effort, according to the following independent variables: gender, age, length of service, diploma or degree, level of education in which the teacher works, subject of teaching, and previous experience of teaching CS courses. The aim was to verify if these matters have a statistically significant impact on the dependent variables. For this reason, a programming workshop was organised, which was preceded and followed by the administration of: 1) pre-test and post-test questionnaires on the beliefs of self-efficacy and intrinsic motivation; and 2) a brief unstructured interview, post-laboratory, aimed at investigating the perceived effort.

The first questionnaire was based on the New General Self-Efficacy scale [START_REF] Chen | Validation of a new general self-efficacy scale[END_REF]. This instrument is dedicated to understanding the self-confidence of teachers regarding their learning of CS and their competence to master specific academic domains concerning programming in CS. Pajares [START_REF] Pajares | Teachers beliefs and educational research: Cleaning up a messy construct[END_REF] found that there was a "strong relationship between teachers' educational beliefs and their planning, instructional decisions, and classroom practices". The self-efficacy questionnaire was based on 8 items on a 5-step Likert scale: from 1, on the "completely disagree" pole, to 5, on the "completely in agreement" pole. The questionnaire investigated the following aspects: the level of self-efficacy in activities of teaching and learning about the subject; expectation of success in the CS workshop; mastery of the computer skills required; work commitment required; teachers' abilities in relation to the programming activities; the achievement of the educational objectives; andthe security of managing the evaluation of teaching activities with students.

The teachers' intrinsic motivation, based on a 5-step Likert scale (as with the previous questionnaire), was measured by a selection of 11 items from the Intrinsic Motivation Inventory (IMI) [START_REF] Ryan | Intrinsic and extrinsic motivations: Classic definitions and new directions[END_REF]. The original questionnaire consists of seven subscales. For this research, three subscales were selected: a) "interest" to teach programming: this refers to teachers' general intrinsic motivation; b) "perceived competence" to teach programming: this is theorised to be a positive predictor of intrinsic motivation; c) "perceived effort" required to teach programming: this is a separate variable which seeks to reveal teachers' needs. This instrument required teachers to imagine themselves teaching programming activities to students.

The third tool employed unstructured interviews aimed at exploring in depth teachers' perceptions of effort: 1) effort required for updating in CS and degree of satisfaction; 2) required effort for personal study of CS; 3) self-assessment of progress in one's CS competence level based on the teacher's experience; 4.1) if the teacher has already taught students programming, what are the perceived difficulties of educational activities in the subject?; and 4.2) if the teacher has not yet taught students programming, what are the expectations and when does he or she plan to start teaching? The study was conducted during a 20-hour teacher update workshop that aimed to explore and test elementary programming concepts with the following tools: Lego WeDo, Scratch and Rospino.

Research results

A total of 46 teachers, comprising 17 primary school teachers and 29 lower secondary school teachers, enrolled in this pilot study. The teachers were aged from 35 to 63 years (mean age 47 years). The teachers from scientific areas (mathematics, science and technology) were 34, while the teachers from humanities areas (Italian, history, religion, music, foreign language) were 12; the majority had never experienced programming laboratories (30 teachers), while 16 had participated in at least one laboratory on this topic. According to gender, the participants comprised 12 males and 34 females, while according to their academic degree, the majority of teachers were college graduates (32 participants) compared to high school graduated teachers (12 participants).

Results regarding changes in self-efficacy

To verify whether the educational activities had produced an improvement in the sense of self-efficacy, a t-test for dependent samples was performed on data from 46 teachers; this revealed that the change (Pre: M = 3.65, SD = 0.54, range from 2.14 to 5.00; Post: M = 4.16, SD = 0.56, range from 2.71 to 5.00) was statistically significant (t(45) = -5.14, p < 0.01). Subsequently, in order to answer the research questions, the data collected from the questionnaires were more deeply analysed to understand if the increment of self-efficacy was uniform in the group of participants or particularly relevant in a specific category of teachers. For this reason, some independent variables were identified (gender, age, years of service, educational qualification, programming experience, subject taught), and based on these variables, the impact of the laboratory on teachers' self-efficacy was assessed. Groups by Gender. Grouping data by gender importantly indicated how the change in the sense of self-efficacy was statistically significant in female teachers (t(33) = -4.93, p < 0.01), moving from a mean of 3.61 to a mean of 4.17. By contrast, the change in the sense of self-efficacy was not statistically significant in male teachers (p > 0.05), moving from a mean of 3.76 to a mean of 4.12. Groups by Age. To analyse the change in self-efficacy, 4 groups were created: from 30 to 39 years old, from 40 to 49 years old, from 50 to 59 years old and from 60 to 69 years old. The most numerous groups were those aged 40-49 years (24 teachers) and those aged 50-59 years (13 teachers). In these groups the increment of self-efficacy was greater than the other groups and was also statistically significant. A deeper analysis of these two groups (40-49 and 50-59 years old), found out that the increase in selfefficacy was particularly relevant and statistically significant in teachers who taught science subjects, were college graduated and were women. Groups by Years of Service. To analyse the change in self-efficacy, 5 groups were created: from 0 to 9 years, from 10 to 19 years, from 20 to 29 years, from 30 to 39 years and from 40 to 49 years. Confirming the results obtained by grouping data by age, teachers with 10-19 years of service obtained the more significant increase in selfefficacy, from a statistical point of view, moving from a mean of 3.74 to a mean of 4.24, t(23) = -3.39, p < 0.01. Groups by Educational Qualification. Grouping data according to teachers' degrees, it was possible to notice how the increase in self-efficacy was statistically significant both in college graduated teachers (p <0.01) and in high school graduated teachers (p <0.01). A deeper analysis of these groups revealed that the increase in self-efficacy was more significant in women and in teachers who taught science subjects.

Groups by Programming Experience. Grouping data according to programming experience, it was possible to notice how the increase in self-efficacy was statistically significant both in teachers with experience (p < 0.01) and in teachers with no experience (p < 0.01). A deeper analysis of these groups revealed that the increase of self-efficacy was more significant, from a statistical point of view, in female, college graduated teachers and in teachers who taught science subjects, according to the findings of the previous section. Groups by Subject. Grouping data according to the subject taught, it was possible to notice how the increase in self-efficacy was statistically significant both in teachers who taught humanities (p < 0.01) and teachers who taught science subjects (p < 0.01). As noted in previous sections, the increase in self-efficacy was greater and statistically significant in female and college graduated teachers.

Results regarding changes in intrinsic motivation

To verify whether the educational activities had produced an improvement in intrinsic motivation, a t-test for dependent samples was performed on data from 46 teachers; this revealed that the change (Pre: M = 4.04, SD = 0.41, range from 3.00 to 5.00; Post: M = 4.13, SD = 0.49, range from 3.00 to 5.00) was not statistically significant (t(45) = -1.72, p > 0.05). In order to answer the research questions, the data collected from the questionnaires were more deeply analysed, grouping by different variables. This made it possible to compare the results regarding intrinsic motivation with the results of selfefficacy and to understand if the change in intrinsic motivation was particularly relevant in a specific group of teachers. Groups by Age. Using the same groups identified for self-efficacy (section 4.1), the change in teachers' motivation was analysed. In these groups, the change of motivation was not statistically significant, but it is interesting to highlight that in teachers aged 30-39 and 60-69 years the motivation decreased, while in teachers aged 40-49 and 50-59 years it increased, moving from a mean of 4.07 to a mean of 4.18 (40-49 years old) and from a mean of 3.95 to a mean of 4.14 (50-59 years old). A deeper analysis revealed that the increase in intrinsic motivation was greater and statistically significant in teachers aged 40-49 with programming experience, moving from a mean of 4.26 to a mean of 4.60, p < 0.05. Nothing statistically significant can be said grouping teachers aged 40-49 years by gender or educational qualification, even if the increase in motivation was particularly important in female teachers (from a mean of 4.05 to a mean of 4.18, p = 0.077) Groups by Programming Experience. Grouping data according to programming experience, the increase in motivation was statistically significant only in teachers with experience (p < 0.05). Nothing statistically significant can be said grouping teachers with experience and without experience by gender. Instead, by grouping data according to their degree, there was a significant increase in motivation in college graduated teachers with experience, moving from a mean of 4.09 to a mean of 4.36, p = 0.066. Groups by Subject. Grouping data according to the subject taught, there was no statistically significant change in motivation, even if the change in teachers who taught science subjects was greater (from a mean of 4.08 to a mean of 4.21) than teachers who taught humanities (from a mean of 3.91 to a mean of 3.93). Nothing statistically significant can be said grouping teachers by gender or educational qualification. Instead, it is important to highlight the increase of motivation in teachers who taught science subjects with programming experience (from a mean of 4.13 to a mean of 4.38, p = 0.072), partly supporting the findings of the previous section.

Results regarding changes in perceived effort

To verify whether the educational activities had produced an improvement in perceived effort, a Wilcoxon test was performed on data from 46 teachers; this revealed that the change (Pre: Median = 4.00, Q1 = 3.50, Q3 = 4.50; Post: Median = 4.00, Q1 = 3.50, Q3 = 4.13) was not statistically significant (z = -0.94, p = 0.347). It is important to underline that the higher mean values indicate a lower perceived effort, therefore a better teacher response. As with self-efficacy (section 4.1) and intrinsic motivation (section 4.2), a deeper analysis for perceived effort was also performed, grouping data by different factors. Groups by Age. Using the same groups identified for self-efficacy (section 4.1) and intrinsic motivation (section 4.2), the change in teachers' effort was analysed. In any groups the change of effort was not statistically significant, but it is interesting to highlight that in teachers aged 40-49 and 50-59 years the perceived effort increased less than teachers aged 30-39 and 60-69 years. The values are respectively: teachers aged 30-39 years: from 4.00 to 3.67; teachers aged 40-49 years: from 4.04 to 3.98; teachers aged 50-59 years: from 4.00 to 3.96; teachers aged 60-69 years: from 3.83 to 3.67. Grouping data according to the years of service, the perceived effort increases, or remains the same, in all groups, except in teachers aged 40-49 and 50-59 years having 10-19 years of service. Although the change in effort is not statistically significant, in teachers aged 40-49 years with 10-19 years of service the perceived effort decreases from 4.00 to 4.04, while in teachers aged 50-59 years with 10-19 years of service the perceived effort decreases from 3.83 to 4.08. A similar result is obtained grouping data by programming experience. Groups by Programming Experience. Grouping data according to the programming experience, there is not a statistically significant change in perceived effort, although in teachers without experience the effort increases (from 3.92 to 3.75), while in teachers with experience the perceived effort decreases (from 4.19 to 4.22). A deeper analysis highlights that the reduction in perceived effort among teachers with experience is particularly relevant in college graduated teachers (from 4.05 to 4.25), while among high school graduated teachers the perceived effort increases both in teachers with experience and without experience in programming. Groups by Subject. Grouping data according to the subject taught, there is no statistically significant change in perceived effort. Grouping data by subject and programming experience, perceived effort increases in all groups, except among teachers who teach science subjects and have experience in programming (from 4.23 to 4.27). The change is not statistically significant, but it confirms what was highlighted in the previous section, where teachers were grouped according to programming experience.

Interview on motivation and perceived effort

The transcribed interviews were codified and on the basis of a subsequent reelaboration some categories were extrapolated to provide the conceptual structure that allowed us to select the results reported here. Whereas, in general, we have found a shared accord in the opinions of the teachers, we report the answers as cumulative percentages. From the analysis of the unstructured interviews, to which 96% voluntarily responded, it emerged that many teachers (80%) expressed a determined intention to master the subject and a willingness to continue learning, despite the difficulties that they were encountering. Nevertheless, 73% expressed uneasiness about the training on offer to them. Although characterised by many valid and interesting offers, they found it difficult to form a satisfactory overall picture of what was available to them. Another crucial point is that teachers asked for their commitment to training in this new discipline to be recognised by the schools. They believe that in the future they will be asked to teach their specialised subject and CS at the same time, and will have to keep up-to-date with developments in both of them. The teachers interviewed manifested a good ability: to analyse the commitment required (80%); to reflect on their self-assessment of progress in the discipline (70%); but they considered that they did not yet possess appropriate criteria for assessing the subject; to maintain a proactive attitude, advancing proposals and alternative solutions to existing training (80%); and to show willingness to assume the responsibilities for teaching CS at school (80%). Far fewer teachers expressed their strong disagreement with the current training (7%) or did not wish to continue because the subject was too difficult and complex (9%). Most teachers wondered how many years it would take to master the discipline. Ninety-one per cent expressed concern that they did not feel as expert in CS as in their own disciplines, which they had been teaching for years, and 36% believed that the students knew more than they did. The proposals advanced by 80%, mentioned above, were all oriented to informal learning and to classroom practice, such as: having opportunities to share and discuss problems and learning solutions with colleagues in CS who, unfortunately, are not present in their school; to observe a CS colleague when teaching programming lessons to students; and to start their own programming lessons in coteaching with a more experienced colleague. They expressed a desire: to analyse the various alternative ways of teaching CS (68%); to review their choices and solutions with colleagues in the subject (73%); and to discuss with expert colleagues the management strategies needed to face any difficulties (80%). The percentage of teachers who were aware of their strengths and weaknesses in the CS area was also high (95%). Many had shown the ability to predict the difficulties they might encounter in the implementation of CS with students in the classroom (66%), even if only a few (25%) were able to think positively about the possibility of failure and to recognise the typical causes of their own mistakes, due to lack of experience. Most of the novices in CS (93%) believed that they would not begin teaching programming in less than a year; while those who already taught it stressed that they did not feel expert in the subject and were proceeding very cautiously with small workshops with pupils. It was therefore too early for them to analyse the educational difficulties.

Conclusions and limitations of the research

From the results of the questionnaire, we can deduce that the course in programming completed by the researchers led to an increase in their sense of self-efficacy. To answer the first research question, we can state that the most significant improvement is in female teachers who were college graduated and taught science subjects. These groups of teachers also reported increased motivation and no worsening of perceived effort, which remained almost the same in pre-and post-tests (these improvements were particularly notable among teachers with programming experience). The groups that appeared to be the weakest were the younger and older teachers: this appeared to be a consequence of inexperience in the young and lack of energy in the old. However, we must not forget the problematic context that creates the difficulties and obstacles these teachers have to face (section 5). To answer the first two research questions, it is important to note that self-efficacy and intrinsic motivation have increased in some groups despite the fact that a high level of perceived effort was required. Moreover, in the interviews they demonstrated a high level of analysis, critical reflection and professional self-assessment in carefully evaluating their present and future actions. From their reflections, we conclude that immediate intervention measures are needed to support teachers who are facing genuine challenges. This is especially important in this delicate period in which the teaching of CS is being introduced in schools. "It has been demonstrated that students generally learn more from teachers with high selfefficacy than from those whose self-efficacy is low. In fact, teachers' beliefs in their instructional efficacy is a very strong predictor of academic attainment in young children" [START_REF] Mulholland | Teacher induction and elementary science teaching: Enhancing self-efficacy[END_REF]. Confirming other evidence reported by other authors [START_REF] Yadav | Expanding computer science education in schools: understanding teacher experiences and challenges[END_REF][START_REF] Gal-Ezer | Computer science teacher preparation is critical[END_REF][START_REF] Qian | Who Needs What: Recommendations for Designing Effective Online Professional Development for Computer Science Teachers[END_REF] in section 2, and answering the third question, these teachers demonstrated that: 1) their perceived effort increases especially because they have difficulty in re-elaborating the subject teaching on their own. The training courses are not intended to be established as an organised system of learning, but they are fragmentary and unrelated to each other; therefore, it is difficult for teachers to draw up a systematic overall picture of CS and its pedagogical aspects; 2) based on their teaching experience, they believe this "adventure" is absolutely new (it does not have anything similar to their past work) and unexpectedly puts into question their professional role (as some teachers have said: "At this moment, the children know more than we do"); and 3) the teachers have analysed their commitment as perceived by the school and believe that it does not have the deserved recognition because colleagues often confuse it with a training in general ICT, rather than with a specific disciplinary area, computer science education.

The principal limits of this exploratory study are: the selection of the group of participants is based on criteria of convenience (volunteer teachers participating in a training course); and is subject to bias. Finally, it should be observed that as these teachers are Italian, the problems identified by them may not be relevant in other contexts. This exploratory study has investigated only some characteristics and criticalities; however, if we take into account the multidimensionality of teacher professionalism and the constant pressure to which it is subjected (in this phase of digital innovation in the schools), it is necessary to expand and deepen the investigation to further dimensions.

Note: for reasons of national assessment of Italian university research, the authors must declare which sections each has written, in spite of the fact that work is entirely the result of continuous and intensive collaboration. Sections 1, 2, 3, 5 and 6 are by M. Banzato. Section 4 is by P. Tosato. Our thanks to Matthew Hoffman.


Amber Mcleod
email: amber.mcleod@monash.edu

Kelly Carabott
email: kelly.carabott@monash.edu

Who's teaching the Teachers? Viewing the ICT Content of a Teaching Degree through the Eyes of Pre-service Teachers

Keywords: Initial teacher education, digital competence, pre-service teachers, information and communication technology

The myth of the "digital native", pedagogical beliefs about ICT and its place in education, and the reality of a teacher as an ICT role model each contribute to the attitudes school students develop about ICT. All Australian teachers, regardless of discipline, are required to incorporate ICT in their lessons. The way pre-service teachers (PSTs) are educated has a direct impact on their ability and desire to teach digital competence to school students. Using 482 first year PSTs' experiences and expectations as a lens, teaching degrees at an Australian university were investigated, using a mixed methods approach, to find out whether the ICT content was appropriate to prepare graduate teachers to implement the national curriculum. Findings indicated that the teaching degrees did not meet all PSTs' needs. PSTs wanted more explicit instruction in the practical and pedagogical implications of using ICT in the classroom, and some even wanted training to navigate the university's online systems. These findings indicate that assumptions implicit in universities about digital competence may be invalid. Recommendations include suggestions that universities review their expectations of PST digital competence and consider including both embedded and explicit methods of teaching ICT in teaching degrees.

Introduction

In order to prepare students for the future, the current Foundation to Year 10 Australian Curriculum includes digital competence in two ways. The first is as a discipline areathe digital technologies strand -taught either as a separate subject or embedded across the curriculum. The second is as a general capability in information and communication technology (ICT) which is taught by all teachers regardless of their discipline areas. The ability of teachers to successfully implement the curriculum, however, is varied [START_REF] Selwyn | Left to their own devices: the everyday realities of "one-to-one" classrooms[END_REF] and this phenomenon appears to be worldwide [START_REF] Haydn | Lessons learned? Teaching student teachers to use ICT in their subject teaching: view from the UK[END_REF]. One problem is that institutions that deliver teaching degrees hold inaccurate assumptions about pre-service teacher (PST) digital competence and so do not cover this area properly. Upon graduation, these new teachers may be unable to teach school students the basic digital literacy expected in today's society.

In addition, to be effective, teachers require more than just the technical skills needed to deliver the curriculum. As digital technology advances at a rapid rate, it is not always possible to keep up with the latest developments and risk-taking skills need to be developed, leading to courage and a confident attitude when presented with new digital technologies. Teachers are socialisers who transmit their values, attitudes and priorities to their students regardless of the curriculum [START_REF] Whittle | The 'Perfect' Senior (VCE) Secondary Physical Education Teacher: Student Perceptions of Teacher-related Factors that Influence Academic Performance[END_REF]. Teaching degrees should build PSTs' digital competence to the point where they become positive ICT role models if the envisaged outcomes of the curriculum are to be achieved. This is particularly pertinent for female teachers, who make up 86% of primary school teachers and 62% of secondary school teachers in Australia, as females are underrepresented in the ICT field [START_REF]Australian Jobs 2017 -Occupation Matrix[END_REF]. Research in the field of initial teacher education in ICT has mainly focused on problems of practice, but, as highlighted by Tondeur, Roblin, Van Braak, Voogt and Prestridge [START_REF] Tondeur | Preparing pre-service teachers to integrate technology in education: A synthesis of qualitative evidence[END_REF], the links between graduate teacher digital competence and future success of their students remains an area in the literature which needs further review. The adequacy of one Australian university's teaching degrees' ICT content is investigated in this paper.

Literature review

Digital competence is more than knowing how to use computers, tablets and smartphones. Ferrari [START_REF] Ferrari | DIGCOMP: A framework for developing and understanding digital competence in Europe[END_REF] suggested that being digitally competent not only encompasses an understanding of technical operations, but should also include information management, collaboration, communication and sharing, creation of content and knowledge, ethics and responsibility, evaluation and problem solving. Mishra and Koehler [START_REF] Mishra | Technological pedagogical content knowledge: A framework for teacher knowledge[END_REF] proposed the Technological Pedagogical Content Knowledge (TPACK) model for effective incorporation of technology into teacher education. The model suggests that educators not only need Technological Knowledge, but an understanding of Pedagogical Knowledge and Content Knowledge (of both the ICT aspects of the curriculum and the discipline area) as well. In order to successfully integrate technology in education, there needs to be an intersection between these three knowledges. These researchers highlight the breadth of digital competence and the importance of understanding how ICT fits within every lesson, no matter what the discipline.

Teachers as role-models

Students are socialised into their attitudes about ICT by teachers, parents, peers, and the media [START_REF] Clayton | Gender stereotypes prevail in ICT: a research review[END_REF][START_REF]Multimedia Victoria: Reality bytes: An in depth analysis of attitudes about technology and career skills[END_REF]. Students themselves have reported that when making decisions about future studies or careers these same groups are the main influencers [START_REF] Roger | Factors underlying persistent gendered option choices in school science and technology in Scotland[END_REF]. Teachers affect the way students see themselves [START_REF] Margolis | Unlocking the clubhouse: Women in computing[END_REF]. Females, in particular, have been found to develop self-efficacy through vicarious modelling in their relationships with teachers [START_REF] Zeldin | A comparative study of the self-efficacy beliefs of successful men and women in mathematics, science, and technology careers[END_REF]. Encouraging, passionate teachers, and good student-teacher relationships have been reported as making students' sense of belonging to an ICT environment stronger [START_REF] Furrer | Sense of relatedness as a factor in children's academic engagement and performance[END_REF]. A good teacher can significantly improve a child's school performance [START_REF]Childhood Development: From New Directions to Action: World class teaching and school leadership[END_REF].

Butler [START_REF] Butler | Gender, girls, and computer technology: What's the status now?[END_REF] argued that teachers can give messages to girls, sometimes unconsciously, that they do not need to participate in digital technology. Teachers' stereotypical beliefs and attitudes about appropriate behaviour and roles for boys and girls, and technology, have been found to distort their perception of actual student abilities [START_REF] Eccles | Understanding women's educational and occupational choices[END_REF] and subtly steer girls away from ICT [START_REF] Barker | The state of research on girls and IT[END_REF]. Similarly, researchers have claimed that student performance can be predicted by examining teachers' expectations and beliefs about student ability [START_REF] Eccles | Families, schools, and developing achievement-related motivations and engagement[END_REF].

Teaching degrees and ICT as a cross curricular priority

While governments, the community and PSTs themselves expect that by participating in teaching degrees students will increase their digital competence, this does not seem to match with what is actually happening in universities [START_REF] Tondeur | Preparing beginning teachers for technology integration in education: ready for take-off?[END_REF]. A lack of preparation in terms of teacher education has been blamed for slowing the journey to digital competence [START_REF] Wang | An investigation of middle school science teachers and students use of technology inside and outside of classrooms: considering whether digital natives are more technology savvy than their teachers[END_REF][START_REF] Sweeney | How prepared are our pre-service teachers to integrate technology? A pilot study[END_REF]. This is problematic as teaching degrees are a vital motivator and contributor to future integration of ICT by PSTs [START_REF] Tondeur | Preparing pre-service teachers to integrate technology in education: A synthesis of qualitative evidence[END_REF]. In order to explore the complexities of preparing PSTs, there first needs to be an appreciation of their digital education prior to university, which for most undergraduates, is school.

The Australian National Assessment Program in Information and Communication Technology (NAP-ICT) results showed that only 52% of Year 10 students reached proficiency level and, notably, there was a statistically significant drop in digital competence since the last assessment across all cohorts of students [START_REF]National Assessment Program: National Assessment Program ICT Literacy, Years 6 & 10, Report 2014[END_REF]. It is unsurprising then, that Murray and Perez [START_REF] Murray | Unraveling the digital literacy paradox: How higher education fails at the fourth literacy[END_REF] found that 72% of the students in their university course could not be considered digitally competent. Studies into the ICT competence of first year university students suggested that they tended to use a limited range of technologies in ways which did not correspond with institutional expectations, with significant variations in digital competence across the university student body [START_REF] Elsden-Clifton | The Pre-Service Teachers' ICT toolkit: Expertise and Expectations[END_REF]. These results add weight to previous findings that so-called "digital natives" [START_REF] Prensky | Digital natives, digital immigrants[END_REF] do not share new ways of working and learning linked to ICT and have failed to achieve the digital competence levels expected [START_REF] Duncan-Howell | Digital mismatch: Expectations and realities of digital competency amongst pre-service education students[END_REF].

The importance of ICT in teacher education is recognised in the Australian Professional Standards for Teachers [START_REF]Australian professional standards for teachers[END_REF], which explicitly mention ICT competence. Graduate level teachers are expected to "implement teaching strategies for using ICT to expand curriculum learning opportunities for students", "Demonstrate knowledge of a range of resources, including ICT, that engage students in their learning", and "Demonstrate an understanding of the relevant issues and the strategies available to support the safe, responsible and ethical use of ICT in learning and teaching" (standard 4.5). The Professional Standards for Teachers form part of the criteria against which teacher education programmes are accredited.

PST education is directly related to school students' results [START_REF]Childhood Development: From New Directions to Action: World class teaching and school leadership[END_REF]. Clearly, as teachers are the ones teaching digital competence, their teacher education programmes should be preparing them for that [START_REF] Svensson | Teacher students' perceptions of their digital competence[END_REF][START_REF] De Silva Joyce | Investigating the literacy, numeracy and ICT demands of primary teacher education[END_REF]. Alarmingly, however, many teacher education degrees are not designed to have a strong influence on the technology use of PSTs and fail to explicitly address digital competence [START_REF] Banas | Authentic learning exercises as a means to influence preservice teachers' technology integration self-efficacy and intentions to integrate technology[END_REF][START_REF] Murray | Unraveling the digital literacy paradox: How higher education fails at the fourth literacy[END_REF].

Buabeng-Andoh [START_REF] Buabeng-Andoh | Factors influencing teachers' adoption and integration of information and communication technology into teaching: A review of the literature[END_REF] suggests that the lack of educational opportunities and support for developing ICT skills was one of the barriers to digital competence. While education degrees may include a single technology unit [START_REF] Brown | From the university to the elementary classroom: Students' experiences in learning to integrate technology in instruction[END_REF], the units are usually deemed insufficient for PSTs to be adequately prepared for the complexities involved in integrating ICT [START_REF] Lawless | Professional development in integrating technology into teaching and learning: Knowns, unknowns, and ways to pursue better questions and answers[END_REF]. Although at many universities the ICT requirements for accreditation have been achieved by embedding digital literacy across the curriculum [START_REF] Murray | Unraveling the digital literacy paradox: How higher education fails at the fourth literacy[END_REF], discipline units often demonstrate or require little to no technology integration [START_REF] Brown | From the university to the elementary classroom: Students' experiences in learning to integrate technology in instruction[END_REF]. Consequently, after graduation, PSTs may be unable to deliver the Australian Curriculum as envisaged.

Black and Smith [START_REF] Black | Hot Topic: ICT in pre-service teacher training[END_REF] found that when PSTs were asked how well they thought their lecturers in education modelled ICT in their units, only 26% thought it was done well with 9% indicating they did not think their lecturers had embedded ICT at all. Even where the focus had been on becoming skilled in using applications, little was done to help the students understand how to include the technology in their own teaching or facilitate subject learning [START_REF] Black | Hot Topic: ICT in pre-service teacher training[END_REF].

Methodology

For entry into teaching degrees at this Australian university, students must demonstrate they have achieved minimum levels of numeracy and English language competence [START_REF]Australian Council for Educational Research: Literacy and Numeracy Test for Initial Teacher Education[END_REF] and have particular personal and professional characteristics [37], which the teaching degree builds upon. Demonstration of a minimum level of digital competence is not required, and so the university has simply assumed a level of digital competence, which informs the ICT content of the degree.

Using PSTs as a lens, this study investigated whether the ICT content of teaching degrees in an Australian university was adequate. This was explored through two questions:

1. How digitally competent do PSTs believe they are, and does this indicate a minimum level which can be assumed by universities? 2. Do PSTs reflections indicate that the ICT content of their degrees are adequate?

This study was conducted using qualitative and quantitative methods interwoven in a mixed methods approach [START_REF] Creswell | Educational research: Planning, conducting, and evaluating quantitative and qualitative research[END_REF]. Ethics for the study were obtained through the Monash University Human Research Ethics Committee.

From 2015 to 2018, during O-week sessions promoting opt-in ICT classes, first year students who were enrolled in either a Bachelor of Education (Foundation-12), Bachelor of Education (Early Childhood/Primary), a Masters of Teaching (Primary/Secondary) or a Masters of Teaching (Primary) were invited to be part of this study. Students were asked to complete a survey that was available online or on paper, according to student preference. Completed surveys were returned to the researchers immediately after the one-hour information sessions. The survey included a question inviting students to take part in follow-up group interviews. Surveys were returned by 482 students (366 females, 110 males, 6 unknown) and three group interviews were conducted with 10 students.

The survey included questions addressing digital self-efficacy as well as confidence with, interest in and attitudes towards ICT based on a five-point Likert scale. The responses were coded as 1=Strongly disagree to 5=Strongly agree, or 1=Low to 5=High. One sample t-tests were used to determine whether attitudes were statistically significantly different to the middle value, "Unsure" or "Moderate". The p-value was set at .05 for significance [START_REF] Tabachnick | Using multivariate statistics[END_REF].

Group interviews were conducted towards the end of the first semester and explored some of the themes which emerged from the quantitative data. The sessions were audio taped and the tapes were later transcribed for analysis. Thematic analysis of the qualitative data were conducted using Nvivo.

Results and discussion

How digitally competent do PSTs believe they are, and does this indicate a minimum level which can be assumed by universities?

From the quantitative data, one sample t-test results indicated that for all but one statement (If something goes wrong with digital technologies I panic, M=3.15, which is statistically significantly different to 3, "unsure") PSTs were, on average, positive about digital technologies and their own digital competence. However, as every teacher is required to competently deliver the ICT components of the curriculum, a measure of the average response is not enough. It is important to gauge the percentage of responses indicating negative attitudes or lack of self-efficacy in order to investigate the minimum level of digital competence that can be assumed. The lower two response categories and the higher two response categories for the questions have been combined and presented in Table 1. 1 shows large differences in PSTs' evaluation of their own digital competence, and attitudes towards ICT. Only 41.4% of students communicated that they had high or moderately high digital technology skills with 12.1% of students actually selfidentifying as having low skills. Answers indicating negative attitudes or low selfconfidence ranged from 4.2% to 44.2%. This suggests that an emphasis on ICT components of teaching degrees would be beneficial for a sizeable minority of PSTs.

Do PSTs' reflections indicate that the ICT content of their degrees are adequate?

Towards the end of the first semester, PSTs had some experience of the ICT content and expectations of the degree. While some PSTs interviewed were clearly very digitally competent, others appeared to have trouble coping and were concerned that the course content did not match their expectations. A thematic analysis of the qualitative data were conducted and four key themes emerged as discussed below:

1) Not all students have the digital self-efficacy assumed The university's expectation of students is that they have the digital competence to engage in the risk-taking associated with navigating new digital technologies such as the university's online enrolment and learning management systems. As summarised earlier in Table 1, at least 12% of students surveyed rated themselves as having low skills, found it difficult to teach themselves new programs, felt nervous when they had to learn something new, did not feel they could fix problems and panicked if something went wrong, and would avoid using digital technologies if they could.

These sentiments were echoed by participants in the group interview who indicated that they had wasted a considerable amount of time learning to navigate the online systems, which they found very stressful. They believed the university should run introductory sessions for the students, suggesting that they would attend weekend sessions or even pay for a workshop. Students also expressed concerns that the changes in digital technologies were so fast they found them overwhelming. They did not feel able to sort out which new developments they should focus their energy on and wanted more instruction from teacher educators. Some participants also revealed how little experience they had with ICT, finding the jump in expectations from high to university surprising. As teachers model behaviours to students, and if teachers do not have the confidence to engage with new technologies, this will be picked up by their students [START_REF] Clayton | Engaging girls in ICTs: Mind the gap![END_REF].

2) Digital technologies need to be explicitly modelled by lecturers, but also played with PSTs signalled that competence with digital technologies was not something that came naturally and needed to be worked on. They compared it to learning a new language that had to be studied and practiced, because not only is it a new concept, but they also have to learn how to use it and when and why it is appropriate. In addition, PSTs found learning on their own was not enjoyable and were looking for alternative ways to gain this knowledge. They believed that use of ICT must be modelled and explicitly demonstrated by the teacher educators and that unless the PSTs themselves were able to play with the technologies in a meaningful way in the workshops, they were unlikely to adopt them for their own classes. If PSTs were left to learn to use digital technologies on their own, they found it a chore. This reflects findings shown in Table 1 where around 16.2% of students indicated that they could not understand how people can become so involved with digital technologies.

Comments from students also suggested that unless the pedagogical implications of programs that they had seen modelled in lectures and tutorials were explicitly discussed, students had difficulty imagining how programs could be adapted to be used in other disciplines, year levels, or other educational contexts.

3) Digital technology education is an extra

As shown Table 1, the results of the questionnaire showed that 11% of students were unsure or did not agree that it was important to use digital technologies for learning. In addition, 36.6% of students believed schools gave a low or moderate priority to computer education. This was reflected in comments from the group interviews where postgraduate students in particular suggested that they believed that their own education, which did not include ICT, was sufficient and that digital technologies were not a requirement of education, rather an add-on to keep students interested. They indicated that they thought of digital technologies as an optional extra that could be used as a tool if the teacher chose to do so. While they conceded that digital competence was important, they believed that students could learn this outside of school, as indeed they had done themselves, albeit painfully. Students also expressed concern that digital technologies in schools were getting in the way of a "proper" education and that they would make classroom management more difficult. These attitudes do not reflect the Australian Curriculum, and if modelled to school students, could result in school graduates without the level of digital competence envisaged. At one extreme the curriculum would be subverted and at a lesser level it would be compromised.

Conclusion

The results of this study indicate that while the current structure of the teaching degrees at this university may be adequate for the majority of year students, university assumptions have meant that the way digital competence is taught in teacher education programmes may not address the needs of all students, as approximately 12% of students have self-identified as having low self-efficacy or negative attitudes towards ICT. While this percentage may not seem large, imagine if 12% of PSTs graduated without the literacy or numeracy skills expected.

The qualitative and quantitative data collected for this study suggest a variety of perceived abilities and a range of attitudes towards ICT among PSTs. This indicates that universities need to carefully consider their assumptions about the minimum level of digital competence PSTs have. Worryingly, at this university, some PSTs struggled to access and understand basic university systems. When instruction is required at this level, it is unlikely that these PSTs will gain a comprehensive understanding of the technological and pedagogical knowledge required to incorporate ICT into their teaching without extensive explicit instruction.

These results show that students' concerns are, perhaps, not presently being addressed in this university's teaching degrees and some students could benefit from, and are actively asking for, more ICT content and opportunities to play with digital technologies as part of their degree. It is clear from their comments and concerns that PSTs felt the pedagogical implications of using technology in classrooms had not yet been adequately addressed. While it must be kept in mind that these were first year PSTs who had only participated in one semester of their degrees, for many, their expectations were not being met. If teaching degrees are to truly prepare PSTs to deliver the envisaged Australian Curriculum, then instruction in ICT needs to happen throughout their degree. Existing assumptions about the way PSTs become digitally competent need to be re-examined and degrees restructured to reflect this, otherwise graduate teachers may not be able to use ICT across the curriculum to support the digital competence of their own students.

Table 1 .

 1 Percentage of student responses for each response category.

	Statement	% Low	% Moderate	% High
	Please indicate what you think is
	the priority given to computer	5.6	31.0	63.3
	education within schools
	How would you rate your skills with digital technologies?	12.1	46.6	41.4
	How would you rate your
	enjoyment of using digital	4.5	27.1	68.4
	technologies?
	How would you rate your
	enjoyment of using digital	7.3	31.0	61.7
	technologies in classrooms?
	If something goes wrong with digital technologies I panic.	33.4	22.3	44.2