Annariina Koivu

Nicholas Mavengere

Mikko J Ruohonen
email: mikko.j.ruohonen@uta.fi

Lucy Hederman

Jane Grimson
email: jane.grimson@tcd.ienicholas.mavengere

Keywords: ICT skills of health professionals, low-and middle-income countries, scoping review, South Africa, case study, training needs

Information is at the heart of healthcare because all stakeholders need fit-for-purpose information to make decisions. However, producing and utilizing information in the data-intensive and ever-changing health environment requires various skills. In the particular context of low-and middle-income countries, this study, consisting of a scoping review and a qualitative case study, explores the information and ICT skills of health professionals. Our review identifies challenges in several areas of health professionals' skills, including computer skills; skills required for using the routine health information system; data security skills; and data management and analysis skills. Our South African case study, based on interviews, adds a more nuanced understanding of the different types of training needs. This assessment shows that training and education aimed at improving the ICT and information skills of health professionals have to be versatile and cater to different groups with varying needs.

Exploring the Information and ICT Skills of Health Professionals in Low-and Middle-Income Countries 1 Introduction

Health professionals in low-and middle-income countries (LMIC) in the 21st century are facing new challenges as a result of the increased need to be able to collect, capture, report and use data alongside the care delivery duties. Besides clinical skills, an ability to understand and analyze data and, increasingly, digital literacy and Information and Communication Technology (ICT) skills are in high demand. The significance of information in healthcare is by now well recognized in public and global health. For instance, the World Health Organization's (WHO) framework for strengthening health systems identifies six building blocks of a health system [START_REF]Everybody's business: strengthening health systems to improve health outcomes: WHO's framework for action[END_REF]. "Health information" is one of the building blocks, but "production, analysis, dissemination and use of reliable and timely information" also informs decisionmaking in each of the other building blocks, namely, health workforce; health services; health financing; governance and leadership; and medical products, vaccines and technologies [START_REF]Everybody's business: strengthening health systems to improve health outcomes: WHO's framework for action[END_REF][START_REF] Nutley | Improving Data Use in Decision Making: An Intervention to Strengthen Health Systems[END_REF]. In other words, high-quality data is essential in health care because it is required basically at every corner of the health system from the clinical decision making in the case of individual patients to the monitoring and evaluating (M&E) of health programs, surveillance of diseases, planning of services and resources, informing policy-making, for global reporting purposes and ultimately, improving population health [START_REF]Guiding Principles for National Health and Social Care Data Collections[END_REF].

In the LMIC context, it is usually the doctors and nurses that are the source of such data, since they are often responsible for collecting and recording it [START_REF] Koivu | Key Challenges in the Current TB and HIV Information System in South Africa. A Case Study in Khayelitsha[END_REF]. Unfortunately, research suggests that in the developing world well-trained health professionals are unevenly distributed as well as insufficient in aggregate numbers, and there are serious challenges in their information and ICT skills [START_REF] Detmer | Capacity Building in e-Health and Health Informatics: A Review of the Global Vision and Informatics Educational Initiatives of the American Medical Informatics Association[END_REF]. This is no wonder, since even in high-income settings various cadres of health professionals struggle to keep up with technical developments in the ever-changing health environment.

The need for systematic assessments focusing on health information systems (HIS) with a view to identifying underlying factors affecting performance has been noted in the literature [START_REF] Ledikwe | Improving the quality of health information: a qualitative assessment of data management and reporting systems in Botswana[END_REF]. Addressing challenges in the health professionals' capacities related to information and ICT is vital for several reasons. First, issues related to human resources are a significant challenge for maintaining data quality in HIS, as has been found in many LMIC countries (e.g., [START_REF] Ledikwe | Improving the quality of health information: a qualitative assessment of data management and reporting systems in Botswana[END_REF][START_REF] Makombe | Assessing the quality of data aggregated by the antiretroviral treatment clinics in Malawi[END_REF]). Second, it is important to understand and appreciate the rationale of health information systems (i.e., why the data is collected and reported) and the importance of the quality of data, because these are likely to affect the extent to which the collected data is used [START_REF] Koivu | Key Challenges in the Current TB and HIV Information System in South Africa. A Case Study in Khayelitsha[END_REF]. Health information should be utilized in various ways from health interventions to program evaluation, but a poor use of data in LMIC settings has often been reported (for instance, [START_REF] Braa | Improving quality and use of data through data-use workshops: Zanzibar, United Republic of Tanzania[END_REF]). It is therefore critical to ensure the sufficient skill levels and understanding of those who provide the data and those who are supposed to use it in decision-making. Finally, health professionals' information and ICT skills have been linked to the utilization of ICT tools and eHealth [START_REF] Olok | Knowledge and attitudes of doctors towards ehealth use in healthcare delivery in government and private hospitals in Northern Uganda: a cross-sectional study[END_REF] and the readiness to use electronic medical record systems (EMR) [START_REF] Biruk | Health Professionals' readiness to implement electronic medical record system at three hospitals in Ethiopia: a cross sectional study[END_REF].

In order to identify the key issues regarding the information and ICT skills of health professionals we conducted a scoping review on the topic. A scoping review is a specific type of review which can provide a snapshot of the existing literature, as well as background information for further inquiry [START_REF] Armstrong | Cochrane Update. 'Scoping the scope' of a cochrane review[END_REF]. Specifically, we focused on limitations in health professionals' information and ICT skills, i.e., the skills required to collect, report, understand and utilize data required in the health systems in lowand middle-income countries (LMIC). In addition to the review, a qualitative case study was conducted in South Africa. For the case study we interviewed South African health professionals whose duties included working with health data.

For the purpose of this study, "health professional" refers to a range of people working with health information in the health system. In other words, it includes the frontline health workers, such as nurses and doctors, but also data capturers and clerks working in the clinics, and it refers to managers and other health officials at the different levels of the health system whose tasks require the utilization of health information for resource allocation and better services. As noted by previous LMIC research [START_REF] Ledikwe | Improving the quality of health information: a qualitative assessment of data management and reporting systems in Botswana[END_REF], the M&E and data management responsibilities are not always clearly assigned at field level in a low-resource context, and the responsibility for certain data may be assigned to an M&E officer in one place and a community health nurse or program officer in another place.

Over recent decades, the quality of healthcare has become increasingly connected to the systematic application of information processing; thus, a health worker needs to be skillful in the efficient use of the available information systems (IS), technologies and tools [START_REF] Haux | Transformation of health care through innovative use of information technology: challenges for health and medical informatics education[END_REF][START_REF] Irinoye | Nurses' perception and barriers to use of information communication technology in a teaching hospital in Nigeria[END_REF]. Therefore, this study seeks to explore the skill levels of health professionals in relation to information and ICTs and to propose measures to address challenges related to this.

Methodology

In order to map the existing evidence base regarding our research topic, we conducted a scoping review. Such reviews are an increasingly accepted methodology for reviewing health research evidence [START_REF] Levac | Scoping studies: advancing the methodology[END_REF], and they have been successfully applied in a variety of fields relevant to this study, such as assessing e-health programs in LMIC [START_REF] Piette | Impacts of e-health on the outcomes of care in low-and middle-income countries: where do we go from here?[END_REF]. Additionally, further characteristics of scoping reviews support this approach: the scoping process benefits from an analytical reinterpretation of the literature, while it does not usually require assessing the quality of the studies included, and it may be especially relevant in disciplines with emerging evidence. Furthermore, scoping reviews allow the incorporation of a variety of evidence and study designs and may produce novel findings [START_REF] Levac | Scoping studies: advancing the methodology[END_REF]. Two researchers (AK and NM) conducted searches of electronic databases in January and February 2016, including PubMed and Academic Search Complete (all databases) from the year 2000 to the present. Given the interconnected ways in which the terms "skills" or "information" are employed in health-related literature and their relationship to other multi-dimensional concepts, such as "data quality", "health system", "health workers" and so on, we kept the search terms broad to warrant a sufficiently wide coverage of the topic. These initial searches yielded many non-relevant papers, so an additional search string was included to increase the relevancy of the results, and this returned 519 articles that were reviewed by title or abstract. The database searches were complemented with additional searches on relevant organizational websites, reference lists of key papers and hand searching of journals online. We limited our search to full-text studies published in English and focusing on LMICs. Our review identified 18 studies that captured the key ideas relevant to the research question [START_REF] Mcdougall | Reviewing Literature in Bioethics Research: Increasing Rigour in Non-Systematic Reviews[END_REF].

In addition to the scoping review, we interviewed a number of key informants involved with HIS in under-privileged settings in South Africa. This component of the study was conducted in 2012 by AK and was approved by the Trinity College Research Ethics Committee (in Dublin, Ireland) and the Western Cape Provincial Research Health Committee (in South Africa), as well as by the local health authorities. More specifically, the interview data was obtained from 28 informants, who worked in various roles at four levels (clinic, sub-district, district and provincial level) of the Western Cape public health system. Since each informant was in a role that involved collecting, producing, recording, reporting or using health data, they could inform us on how they perceive challenges in the HIS, including the human aspect of it. The semi-structured interviews were conducted face-to-face, predominantly at the place of work of each informant, as 18 individual interviews and 4 group interviews with 2 to 3 informants. The informants included females and males, as well as people of black, coloured and white origin. Not all informants were asked the same questions. Some of the informants did not focus primarily on the information skills of the health professionals, whereas others raised that as a key issue. Nevertheless, they all provided rich data relevant to the research question.

Information and ICT Skills of Health Professionals -Literature Review

Studies that have examined the health professionals' role in health information systems in an LMIC context have generally focused on the availability of human resources rather than on competence [START_REF] Nicol | Human factors affecting the quality of routinely collected data in South Africa[END_REF]. Furthermore, while the importance of human capacity building for the efficient functioning of various HIS in underprivileged settings is long acknowledged in academia [START_REF] Luna | Health Informatics in Developing Countries: Going beyond Pilot Practices to Sustainable Implementations: A Review of the Current Challenges[END_REF], most studies merely mention the challenges in this area as a matter of course. In line with that notion, our review found a limited number of relevant articles on limitations in the information and ICT skills of health professionals.

Computer Skills

Despite the obvious benefits of ICTs to healthcare, their adoption has been problematic and the rates of use have been limited, particularly in many resourcerestricted settings. Two large-scale surveys have been conducted in Ethiopia recently. In Addis Ababa hospitals, out of 270 participants, 91 (34%) showed an adequate knowledge of computers, while 108 (40%) had fair knowledge and 71(26%) inadequate knowledge [START_REF] Mohammed | Knowledge and utilization of computer among health workers in Addis Ababa hospitals, Ethiopia: computer literacy in the health sector[END_REF]. In northwest Ethiopia, 356 (59%) of the 606 health professionals studied were computer literate [START_REF] Biruk | Health Professionals' readiness to implement electronic medical record system at three hospitals in Ethiopia: a cross sectional study[END_REF]. A smaller-scale assessment in Kenya found that approximately half (58 out of 121) of the health workers had no knowledge on how to use computers, which contributes to further limitations in data analysis and utilization skills [START_REF] Kiilu | Human resource capacity for information management in selected public healthcare facilities in Meru County, Kenya[END_REF]. Studies in Iran [START_REF] Habibi-Koolaee | Nurses readiness and electronic health records[END_REF], Nigeria [START_REF] Irinoye | Nurses' perception and barriers to use of information communication technology in a teaching hospital in Nigeria[END_REF] and India [START_REF] Raja | An exploratory study to assess the computer knowledge, attitude and skill among nurses in health care setting of a selected hospital in Ludhiana, Punjab, India[END_REF] have found variation in nurses' computer skills, the Iranian study reporting the mean of 43% of computer skills based on the International Computer Driving License (ICDL) [START_REF] Habibi-Koolaee | Nurses readiness and electronic health records[END_REF]. In Nigeria, approximately one fifth of the 180 nurses studied rated themselves as "novices" in information technology and claimed to possess little or no skill in the use of computers, with 68 (38%) maintaining that they had never had formal training in information technology [START_REF] Irinoye | Nurses' perception and barriers to use of information communication technology in a teaching hospital in Nigeria[END_REF]. Similarly, an earlier Nigerian study [START_REF] Bello | Knowledge and utilization of Information Technology among health care professionals and students in Ile-Ife, Nigeria: a case study of a university teaching hospital[END_REF] reports that approximately one fifth (22%) of the surveyed 148 health record officers, medical students and doctors showed poor knowledge of computers, while a majority (59%) had average knowledge and 19% demonstrated a good knowledge. Conversely, in Uganda, 82% of the 68 investigated self-reported being good or very good at using PCs, but reported significantly lower skill levels in using fax, printers or specific software, such as Microsoft Excel [START_REF] Olok | Knowledge and attitudes of doctors towards ehealth use in healthcare delivery in government and private hospitals in Northern Uganda: a cross-sectional study[END_REF]. The authors suspect that the selfreported figures reflect the doctors' confidence and some understanding of ICT, but that the level of skills demonstrated in the study may nevertheless form a significant barrier to the use of ICT.

Skills and Understanding of the Routine Health Information System

While it is important to possess a set of basic skills to be able to use computers in general, sometimes it is a specific health information system that health professionals need to work with. These systems may be routine health information systems (RHIS) that "provide information at regular intervals of a year or less through mechanisms designed to meet predictable information needs" [START_REF] Hotchkiss | How can routine health information systems improve health systems functioning in low-resource settings? Assessing the evidence base[END_REF]. In Benin, a data quality assessment including a health worker survey (n=116) noted that the health professionals were inadequately qualified, as few had been trained on RHIS and the training received had not always been fit for purpose [START_REF] Glèlè Ahanhanzo | Factors associated with data quality in the routine health information system of Benin[END_REF]. Kenya established their RHIS in 1984; however, 55% (67) of health workers had no knowledge about it [START_REF] Kiilu | Human resource capacity for information management in selected public healthcare facilities in Meru County, Kenya[END_REF]. In South Africa, researchers call the lack of knowledge about the RHIS rationale "disturbing" [START_REF] Nicol | Human factors affecting the quality of routinely collected data in South Africa[END_REF]. Since the average knowledge of the RHIS rationale was 22%, this suggests a limited understanding of the significance of the health data collected. Correspondingly, more than half of the surveyed 161 health professionals claimed that they had not received RHIS training in the last six months prior to the study [START_REF] Nicol | Human factors affecting the quality of routinely collected data in South Africa[END_REF]. There is a need to allocate, train and support management and end users in RHIS use in South Africa [START_REF] Nicol | Routine Health Information Systems in South Africa -Opportunities for Improvement[END_REF].

Data Security Skills

A further category which is connected to the daily use of computers and the RHIS are data security skills. Confidentiality and data security, amongst other issues, are seen as an important part of any health record system both in the developed and the developing world, but literacy rates and cultural differences may influence the application and relevance of these concepts in LMIC [START_REF] Were | Ethics of implementing Electronic Health Records in developing countries: points to consider[END_REF]. A Botswanan [START_REF] Ledikwe | Improving the quality of health information: a qualitative assessment of data management and reporting systems in Botswana[END_REF] and an international study [START_REF] Forster | and the ART-LINC Collaboration of the International Epidemiological Databases To Evaluate AIDS: Electronic medical record systems, data quality and loss to follow-up: survey of antiretroviral therapy programmes in resourcelimited settings[END_REF] report a lack of standard practices for backing-up the data, storing source documents and safeguarding confidentiality. Health data may not be backed up, and if it is, memory sticks are a customary method, which may result in the loss of data [START_REF] Ledikwe | Improving the quality of health information: a qualitative assessment of data management and reporting systems in Botswana[END_REF].

Data Management Skills and Analysis Skills

The ability to use a computer or a RHIS is closely connected to the ability to manage, manipulate, validate and analyze data. Hence, these overlapping categories are not distinguished as separate entities in all studies, though they may not necessarily be the same thing, either. Besides the health professionals' computer knowledge, the aforementioned Kenyan study [START_REF] Kiilu | Human resource capacity for information management in selected public healthcare facilities in Meru County, Kenya[END_REF] also evaluated the health workers' ability to perform basic information management functions. The study revealed that the majority were not able to perform such tasks: only 17% of the 121 health professionals could check for data accuracy, compute trends from bar charts or convert the collected data into meaningful information for use. This discovery is consistent with their finding that 91% had never received training for information management [START_REF] Kiilu | Human resource capacity for information management in selected public healthcare facilities in Meru County, Kenya[END_REF]. These figures show no progress compared to earlier evidence from a 15-country study: 13 out of 21 LMIC sites studied included personnel trained in data management or data quality control [START_REF] Forster | and the ART-LINC Collaboration of the International Epidemiological Databases To Evaluate AIDS: Electronic medical record systems, data quality and loss to follow-up: survey of antiretroviral therapy programmes in resourcelimited settings[END_REF]. In South Africa, 64% of the 161 health professionals had poor numerical skills as regards being able to calculate percentages, amongst other things. Although the average confidence levels at performing routine tasks related to health information was 69%, only 22% actually displayed a competence above 50%. Furthermore, the average ability to check data quality was 36% [START_REF] Nicol | Human factors affecting the quality of routinely collected data in South Africa[END_REF]. Similar challenges have been experienced elsewhere [START_REF] Ledikwe | Improving the quality of health information: a qualitative assessment of data management and reporting systems in Botswana[END_REF], [START_REF] Mbondo | Organizational HIV monitoring and evaluation capacity rapid needs assessment: the case of Kenya[END_REF][START_REF] Garrib | An evaluation of the District Health Information System in rural South Africa[END_REF][START_REF] Kintu | Development of HMIS in poor countries: Uganda as a case study[END_REF]. A study on the Kenyan HIV information system describes the staff's ability to check data quality, undertake basic data analysis and interpretation, and utilize the data for decision making as "limited" [START_REF] Mbondo | Organizational HIV monitoring and evaluation capacity rapid needs assessment: the case of Kenya[END_REF]. Likewise, in Botswana, most monitoring and evaluation (M&E) personnel at the national and district levels had received basic training in M&E, but their analytical skills were reported to be notably weak [START_REF] Ledikwe | Improving the quality of health information: a qualitative assessment of data management and reporting systems in Botswana[END_REF].

Empirical Findings

In addition to the scoping review, interview data was collected in South Africa in connection to a case study. There are ways to measure the actual skill and confidence levels of health professionals for information tasks, but such assessments were not made within this study. Rather, the empirical component of this study reports issues that are perceived as challenges by South African health professionals whose tasks include working with health data.

The first finding that was not revealed by the scoping review, but which feeds into all the further limitations in information and ICT skills is the issue of language skills. For a large proportion of the population in South Africa, the language barrier continues to compromise the quality of and access to healthcare [START_REF] Hussay | The language barrier: The overlooked challenge to equitable healthcare[END_REF]. The data of the current study suggests that such challenges may occur at every level, but the issue is particularly pressing at the facility level: a lot of time is spent "getting information out of people" (Informant, field level). In the clinic and sub-district investigated in the case study, most patients speak Xhosa as their first language and therefore recruitment at the facility and sub-district levels has been done locally. However, all the forms, registers and information systems that the staff is required to use, are in English.

A second finding, which did not emerge directly in the evidence reviewed but which is nevertheless consistent with the limitations reported in the literature, relates to the fact that a sizable number of South Africans still lack an adequate basic education. Amongst them are individuals who work in the health system or with a close connection to it (such as tuberculosis community workers). They may be providing lifesaving care, but still have rather limited basic mathematical, reading and writing skills. According to the informants, this may create challenges in understanding very basic concepts.

"… a lot of people don't even have proper mathematical literacy skills. … a lot of people are struggling with words … so how can they interpret data?" (Informant, Province) This reinforces the message from research from elsewhere in South Africa [START_REF] Nicol | Human factors affecting the quality of routinely collected data in South Africa[END_REF] and can be considered an underlying factor that contributes to further limitations in ICT and information skills.

Not surprisingly, several informants indicated that poor computer literacy prevails at the hospitals and clinics.

"Most people who work in the hospitals are not computer literate when it comes to Microsoft Office … … Most of them can't work with pivot-tables, or most of them can't work well with Excel …" (Informant, M&E position)

While the majority of the facility managers do have email, it was reported that they are not always confident about using it or other modes of electronic communication. Whilst most challenges were reported at the clinic level, it was also maintained that the higher-level health authorities have difficulties in understanding and utilizing technology.

Therefore, many agreed that while further computerization would be beneficial in terms of data quality and in reducing reporting duties, it should not be attempted until staff are a lot more information savvy. Weaknesses in data security skills were also reported. According to the informants, facility staff members use their own memory sticks on the computers, introducing viruses. Since many frontline health workers in under-privileged settings do not have computers at home, we assume that when they need to, they may be inclined to use the computer at work for non-work activities.

In the setting of our case study it was not possible to make a distinction between the skills and understanding of the RHIS and the overall data management and analysis skills. However, we did find challenges in this area. An informant argued that the utilization of data is limited by the lack of skills required to manipulate the data so as to make it easily usable for decision-making.

"[People cannot] … do the manipulation to get whatever information you want. So, it is a headache in that sense and the managers are not prioritizing this dataset … … because they can't get anything out." (Informant, M&E position)

We also found that concepts that help in interpreting data might be unfamiliar. This is consistent with the South African studies reviewed [START_REF] Nicol | Human factors affecting the quality of routinely collected data in South Africa[END_REF], [START_REF] Garrib | An evaluation of the District Health Information System in rural South Africa[END_REF]. In our case study setting, shortcomings in this area became particularly evident in the context of the feedback loop. Informants described how, after attending meetings, coordinators feedback the key messages from the printouts in such a way that the lowest-level staff and community workers can understand the trends. This requires that the concepts that communicate the information in ratios or visually are also explained. For instance, a field-level informant reported: "Sometimes they don't understand this percentage thing. So, I have to explain to them". Or: "Graphs gotta be very basic, bar graphs, you can't go for pies or anything just bar graphs, just plain and simple" (Another informant) and even the bars need to be fully explained: "This bar means this. This bar means that. What we are looking for is to see this bar going up".

Finally, we also identified a lack of epidemiological or clinical understanding of the data elements and indicators required. Those whose role was within a single health program (such as management of the antiretroviral medication (ART) program for HIV) were keen to point out a difference to those who mainly work with general health information:

" This is consistent with the results from another South African case study [START_REF] Kawonga | Administrative integration of vertical HIV monitoring and evaluation into health systems: a case study from South Africa[END_REF]. The researchers assessed the degree to which general health service (horizontal) managers exercise authority over the HIV program's M&E function, namely, HIV data collection, collation, analysis and use. They found that compared to vertical managers, i.e., managers working in a disease-specific (HIV) position, horizontal managers had lower HIV M&E knowledge, and were more likely to produce HIV data but less likely to use it [START_REF] Kawonga | Administrative integration of vertical HIV monitoring and evaluation into health systems: a case study from South Africa[END_REF].

Discussion

Our study illustrates that there are various weaknesses in health workers' skills in an LMIC context and it is vital to address them, whether they relate to the ability to use computers in general or to the RHIS concerned or to capabilities to convert that data into meaningful information. Therefore, our study reinforces earlier evidence of a need to identify effective solutions and simple interventions to address the human competence and other underlying factors in order to improve data quality and to strengthen the health systems [START_REF] Ledikwe | Improving the quality of health information: a qualitative assessment of data management and reporting systems in Botswana[END_REF].

How to address the weaknesses identified? A global vision to develop informatics has been proposed by the American Medical Informatics Association. It recommends a system-wide approach where education and training should be offered at the leader/policy level (for instance, in the form of seminars); to health professionals (degree and certificate programs; short courses); and ICT technology and support level (degree and certificate programs; short courses) [START_REF] Detmer | Capacity Building in e-Health and Health Informatics: A Review of the Global Vision and Informatics Educational Initiatives of the American Medical Informatics Association[END_REF], thus echoing other studies that have highlighted the importance of educational strategies, curricula development and tertiary training programs to improve the level of informatics knowledge, skills, and attitudes in both formally and informally trained health professionals [START_REF] Ledikwe | Improving the quality of health information: a qualitative assessment of data management and reporting systems in Botswana[END_REF], [START_REF] Luna | Health Informatics in Developing Countries: Going beyond Pilot Practices to Sustainable Implementations: A Review of the Current Challenges[END_REF].

However, this approach should be complemented with a decentralization of mentoring and training initiatives from the national to the district levels for the health work force delivering care at the facility level [START_REF] Ledikwe | Improving the quality of health information: a qualitative assessment of data management and reporting systems in Botswana[END_REF]. On-the-job training and mentoring and short interventions have yielded positive results in LMICs [START_REF] Ledikwe | Establishing a health information workforce: innovation for low-and middle-income countries[END_REF].

Conclusions

Healthcare is an ever-evolving and profoundly information-based discipline. It is vital that health professionals have sufficient capabilities to respond to the increasingly complex information needs. These needs may be particularly pressing in settings where the resources are already thinly spread. Our findings reinforce the message from previous research [START_REF] Detmer | Capacity Building in e-Health and Health Informatics: A Review of the Global Vision and Informatics Educational Initiatives of the American Medical Informatics Association[END_REF]: a basic-level information/ICT training is essential for all practicing health professionals, and more effort should be invested in it, especially at the field level, but also at other levels of the health system. What our study adds is a more nuanced understanding of the different types of training needs that exist. Moreover, it highlights the challenges related to language and basic literacy skills that undermine any improvement efforts, unless addressed first. Therefore, the training and education aimed at improving the ICT and information skills of the health professionals have to be versatile and cater to varying groups with differing needs, and approaches should be adapted to the local context. It is only then that the benefits of the information collected can be utilized to strengthen the health system.

  they don't pick up the errors, so they are not Program people; they're just punching numbers ... because for us from Programs you can very easily spot something that looks funny ... ... whereas someone who's just punching number; if they punch 1, 2, 3 … 1, 2, 3, 48 they won't necessarily think about the 48, whereas you would immediately realize that, that 48 can't be." (Informant in Program Management)


Nicholas Mavengere
email: nicholas.mavengere@uta.fi

Mikko Ruohonen
email: mikko.j.ruohonen@uta.fi

Digital Pedagogy for Enhanced Social Qualities, Collaborative Processes and Quality of Learning

Keywords: Digital pedagogy, social qualities, collaborative processes, quality of learning

The best learning environment that enables excellence should always be sought. This research seeks to draw measures to promote virtual learning experience based on promotion of social interaction and collaborative processes. The research is based on a total virtual learning experience of a masters' level information and communication technology for development (ICT4D) class at the University of Tampere. A questionnaire was conducted at the end of the course to assess social qualities, collaborative processes and qualities of learning. The research seeks to promote quality learning by developing a social and collaborative learning environment. The results of this study included measures, such as, pedagogical techniques and technological tools that could foster such an environment.

Introduction

There are increasing opportunities for information technology (IT) applications in education. Chou and Liu [START_REF] Chou | Learning effectiveness in a Web-based virtual learning environment: a learner control perspective[END_REF] noted the significant transformation in the learning and teaching process because of the web-based technology. Digital pedagogy involves the use of technology in enhancing the learning process. The value of digital pedagogy is derived from attempting to fully enjoy the benefits of technology in learning. The advantages of technology in adoption for learning could be summarized as "learning any time" and "learning anywhere" and thus self-control, independent thinking, diverse viewpoints and diffuse thinking models [START_REF] Chou | Learning effectiveness in a Web-based virtual learning environment: a learner control perspective[END_REF].

As much as we appreciate the advantages of technology in learning as presented above, there are also disadvantages that we should put efforts to minimize. Disadvantages, such as, possibility of students feeling isolated thus leading to anxiety and confusion thus reducing learning effectiveness [START_REF] Chou | Learning effectiveness in a Web-based virtual learning environment: a learner control perspective[END_REF]. Hence, as we adopt learning technologies we must try to maximize the benefits gained and reduce the negative impacts [START_REF] Mavengere | Using Open Source Software for Improving Dialog in Computer Science Education -Case Mozambique University[END_REF]. Digital pedagogy plays a crucial role in doing that. Spiro [START_REF] Spiro | Defining Digital Pedagogy[END_REF] defined digital pedagogy as "engaged and reflexive practice and scholarship of teaching and learning through digital technologies". Typical features of digital pedagogy include [START_REF] Spiro | Defining Digital Pedagogy[END_REF]:

• Combining theory and practice, making and thinking

• Fostering creativity, play and problem solving • Encouraging participation, collaboration and public engagement • Aiming to increase critical understanding of digital environment

In this research, we seek to promote virtual learning by making use of digital pedagogy that enhance social qualities, collaborative processes and quality of learning. Kreijns et al. [START_REF] Kreijns | Measuring perceived sociability of computer-supported collaborative learning environments[END_REF] advocated for sociability in virtual environments for enriched learning experience. Sociability is defined as the extent to which a virtual environment is "perceived to be able to facilitate the emergence of a sound social space with attributes as trust and belonging, a strong sense of community, and good working relationships" [START_REF] Kreijns | Measuring perceived sociability of computer-supported collaborative learning environments[END_REF]. This would foster collaboration processes and qualities of learning. Thus, the research question is: how could digital pedagogy promote social qualities and collaborative processes for learning quality?

In the next section the theoretical background is elaborated. After that, the methodology is specified then the results are noted. The discussion section follows and finally conclusion.

Theoretical Background

Kreijns et al. [START_REF] Kreijns | Measuring perceived sociability of computer-supported collaborative learning environments[END_REF] argue the need to consider and integrate social factors for successful virtual learning. Disregarding social psychological processes, such as, starting groups and sustainable social relationships could lead to failure of virtual learning. In fact, it is essential to promote social interaction in virtual learning environment, in that it reduces loneliness and isolation. As shown in Figure 1, Kreijns et al. [4, p. 180] suggest that sociability influences social interaction, "the greater the sociability of an environment, the more likely it is that social interaction will take place and that it will result in the emergence of a sound social space". They defined social presence as "the perceived degree of illusion that the other in the communication appears to be a real physical person in either an immediate (i.e., real time or synchronous) or a delayed (i.e., time-deferred or asynchronous) communication episode". Pedagogical techniques depending on the contents of the course also promote social presence and interaction.

We should always strive for the best. There are fundamental aspects of learning, referred to as quality learning which we should always seek to achieve. These are the cornerstones which technology could aid to build upon to foster the learning process. The following qualities of learning are suggested [5, p. 293];

1. Active -Learners' role in learning process is active; they are engaged in mindful processing of information and they are responsible for the result. These learning qualities were evaluated in the survey, which was conducted in the ICT4D course. This was done in order to adopt technological tools and pedagogical practices that promote the above-mentioned qualities of learning. Please see methodology section below for more information about the survey. We suggest that digital pedagogy supported by social interaction and collaborative processes could create a learning environment that is conducive for enhancing qualities of learning as shown in Figure 2. The qualities of learning reinforce the environment that is social and collaborative.

Illeris [START_REF] Illeris | How we learn: Learning and Non-learning in school and beyond[END_REF] suggested the three dimensions of learning and competence development as content, incentive and interaction as shown in Figure 3. Fig. 3. The three dimensions of learning and competence development [START_REF] Illeris | How we learn: Learning and Non-learning in school and beyond[END_REF] Content dimension is what is learned, for example, knowledge and skills. Incentive dimension is source of the mental energy that is essential for the learning process, such as, motivation and emotions. These two dimensions are initiated and cemented by the interaction process. These dimensions of learning are an elaboration of the social and collaborative learning environment shown in Figure 2.

Methodology

The University of Tampere hosted an international virtual course with students from Finland, Germany, and South Africa. The course topic was Development 2.0, that is, Information Communications Technologies for Development 2.0. The course comprised of 33 participants who had diverse virtual learning experience as illustrated in Figure 4. About 50% of the participants had 5 or more times of virtual learning experience. Number of times in a virtual class (100% or partially virtual)

Number of times in a virtual class (100% or partially virtual)

Fig. 4. Virtual learning experience

A survey was conducted at the middle and end of the course. The objectives in conducting the survey included assessing the virtual learning experience for the class and adopting measures, that is, pedagogical practices and technology tools to enhance virtual learning. This was done to improve the learning process the next time the course is offered.

Research Findings / Results

The study seeks to highlight how pedagogy in a virtual environment could be enforced to promote social qualities and collaborative processes for quality learning. In this section, we highlight results from the ICT4D course. These results are context dependent [START_REF] Mavengere | Pursuit of Quality Learning: Context and User Needs in Virtual Learning[END_REF] in that the reflections of the participants could have been strongly influenced by factors, such as, study topic but nevertheless there are lessons that could be drawn that are useful for the wider audience interested in virtual learning.

Social Qualities

The measures of social qualities namely, loneliness, good impression of others and spontaneous informal conversation in the online course environment reflects a moderate satisfaction as shown in Figure 5. Although a significant percentage of participants, 37.5% highlighted that they feel loneliness in the course environment. The interaction enabled by the course environment measures, these are, course environment enabled comfort, non-task related conversion and friendship development were deemed to be not applicable by the course participants as shown in Figure 6. This is especially so for the perceived friendship development enabler by the course environment in which 68.8% of the course participants perceived it not applicable.

Collaborative Processes

Figure 7 shows that the collaborative processes were not well supported by the course environment. Collaborative processes measures, such as, course environment as enabler of team development and good work relationship among participants had 37.5% and 50% respectively which were the highest at second applicability.

Collaborative processes

Course environment as an enabler of team development

Course environment as enabler of good work relationship among participants

Course environment enabled participants to identify themselves with other participants Fig. 7. Collaborative processes

Quality of Learning

The qualities of learning proposed by Ruokamo and Pohjolainen [START_REF] Ruokamo | Pedagogical Principles for Evaluation of Hypermedia-Based Learning Environments in Mathematics[END_REF] were also evaluated at the middle and end of the course as shown in Figure 8. Some qualities of learning were perceived to have increased from middle to end of course, such as, active, collaborative and reflective learning. However, some qualities, such as, collaborative, intentional, conversational, contextualized and transfer learning were perceived to have decreased.

Discussion

The use of technology to promote learning is not new, for example, in the 1990s Leidner and Jarvenpaa [START_REF] Leidner | The Use of Information Technology to Enhance Management School Education: A Theoretical View[END_REF] advocated for technology inclusion to improve business management school. However, the advance in technology calls for constant research to improve how technology is incorporated in learning. Thus, this research seeks to highlight digital pedagogy that promote social qualities, collaborative processes and quality of learning in virtual learning. However, we would also bring to light the potential confliction between social qualities promotion and learning outcomes. That is to say, as we achieve social aspects this could limit or loose focus from the desired learning objectives. Therefore, there has to be an analysis of promoting social qualities that are in line with learning process and thus delivering positive learning outcomes.

Digital pedagogy, such as, embracing techniques that promote social qualities and collaboration like online open learning diaries could be used to create a social and collaborative learning environment. Open learning diary is a reflective essay, which students share with the rest of the participants in a virtual learning environment. This promotes interaction as students comment and discuss experiences. The value of interaction in learning is well emphasized, both in research and in practice [START_REF] Beauchamp | The Influence of ICT on the Interactivity of Teaching[END_REF]. In doing so, collaborative processes are enabled from this sharing of background knowledge of fellow participants. Technological tools, such as, wikis, blogs and discussion forums could also foster the desired social and collaborative learning environment. For example, in the development 2.0 course students were grouped into teams of 3 or 4 participants and each team wrote a page summary of a course book and together developed the book wiki. This wiki development fostered social interaction and collaborative processes with the aim of promoting qualities of learning.

Conclusion

Excellence should always be sought in learning. In this research, this excellence has been defined based on the qualities of learning, namely, active constructive, collaborative, intentional, contextual, transfer and reflective learning. In addition, technology advances offer another avenue in promoting learning. To do so, digital pedagogy has to be set to promote the desired factors that promote qualities of learning. In this research, a social and collaborative learning environment is proposed as the conducive environment to be fostered as a platform for quality learning promotion.

The research seeks to promote quality learning by developing a social and collaborative learning environment. A conceptual model is developed in this research to highlight how qualities of learning are enabled in a social and collaborative learning environment. In practice, in the case study conducted, there were diverse results (some positive and some negative) in the qualities of learning pointing to the fact that efforts need to be made to specifically try and improve each of the qualities of learning, namely, active constructive, collaborative, intentional, contextual, transfer and reflective learning.

2 .

 2 Constructive -Learners construct new knowledge on the basis of their previous knowledge. 3. Collaborative -Learners work together in building new knowledge in cooperation with each other and exploiting each other's skills. 4. Intentional -Learners try actively and willingly to achieve a cognitive objective. 5. Contextual -Learning tasks are situated in a meaningful real world tasks or they are introduced through case-based or problem-based real life examples. 6. Transfer -Learners are able to transfer learning from the situations and contexts, where learning has taken place and use their knowledge in other situations. 7. Reflective -Learners articulate what they have learned and reflect on the processes and decisions entailed by the process.

Fig. 1 .

 1 Fig. 1. Relationships between the variables sociability, social presence, pedagogical techniques, social interaction, and social space. Adapted from [4].

Fig. 2 .

 2 Fig. 2. Qualities of learning enabled in a social and collaborative learning environment tentative conceptual model

Fig. 5 .Fig. 6 .

 56 Fig. 5. Social qualities in the online environment

Fig. 8 .

 8 Fig. 8. Qualities of learning evaluation


Jaana Holvikivi
email: jaana.holvikivi@metropolia.fi

Minna Lakkala
email: minna.lakkala@helsinki.fi

Hanni Muukkonen
email: hanni.muukkonen@helsinki.fi

Introducing Collaborative Practices to Undergraduate Studies

Keywords: ICT education, collaborative practices, professional competences, project-based learning

The changes in software industry and software development methods call for appropriate teaching methods in academia. In addition to theoretical knowledge and coding practice, familiarity with common practices in the industry is expected from the graduates. Teamwork, collaboration and communication skills are essential demands for software engineers. These skills take years to develop, and therefore, this study presents how collaborative practices were introduced right in the beginning of information technology studies. The results of project based courses were encouraging in terms of student achievements and course completion rates. Additionally, feedback from students through an extensive survey was largely positive.

Introduction

Software industry and software development practices have undergone tremendous changes in the last decades. These changes have not entered software engineering education in academia with the same force. Much of higher education still depends on very traditional teaching practices and conventional curricula. Several attempts to bring real-life engineering practices into education have been made, most notably the CDIO initiative by a number of technical universities worldwide [START_REF] Crawley | Rethinking engineering education. The CDIO Approach[END_REF] and the Aalborg Project Based Learning model [START_REF] Dalsgaard | Innovative application of a new PBL model to interdisciplinary and intercultural projects[END_REF]. Even though the theoretical backgrounds of these efforts differ, the practical implementation usually follows similar patterns on applying a project development cycle to course work, and concentrating on a problem and its solution instead of listed curriculum items (contents, skills, competences).

The Helsinki Metropolia University of Applied Sciences has a long history of undertaking curriculum changes to address a severe problem of low retention in information technology and engineering studies [START_REF] Schrey-Niemenmaa | The Metropolia Adaptation of CDIO in All 20 of its Engineering Study Programmes[END_REF]. As long as the university funding from the government depended largely on the number of students, the motivation for implementing changes was low but when the funding scheme was changed to be based on study completions and graduations, the dilemma had to be addressed seriously three years ago [START_REF] Valmu | Significantly improved student progression results by means of course integration and collaborative pedagogy in the degree programmes of electronics, electrical engineering and automation technology of the Helsinki Metropolia UAS[END_REF]. Therefore, the university appointed a team of teachers in various disciplines to explore educational innovations in other European universities.

The Aalborg model as an overall solution was chosen as the starting point for curriculum reform, which was implemented for the 2014-15 study year. The general guideline for the whole university encompassed larger study modules (5-15 ECTS credits), flexible curriculum choices as well as problem and project based learning methods.

Currently, the education at the university aims at developing the knowledge, skills, ethics, communication, and emotional component of the professional expertise to meet the need for highly integrated competence in the present day working environments. Dialogue with companies has revealed that education has failed to fully respond to the new requirements in the software industry. Demands from companies increasingly stress abilities for collaboration, efficient teamwork and professional communication. Therefore, project-based methods were included into most modules in the new curriculum. Additionally, the concepts of progressive inquiry [START_REF] Muukkonen | Exploring metaskills of knowledge-creating inquiry in higher education[END_REF] and problem-based learning were applied in course design. Moreover, participatory online learning environments were used to support the learner's development of understanding through collaborative construction of a shared product [START_REF] Paavola | Trialogical approach for knowledge creation[END_REF].

In information technology education, the first study year was divided into four 15 ECTS modules. Each module had a theme that introduced different major options of the studies: networks, programming and web-development, electronic devices and object-oriented programming. The project work in each module was supported by a varying amount of basic and theoretical studies such as mathematics.

The aim was to change studies in a way that makes entering the information technology profession a natural and exciting process regardless of the student background. In this paper, we discuss the results of applying working life practices immediately in the beginning of the studies.

Educational Practices at Metropolia

The studies in IT were previously loosely structured. In the old curriculum, students had 8 small courses in the fall semester that were partially studied in a large group (50 or 90 students). The incoming groups started with short courses in mathematics, physics, and the basics of information technology or media technology. Most courses were from 3 to 5 ECTS credits, and they consisted of lectures to a large audience and laboratory practice for groups of 24 students. If students failed courses, they could retake them during the following academic years. Unfortunately, this freedom and independence did not result in good retention, on the contrary: nearly 40% of students interrupted their studies already after the first year and only 40-49% graduated in 5 years (see Table 1). Interrupting studies is quite common in Finland where higher education is free. However, the high dropout rate became costly for the university, causing losses of income from the government when the funding depended on ECTS credits and graduates. In the reform 2014, the curriculum was swapped around to include team projects and product development already in the beginning, whereas mathematics and physics were taught all along the studies. This was very different from the previous curriculum where first year was devoted mainly to basic scientific studies, languages, and other rudimentary skills. The drop-out after first year before the reform was regularly around 40%, as many students could not cope with the amount of physics and electronics that was required, and they lost interest in the profession.

The majority of students in the undergraduate programmes discussed here come directly from high school or after their military service year. In August each year, about 210 Finnish students, and 50 international students who study in English begin their studies. The majority of students, around 85%, are male. Currently, about a half of the students in the international group come from Vietnam, and 20% from Nepal. Others come from various countries, especially form Eastern Europe. Their former high school studies have followed a very traditional mode of teaching, including lectures, home work and examinations [START_REF] Holvikivi | From theory to practice: adapting the engineering approach[END_REF].

Opposed to the previous organization of studies, the new curriculum was based on large 8 week modules that take full time. Each class of 25 to 30 students had one fixed classroom where they had all lessons except certain laboratory sessions that needed special equipment. The classrooms were furnished with small tables that could easily be rearranged. There were also movable white boards and connections for laptop chargers [START_REF] Hjort | Student Collaboration and Independence from Day One in Higher Education[END_REF][START_REF] Lukkarinen | Creation of a collaborative study community in engineering studies[END_REF].

Research Methods

The Metropolia University of Applied Sciences conducted this educational reform in a very short time and with a loose outline. Therefore, the implementations of the new curriculum were quite diverse in different departments. The follow-up of results requires collection of a big amount of data and various evaluation methods. Some of the data are presented in this study.

The outcomes of the studies were measured by several means. Most obvious measurement was the study completion rate. The statistics were gathered in four successive years, of which two represented the previous curriculum and two the new curriculum. Student and teacher views on learning of collaborative knowledge work practices were collected using a validated questionnaire.

The Collaborative Knowledge Work Practices Questionnaire (CKP) is designed for investigating students' self-reported evaluation of collaborative working practices and competence development in courses [START_REF] Muukkonen | Assessment of competencies in knowledge work and object-bound collaboration during higher education courses[END_REF]. In particular, aspects of collaboration and the use of digital technology are targeted as central components in modern knowledge work. These include learning to collaborate on shared objects (e.g., reports, products, designs), iterative development through feedback, integrating individual and collaborative working, understanding various disciplines and practices, interdisciplinary collaboration and communication, and learning to exploit technology. The questionnaire is based on the theoretical framework of the trialogical approach on learning that emphasises collaborative knowledge creation [START_REF] Paavola | Trialogical learning. The roles and uses of design principles in a project on trialogical learning[END_REF].

The questionnaire was available online and had 27 statements on Likert scale plus three open questions. The 5-point statements were of the type: "During the course/study unit I have learned … e.g. -to develop products collaboratively by using technology." The answer choices were: "Not at all (1); Just a little; Somewhat; Quite a lot; Very much (5)".

The questionnaire was answered by students and teachers in 13 modules of the new curriculum, including 24 teachers and 198 students who started the study in the year 2014/15. Students' and teachers' scores on the seven scales of the CKP questionnaire were compared statistically (t-test). The teachers answered the same questionnaire items with the instruction "In my opinion, students learned during the course … e.g., to define sub-goals for the collaborative work".

Moreover, qualitative methods were included in the evaluation of the results, such as the analysis of open-ended questionnaire responses, student work results, and teacher and student interviews that help in explaining statistical results. The readiness for teamwork and other basic abilities were mainly evaluated based on classroom performance.

Results

Retention Rates

The initial passing rate of the 15 ECTS modules of the new curriculum was 85-100%. More importantly, the retention after first year was equally high, practically all students continuing their studies in the second year. [START_REF] Vesikivi | Adoption of a New Project -Based Learning (PBL) Curriculum in Information Technology[END_REF] Table 2 The ECTS credit accumulation was investigated by examining students who stayed on track with their studies, completing the expected 30 credits in the two study periods of the first semester (Tables 2 and3). As can be seen from the figures, the amount of students with at least 30 ECTS increased from 48% in 2013 to over 80% in the years 2014 and 2015 in the Finnish groups. The positive effect of the new curriculum model was even stronger in the international groups, where earlier completion rates were between 17% and 31% (Table 3). Equally, other degree programmes in engineering reveal a similar pattern. The Electrical Engineering and Automation Technology results after the first semester show that the new curriculum significantly improved results, and more than 93% of the students in these degree programmes passed all the courses in time compared to the relatively low values of 70% in 2013 and 50% in 2012 [START_REF] Valmu | Significantly improved student progression results by means of course integration and collaborative pedagogy in the degree programmes of electronics, electrical engineering and automation technology of the Helsinki Metropolia UAS[END_REF].

In addition to a good passing rate, course grades were also exceptionally high, especially in the international groups where the average grade was 4.0 (of 5), Table 4. A comparison between international and Finnish students reveals that the international students were more ambitious and targeted at good grades, which they also achieved. The grade average in the Finnish groups in the spring term 2015 was 3.4 (of max. 5).

Questionnaire Answers

Student and teacher opinions on the learning in the new modules were collected using the same questionnaire after each module. The responses of students and teachers from 13 modules were compared on each of the seven scales of the CKP questionnaire (Fig. 1). There were no statistical differences between the scale scores. Students and teachers gave the highest scores to learning about collaborating on shared objects (e.g., plans, designs, reports), integrating efforts in collaborative working, development through feedback, persistent efforts, and exploiting technology. Lower scores were given to understanding about various disciplines and practices as well as learning about interdisciplinary collaboration and communication. • Organization (28): Good entity (15), Appropriate phase of work [START_REF] Holvikivi | From theory to practice: adapting the engineering approach[END_REF], Good integration of subjects ( 6);

• Facilities ( 4): Good study premises and equipment (4).

The negative aspects were partially the same, or sometimes the other side of the coin (180 mentions in general):

• Organization (70): Tight timetable and heavy workload (24), Uneven workload ( 13), Confusing and incomplete arrangements [START_REF] Vesikivi | Adoption of a New Project -Based Learning (PBL) Curriculum in Information Technology[END_REF], Poor integration of subjects ( 12), Poor communication practices (9); • Working methods (60): Problems in group work (18), Too little teaching of theory [START_REF] Markkanen | The Knowledge Practices Environment: a Virtual Environment for Collaborative Knowledge Creation and Work around Shared Artefacts[END_REF], Challenging working methods [START_REF] Muukkonen | Assessment of competencies in knowledge work and object-bound collaboration during higher education courses[END_REF], Restless classroom (5), Too much group work [START_REF] Holvikivi | From theory to practice: adapting the engineering approach[END_REF], Too much teaching of theory ( 4 As was noted before the arrangements of the modules were varied, therefore some aspects such as amount of theory or guidance could be very different between modules.

Adoption of Teamwork

According to student interviews in the classroom, none of the international students had previous experience in teamwork or team projects. However, the transition to university studies seemed to make the students open to a new way of learning, and all approached the teamwork mode positively. As contrasted to students who had previous university studies particularly in Finland, they adapted quickly to selfdirected project work. Only students who had started university studies elsewhere and transferred to our university, found the new mode difficult. Many of them had problems in adapting to regular attendance and shared responsibility of teamwork. Interestingly, even one year of previous studies had fixed their study habits.

Teamwork and project work were more challenging to assess fairly than individual deliverables. Teams of six or seven students were problematic in this respect but unfortunately the lack of equipment forced to do some projects in such large teams. Most projects were executed in teams of four. Instructors tried to avoid unfair situations by choosing half of the deliverables as individual assignments (such as mathematics, programming exams and student home pages). Moreover, the continuous presence of teachers in the classroom gave them an understanding of each student's skills. At the end of some projects, students were requested to indicate how much each team member contributed to the project. In most cases, students gave a fair assessment of each other and even downplayed their own part.

As teamwork is not completely new in the curriculum, the results can also be compared with earlier experience. Numerous reports show that project work has always had several benefits such as better course completion and student satisfaction [START_REF] Markkanen | The Knowledge Practices Environment: a Virtual Environment for Collaborative Knowledge Creation and Work around Shared Artefacts[END_REF][START_REF] Muukkonen | Pedagogical design for knowledge creating inquiry in customer projects[END_REF]. On the other hand, it had no discernible overall impact on retention and graduation when it was only a small part of the curriculum.

The orientating courses of the first year enabled students to acquire basic knowledge of the study environment together. The collaborative practices developed during the first courses could later be observed when the students proceeded with their studies. How well the teamwork and collaboration skills persisted during the second academic year, was monitored and the results varied depending on the arrangements in the subsequent modules. When courses were built on collaboration the effect was positive, but when they followed traditional learning patterns, the students also reverted to those.

Discussion and Conclusion

The new study modules were by no means similar to each other. Much of the realization depended on the teacher team, some of whom preferred to divide the module into smaller parts among each other. All modules except one had a larger project, though. From the students' point of view, having a permanent home class was a significant difference to earlier years. Arguably, belonging to a group could have been a decisive factor in student retention. Despite the fact that the students were split to new groups according to the major subject selection the second year, where only part of the former classmates followed, they already had a network of study mates.

As the curriculum contents also changed in the reform, the results can be challenged in claiming that the students passed because of low requirements. However, the teachers involved would not agree, as actually the workload and requirements were set high. On the other hand, the number of individual deliverables within courses was certainly lower than before, which made the management of studies easier than before. Peer pressure within teams presumably added to the amount of work that was actually done because individuals were denied the privilege of procrastination. In particular students who had weaker self-regulation skills benefitted from a well-organized study mode. They had no choices to make in the first year, and the schedule for a week was always nearly similar.

According to the survey results, the teachers' and students' responses to the scales evaluating students' learning of knowledge work competences during the study modules were aligned. The average scores were quite high except the two scales that measured understanding practices and knowledge of various disciplines and interdisciplinary collaboration, which were not emphasized in the first-year studies. Open-ended responses revealed that students valued especially the working methods as well as teaching and guidance. They found disturbing particularly the organization of working, which might reflect the novelty of the implementation of teaching in larger modules together with a team of teachers. Also, various working methods were mentioned by students as unsatisfactory, including challenges in group work. However, it is encouraging that there were, in all, clearly more mentions about positive than negative aspects of the study experiences.

The main aims of the reform such as improved retention rate and high rate of completed study modules were achieved in an impressive manner. Despite the great variation of methods in course organization and teaching, the collaborative mode of learning and teaching seemed to contribute to a successful beginning of studies, and in particular to student enthusiasm and goals. Additionally, there were some indications that students adopted collaborative working practices that are expected by the software industry. Further research could address teachers' team teaching and the variation of course designs, their execution and related learning outcomes. Further, how students continue in their study programs is of interest to see; whether the reorganization of the first year of studies has continuous effects in terms of retention, or the completion of study credits. Further research could document the development of collaboration skills and other working life competences and aim to follow the students through their future transition to industry.

Fig. 1 .•

 1 Fig. 1. Students' and teachers mean responses on the scales of the Collaborative Knowledge Practices Questionnaire. (Not at all = 1, Very much =5). The questionnaire contained also three open-ended questions. According to a thematic analysis, in the open-ended questionnaire responses the students addressed the following aspects as positive or impressive in the courses (233 mentions in general): • Working methods (90): Practical tasks (24), Group work (30), Learning community and atmosphere (17), Working methods in general (11), Project work (4), Freedom of choice (4);

  ), Compulsory attendance (3); • Content (42): Difficult content (29), Not interesting or useful content (13); • Teaching and guidance (27): Poor teachers and teaching (13), Too little guidance and feedback (6), Unclear goals and evaluation criteria (8); • Facilities (8): Poor study premises and equipment (8).

Table 1 .

 1 Retention rates at Metropolia in all engineering degree programs.

	Year	2009	2010	2011	2012	2013
	Retention	42%	40%	43%	45%	49%

.

  Course completions of Finnish study groups in IT (one campus)

		2012	2013	2014	2015
		n=143	n=149	n=106	n=107
	1-14 ECTS credits	14%	11%	0%	2%
	15-29 ECTS credits	26%	41%	14%	9%
	30->	59%	48%	86%	89%

Table 3 .

 3 Course completions of international study groups in IT

		2012	2013	2014	2015
		n=74	n=76	n=45	n=48
	1-14 ECTS credits	23%	14%	0%	2%
	15-29 ECTS credits	46%	68%	20%	8%
	30->	31%	17%	80%	90%

Table 4 .

 4 Grade averages in international study groups in IT 2014-15

	Course	N	Average grade
	Networks A	27	4.3
	Networks B	24	4.5
	Games A	27	3.5
	Orientation A	22	3.9
	Orientation B	26	3.8

Acknowledgments. The study was partially supported by the Lifelong Learning Programme of the European Union under Grant KA3MP-2013-4764/001-001 (Promoting Knowledge Work Practices in Education; the KNORK project; http://knork.info).