Giasemi Vavoula

Meeting the Challenges in Evaluating Mobile Learning: A 3-level Evaluation Framework

Keywords: Mobile learning evaluation, learning context, evaluation framework, learning outcomes, ethics, informality and formality of learning, requirements for evaluation

We propose six challenges in evaluating mobile learning: capturing and analysing learning in context and across contexts, measuring mobile learning processes and outcomes, respecting learner/participant privacy, assessing mobile technology utility and usability, considering the wider organisational and socio-cultural context of learning, and assessing in/formality. A three-level framework for evaluating mobile learning is proposed, comprising a micro level concerned with usability, a meso level concerned with the learning experience, and a macro level concerned with integration within existing educational and organisational contexts. The paper concludes with a discussion of how the framework meets the evaluation challenges and with suggestions for further extensions.

Introduction

Mobile learning is a relatively new research area, with the first research projects appearing in the second half of the 1990s and the first international research conferences less than a decade ago. It is a field whose practice has not yet been standardised in terms of research frameworks, methods and tools. Thankfully, mobile learning has a lot of common ground with related research areas including Technology-Enhanced Learning (TEL) and Mobile Human-Computer Interaction (mobileHCI). 'Borrowing' frameworks and methods from these areas has been common practice for early mobile learning research, providing researchers with useful starting points.

As our conceptions and understanding of mobile learning deepen, these 'borrowed' frameworks and tools might no longer be adequate. We now appreciate mobile learning not just as learning that is facilitated by mobile technology, but also as the processes of coming to know through conversations and explorations across multiple contexts amongst people and personal interactive technologies (Sharples et al. 2007a). Such evolving conceptions introduce challenges to all aspects of mobile learning research, including evaluation. As the field matures, our frameworks and tools need to address these challenges.

In this paper we summarise six challenges in evaluating mobile learning: capturing and analysing learning in context and across contexts, measuring the processes and outcomes of mobile learning, respecting learner/participant privacy, assessing mobile technology utility and usability, considering the wider organisational and socio-cultural context of learning, and assessing in/formality. The paper proposes an evaluation framework with three levels: a micro level concerned with usability, a meso level concerned with the learning experience, and a macro level concerned with integration within existing educational and organisational contexts. The paper demonstrates how this framework has guided data collection and analysis in one mobile learning evaluation project, and concludes with a discussion of how it meets the evaluation challenges and with suggestions for further extensions.

Challenge 1: Capturing learning context and learning across contexts

A major task for educational evaluation is to identify and analyse learning within and across contexts. For mobile learning, the interest is not only in how learning occurs in a variety of settings, but also how people create new contexts for learning through their interactions and how they progress learning across contexts. This poses a significant challenge to evaluators of mobile learning. In order to establish, document and evaluate learning within and across contexts, a researcher needs to analyse: the physical setting and the layout of the learning space (where); the social setting (who, with whom, from whom); the learning objectives and outcomes (why and what); the learning methods and activities (how); the learning progress and history (when); and the learning tools (how).

When evaluating learning in a traditional classroom, researchers generally have access to information about these context elements before, during and after the learning experience. Thus, they can inspect the classroom and interview the teacher and learners in advance of a lesson to discover the objectives, methods, lesson plan and tools. To evaluate a school museum visit or field trip, the researcher can visit the site and inspect the lesson plan, but will generally not know in advance the route that each student will take. For personal or family visits to museums or other learning sites, neither the objectives nor the trajectory may be known in advance. Learning objectives may arise as a response to interactions with the environment and learning trails may be guided by curiosity or unplanned events. The learners themselves may not be known in advance, for example when evaluating the learning experience of museum visitors randomly selected at the museum entrance. Personal mobile learning embraces any learning event where people, individually and collectively, continually create micro-sites for learning out of the available physical and social resources. In considering this generic case, the setting, objectives, methods and processes may all be unpredictable.

Table 1 below portrays the increasing vagueness in moving from evaluating a classroom lesson, to a school museum visit, to personal or family museum visits, to personal mobile learning across formal and informal settings. Each set of context elements requires specific evaluation methods, to match the actual learning processes and outcomes to expectations, or to capture contingent and unexpected learning events. Recent research efforts have focused on devising tools and methods appropriate for capturing and analysing mobile learning contexts. Some efforts concentrate on implementing technology-based solutions for data collection, such as mobile eye tracking or wearable interaction capture kits [START_REF] Roto | Examining mobile phone use in the wild with quasi-experimentation[END_REF]. Although these have the advantage of capturing accurate data in context, they have some disadvantages, not least the obtrusiveness of the apparatus used. Other efforts opt for cooperative inquiry-based solutions [START_REF] Hsi | Conceptualizing Learning from the Everyday Activities of Digital Kids[END_REF], such as using learners' accounts of the experience through retrospective interviews, diaries, or attitude surveys [START_REF] Clough | The Uses of PDAs and Smartphones in Informal learning[END_REF][START_REF] Vavoula | WP4: A Study of Mobile Learning Practices[END_REF]. These have different shortcomings such as the accuracy of recall, the degree to which post-rationalisation skews data, and the effect of the participants' concern over the image they project.

Increasingly, mobile evaluation designs include mixed methods. These are useful not only for validating data, but also for capturing different perspectives of the learning experience. Thus, collected data might include recorded video, audio transcripts, observation notes, artefacts produced by the learners, and application screenshots. Interpreting such rich collections of data can be challenging too, in terms of assembling it into a meaningful, accurate and elaborate account of the learning experience. Related research addresses the design of tools and methods to support the sequencing, synchronisation, inter-relation and visualisation of evaluation data [START_REF] Greenhalgh | Meeting the Challenges in Evaluating Mobile Learning: a 3-level Evaluation Framework[END_REF][START_REF] Papadimitriou | Analysis of an informal mobile learning activity based on activity theory[END_REF][START_REF] Smith | Reconstructing an informal mobile learning experience with multiple data streams[END_REF].

Challenge 2: Has anyone learned anything?

A second challenge that faces mobile learning evaluation is the assessment of learning processes and outcomes. In traditional learning settings such as the classroom there are well-established and accepted methods for the assessment of learning activities, such as essay writing, multiple choice tests, open-book exams, and unseen examinations. Distinctions have been made between formative assessment (aiming to provide students with feedback regarding their progress) and summative assessment (aiming to judge and sum up the students' achievements) [START_REF] Scriven | The methodology of evaluation[END_REF], with formative assessment bearing the greater potential to aid and complement teaching and learning (Black & Wiliam 1998a;[START_REF] Black | Inside the black box: raising standards through classroom assessment[END_REF].

Summative assessment is often used as a measure of success of the teaching as well as a measure of effectiveness of the learning [START_REF] Boud | Assessment and learning: contradictory or complementary?[END_REF], but with many (often unresolved) issues regarding the reliability and validity of summative assessment methods (see [START_REF] Knight | A Briefing on Key Concepts: Formative and summative, criterion and normreferenced assessment[END_REF] for a discussion of these issues). Despite these difficulties, summative assessment can be meaningful in formal learning contexts where learning objectives and desired outcomes are well specified in advance. By contrast with formal education, mobile, informal learning can be both personal and elusive. The learning may be personally initiated and structured, such that it is not possible to determine in advance where the learning may occur, nor how it progresses or what outcomes it produces. It may also be difficult to track the progress of learning if it occurs across multiple settings and technologies.

An alternative approach is to examine the experience for evidence which might suggest that productive learning is taking place. For example, in the context of museum learning, [START_REF] Griffin | Finding Evidence of Learning in Museum Settings[END_REF] suggest to watch for instances where learners initiate and show responsibility for their own learning (e.g. by writing, drawing, or taking photos by choice; deciding where and when to move), are actively involved in learning (e.g. by absorbed, close examination of resources; or persevering with a task), make links and transfer ideas and skills (e.g. by comparing evidence), and share learning with experts and peers (e.g. by talking and gesturing; or asking each other questions). Adaptations of the Critical Incidents method [START_REF] Flanagan | The critical incident technique[END_REF] provide one way to achieve this. For example, activities of learners who wear radio microphones are videotaped at a discrete distance. The evaluators then watch the videotapes to identify observable critical incidents that appear to be breakthroughs (indicating productive new forms of learning or important conceptual change) or breakdowns (where a learner is struggling with the technology, is asking for help, or appears to be labouring under a clear misunderstanding). These incidents can be assembled into a compilation tape and reviewed with the learners for further elaboration (Vavoula et al. forthcoming) or analysed as is [START_REF] Anastopoulou | Learning 21st Century Science in Context with Mobile Technologies[END_REF].

Another alternative is to focus on learner perceptions of the learning experience rather than learning outcomes in terms of cognitive gains. Attitude surveys have been used extensively in the mobile learning literature to measure learner attitudes towards the technology and their enjoyment of the experience. Since attitudes are closely related to intrinsic motivation and learning agency [START_REF] Hidi | Motivating the Academically Unmotivated: A Critical Issue for the 21st Centrury[END_REF], they can be a reliable predictor of conditions for effective learning (though not necessarily of learning outcomes). However, the mobile learning community has yet to produce standardised attitude measurement instruments such as those available in other fields (e.g. science learning -Moore & Sutman 1970).

Information useful in assessing learning can also be found in learner-created artefacts, such as log files of computer activity or web access, the results of online quizzes, learner-created media, and personal reflective documents such as blogs and e-portfolios. Further work is needed to integrate these into a revealing and valid assessment of learning.

The challenge of assessing learning is not unique to mobile learning and is not easily solved. Although a learning experience can be a well defined event with a start and a finish, learning is an ongoing, lifelong process of personal transformation and, as such, requires longitudinal, historical assessment.

Challenge 3: An ethical question

Research ethics frameworks have governed research involving human subjects for decades. With the increasing use of the Internet as the research object and medium, accounts of virtual research ethics prevailed, along with analysis of differences in the nature of ethical issues confronted by virtual versus traditional research [START_REF] Buchanan | Readings in Virtual Research Ethics: Issues and Controversies[END_REF]). The challenge for mobile learning evaluation is to translate these issues into ethical guidelines appropriate for mobile contexts.

The evaluation of mobile learning presents particular ethical problems, beyond those routinely associated with study of people and technology, of ensuring their safety and informed cooperation. A fundamental need is to explicate the purpose and principles of the evaluation within an ethical framework. Ethics can arise from differing philosophical foundations regarding the nature of 'reality' and the value of the scientific method in validating claims by analysis of objective data.

Within a modernist framework the researcher strives for objectivity. Engaging in postmodern research involves researchers reflexively asking why they are doing this research in this way, what it is silent about, what gives it authority, and who is privileged by it. (Traxler & Bridges 2004:204).

As studies of learning move out of the classroom into homes and outdoor locations, so the evaluation will need to rely more on a combination of data collected automatically by mobile devices (such as logs of user interaction, time and location) and self-reports from learners. These do not fit naturally together, since they exemplify objectivist and postmodern approaches to the study of learning. For example, a current study (as yet unpublished) of children using mobile phones to create records of their daily eating habits has found that some children deliberately avoid photographing unhealthy food items. This is not simply a matter of treatment of missing data items, but indicates deeper problems of children's self-image, research by proxy using mobile devices, intrusion into daily life, power and willing cooperation, and the interpretation of silence. As mobile learning grows in scale and scope, evaluators must address the reliability of evidence (particularly when collected outside the lab or classroom) and difficulties of conducting scientifically rigorous studies as a basis for formulating evidence-based policy 1

Traxler and Bridges indicate specific ethical issues of evaluating mobile learning, including: explaining the scope of mobile learning in a succinct and appropriate way, gaining informed consent for novel forms of interaction (such as learning by SMS), and identifying the contribution of learners across multiple devices and contexts. Other issues include identifying the ownership of material collected across contexts (such as field trips), the rights of participants to know when and how they are being monitored during their daily lives, and possible health dangers associated with regular use of wireless technologies [START_REF] Ahonen | Sustainable Wireless Computing -How Public Information Systems Can Be Designed to Reduce Exposure[END_REF][START_REF] Patrick | Health and the Mobile Phone[END_REF].

.

Obtaining informed consent can be problematic: the previous sections described the vagueness of mobile learning context and the elusiveness of mobile learning outcomes. When evaluators are uncertain of what will constitute the mobile learning experience, how accurately can they inform the participants of what data is sought and why? Assuming that a vague description of the requirements for participations is acceptable, how can learners consent to disclosing information about events they currently do not know when, where and under what circumstances will take place?

Even if the essence of the evaluation is successfully conveyed to the participants, and they consent to it, there are still important issues to consider relating to the degree to which they will co-operate in practice -either in terms of disclosing all that might be relevant, or in terms of carrying out related practical tasks such as synchronising their mobile devices as and when requested [START_REF] Trinder | Have You Got Your PDA With You?[END_REF].

Furthermore, when asking participants to record their own learning (either through wearing/carrying recording equipment or by keeping a written record of their learning), we are in essence making them co-researchers. In doing so, we also need to ensure that they are able themselves to follow the ethics rules and regulations; and are more legitimate research partners than inactive participants [START_REF] Bakardjieva | User-Centered Internet Research: The Ethical Challenge[END_REF].

A major challenge then for mobile learning evaluation is to accurately inform participants, to ease their participation, and to build capacity in ethical research and mobile learning practice by providing them with appropriate ethics training. In the process evaluators should ask themselves how much they really need to know, and to investigate best practices in safeguarding and disseminating sensitive personal data.

Challenge 4: Mobile technology

Evaluations of mobile learning often reference inherent limitations of mobile devices, such as their small screens, short battery lives, intermittent connectivity, and associated human factors, all of which affect their usability (see e.g. [START_REF] Corlett | Evaluation of a Mobile Learning Organiser for University Students[END_REF]. As the focus of research shifts from the mobility of the technology to the mobility of the learner, additional issues arise as learners move across multiple, quickly-dating devices, both personal and public, possibly over short time periods in multiple locations. Extracting learning interactions from this mesh of technology interactions requires synchronization of data capture and analysis across multiple devices and interfaces.

Assessing the usability of the mobile technology and the effectiveness of its integration with the mobile learning practice remains a high priority for evaluation.

Thus, challenges of mobile human-computer interaction stemming from the complexity introduced by physical movement and changing variables [START_REF] Kjeldskov | New Techniques for Usability Evaluation of Mobile Systems[END_REF]) and the small scale and ubiquitous nature of mobile devices [START_REF] Hagen | Emerging research methods for understanding mobile technology use[END_REF], add to the challenges already facing mobile learning evaluation.

Challenge 5: Seeing the bigger picture

There is a wealth of literature on the relation between information technology and institutional change (e.g. [START_REF] Fountain | Building the Virtual State: Information Technology and Institutional Change[END_REF]) and methods of performance management such as balanced scorecard [START_REF] Kaplan | Using the balanced scorecard as a strategic management system[END_REF] and Six Sigma [START_REF] Pande | The Six Sigma Way: How GE, Motorola, and Other Top Companies are Honing Their Performance[END_REF]. Becta, the UK Government agency leading the introduction of technology in education, provides a strategy for the introduction and effective use of technology across all education sectors. This describes the progression of an institution in confidence with technology from 'enabled', through 'capable' to 'confident' through introduction of appropriate infrastructure, planning and leadership, learner access to resources, and personalisation of learning. Its Performance Framework (BECTA 2008:47) indicates the systemic changes needed to achieve the goals of improved personalised learning experiences. These are: confident system leadership and innovation, technology confident effective providers, engaged and empowered learners and enabling infrastructure and processes.

For Higher Education (HE), [START_REF] Oliver | What does 'impact' mean in the evaluation of learning technology[END_REF] suggest four kinds of impact of educational technologies: impact on students' learning, impact on individual academics' practice, impact on institution, and national impact. Also in the context of HE, Price and Oliver (2007) identify three types of impact studies: anticipatory, ongoing and achieved. Anticipatory studies relate to preintervention intentions, opinions and attitudes; ongoing studies focus on analysing processes of integration; and achieved studies are summative studies of technology no longer 'novel'. [START_REF] Riley | Educational Technology and Practice: Types and Timescales of Change[END_REF] extends this impact framework by distinguishing between minor modifications and culturally significant changes in practice, and suggesting that different kinds of change will emerge over different timescales.

Mobile learning evaluation has similar issues regarding impact. It needs to examine how learning takes place within a personal, socio-cultural and institutional context, to chart the progression of institutions in their maturity of support for learning with mobile technology, and examine the relation between personal and institutional learning. It needs to address how the immediate learner experience within these contexts blends with or confronts existing practices to lead to new practices, by analysing this change process over extended periods of time. These requirements necessitate an extended view of the role of evaluation as a continual process of adjustment and fine-tuning.

Challenge 6: Formal or informal?

Mobile learning is often defined in terms of the technology that mediates the learning experience: if the technology is mobile, so is the learning. Mobility, however, is not an exclusive property of the technology, it also resides in the lifestyle of the learner, who in the course of everyday life moves from one context to another, switching locations, social groups, technologies and topics; and learning often takes place inconspicuously or is crammed in the short gaps between these transitions. Although this view of learning is inclusive of formal education contexts, it is particularly pertinent to everyday, informal learning.

Nevertheless, characterising a learning experience as formal or informal can be complicated. For example, is the learning of pupils visiting a museum (largely considered an informal learning setting) with their school (an irrefutably formal learning setting) a case of formal or informal learning? There is a large literature related to definitions of informal learning and related terminology, a review of which is beyond the scope of this paper. However, a general tendency seems to be to define informal learning in contrast to formal learning, and formal learning in turn to be confined to learning that takes place in educational settings. [START_REF] Colley | Informality and formality in learning[END_REF][START_REF] Colley | Informality and formality in learning[END_REF] argue that "seeing informal and formal learning as fundamentally separate results in stereotyping and a tendency for the advocates of one to see only the weaknesses of the other … It is more sensible to see attributes of informality and formality as present in all learning situations". They advocate four groups of attributes: those related to the learning process, to the location and setting, to the learning purposes, and to the learning content. They propose that attributes of in/formality are interrelated in different ways in different learning situations, and that those attributes and their interrelationships influence the nature and effectiveness of learning in any situation.

Understanding such attributes of in/formality and their interrelationships in mobile learning is important for evaluation. It is not only a case of analysing pre-existing practices in terms of processes, settings, purposes and content, but also of capturing how the introduction of mobile learning practices, or new ways of supporting them, can change the in/formality of the learning experience.

Precepts for mobile learning evaluation

The challenges discussed in the previous sections translate into a set of basic precepts for mobile learning evaluation:

P1. Capture and analyse learning in context, with consideration of learner privacy (challenges 1, 3) P2. Assess the usability of the technology and how it affects the learning experience (challenge 4) P3. Look beyond measurable cognitive gains into changes in the learning process and practice (challenge 2) P4. Consider organisational issues in the adoption of mobile learning practice and its integration with existing practices and understand how this integration affects attributes of in/formality (challenges 5, 6) P5. Span the lifecycle of the mobile learning innovation that is evaluated, from conception to full deployment and beyond (challenges 1-6) As an illustration of how these might guide evaluation in practice, the following section presents an evaluation framework for mobile learning and its application in the context of the Myartspace project.

M3: A three-level framework for evaluating mobile learning

Myartspace supports structured inquiry learning through technology that connects learning in the classroom with learning in museums and galleries. Detailed descriptions of the project and the evaluation process and outcomes have been presented elsewhere (Sharples et al. 2007b;Vavoula et al. 2006a;[START_REF] Vavoula | Learning Bridges: a role for mobile technologies in education[END_REF][START_REF] Vavoula | Museum explorations in physical, personal and virtual space through MyArtSpace[END_REF]. In summary, Myartspace addresses the problem of how to connect a school museum trip with classroom activities of planning and further study. It enables school students to create their own interpretations of museum exhibits through descriptions, images and sounds they collect at the museum, which they then review, reflect upon and share outside the museum. Before the visit, the teacher will typically set an open-ended question that the students can answer by gathering and selecting evidence from the museum visit. On arrival at the museum, students are given multimedia mobile phones running custom software which they can use to collect multimedia presentations of exhibits, take photos, record sounds, or write text comments. This content is transmitted by the phone into a time-ordered collection on their personal web space. Back at school, the students can organize the material into online galleries to present their findings in the classroom and share with their friends and family. M3, the evaluation framework developed in the context of Myartspace followed the Lifecycle approach to educational technology evaluation proposed by [START_REF] Meek | Adopting a Lifecycle Approach to the Evaluation of Computers and Information Technology[END_REF]. This places evaluation at the centre of the development process, from the early stages of design to a final assessment of the deployed technology in use. The Lifecycle approach can be matched to a sequential systems development process [START_REF] Royce | Managing the Development of Large Software Systems[END_REF], with evaluations undertaken at the end of each stage, or to iterative [START_REF] Larman | Iterative and Incremental Development: A Brief History[END_REF] or socio-cognitive methods [START_REF] Sharples | Socio-cognitive engineering: a methodology for the design of human-centred technology[END_REF] where evaluation activities are undertaken at key points that are of most value to support the design process or inform stakeholders, with the outcomes of each evaluation guiding the next phase of the system development or feeding into an iteration of an earlier phase.

Evaluation under M3 is conducted at three levels of granularity:

1. Micro level, which examines the individual activities of the technology users and assesses the usability and utility of the educational technology system. For Myartspace the activities included collecting objects through exhibit codes, making notes, contacting people who had collected a particular item, recording audio, and taking pictures. 2. Meso level, which examines the learning experience as a whole, to identify learning breakthroughs and breakdowns. It also assesses how well the learning experience integrates with other related activities and experiences. For Myartspace, evaluation at this level involved exploring whether there was a successful connection between learning in the museum and the classroom, as well as identifying critical incidents in the museum that reveal new patterns and forms of learning or where learning activity is impeded. 3. Macro level, which examines the impact of the new technology on established educational and learning practices and institutions. For Myartspace this related to the organisation of school museum visits. The evaluation at this level examined the appropriation of the new technology by teachers, the emergence of new museum practices in supporting school visits, and how these related to the original project visions. The development of Myartspace comprised four broad phases: (1) Requirements analysis, to establish the requirements for the socio-technical system (the users and their interactions with technology) and specify how it would work, through consultation with the different stakeholder groups; (2) Design of the user experience and interface; (3) Implementation of the service; and (4) Deployment of the service. These are compatible with an Agile Development approach [START_REF] Beck | Manifesto for Agile Software Development[END_REF] where requirements can evolve throughout the development process to take account of the evaluations of usability, learning effectiveness and institutional adoption. Thus, the requirements analysis persisted throughout the project lifecycle, and covered all three levels of analysis (micro, meso and macro).

Figure 1 illustrates this gradual introduction of evaluation activities at the three framework levels over all project phases. The horizontal axis in Figure 1 depicts time; the change of focus development phase over time is shown, as is the persistence of requirements analysis throughout the project lifecycle. The shaded areas represent activities for requirements analysis (dark gray) and evaluation at the three levels during design, implementation and deployment (all other shades).

The emphasis on level of requirements analysis changes during the development process. At the start of a project, the requirements analysis must take account of all levels to set initial requirements for an educational experience that integrates technology, effective learning and institutional support. As the project progresses, the technology matures, so that changes to requirements become focused on the learning context and institutional adoption. At the end of the project, the requirements have been finalised and are evaluated at all levels. The emphasis on level of evaluation also changes during the development process. Early evaluations at micro level inform the user interface and human-technology interactions. Once the technology is robust enough to allow assessment of educational value, evaluation activities at the meso level are introduced during the implementation phase. Similarly, the macro level requires that the technology is in place and used for long enough to establish its effects on e.g. school museum visiting practice, so evaluation activities at the macro level may be introduced during the deployment phase.

To establish the value of the service at each of the three levels, evaluation activities explore the gap between expectations and reality and uncover unforeseen processes and outcomes. This is enacted in two stages of data collection and a stage of analysis:

1. Stage 1: collect data about what is supposed to happen at a level. User expectations at each level can be captured through interviews with users (e.g. teachers, students, museum staff) and by analysing technical requirements specifications, user documentation, training sessions and lesson materials. 2. Stage 2: collect data about what actually happened at a level. The user experience is documented through observations and video and audio recordings to establish the reality of technology use for the different users. 3. Stage 3: examines the gaps between user expectations and reality through a combination of reflective interviews with users and critical analysis of the data collected in stages 1 and 2. In summary, M3 follows a Lifecycle approach of continuous strategic evaluation to guide an agile approach to software development and to inform stakeholders in the development process. It assesses the evolving design and implementation at three levels, of usability, educational effectiveness and institutional adoption. For each level the evaluation relates what should happen (through interviews with stakeholders and examination of documents) to what actually happens (through observation of user experience) and examines any gaps between expectation and reality as evidence of a need to modify requirements, design, implementation, or deployment. These findings guide the next phase of the system development or feed into an iteration of an earlier phase. Furthermore, M3 provided an efficient way to structure data analysis for the evaluation of Myartspace, allowing the documentation of the potential of such a service. Successes and failures of the service at all levels (micro, meso and macro) were identified, along with inter-level influences. Table 3 outlines the data analysis processes for the data collected during stages 1 and 2 of the requirements and evaluation activities. Figure 2 gives an illustration of how M3 guided data analysis. It presents a sample of data snippets that were gathered through observations, interviews, and system logs. Snippet 1 shows that the children have created large amounts of content by taking photographs, recording sounds and writing comments. This suggests that at the micro level, creating and collecting items is a quick and easy task for them. Examination at the meso level reveals that this ease of use may not result in productive learning unless it is accompanied by creativity and sense of ownership, as exemplified in snippet C. This is an example of synergy between the two levels.

Group average

At the micro level, students are not able to annotate their photographs and the audio clips they recorded with notes describing what they were about or why they were recorded. Although they could create text notes, such notes could not be directly associated with photos or audio clips. This was puzzling for some students, who expected to be able to record this metadata (snippet B).

Analysis at the meso level revealed frustration back in the classroom when students were trying to interpret their collections (snippet D). This is an example of a micro-level problem that migrates to the meso level, affecting the students' learning. Possible fixes to this problem can be placed either at the micro-level (e.g. changing the system to support annotation of photographs) or at the meso-level by giving advice to the students in effective techniques such as reading an exhibit label into the phone after taking a photograph of it, something that students were actually observed doing.

A final example comes from the analysis of snippets 2 and A. As mentioned previously, snippet 2 suggests that creating and collecting items is a quick and easy task at the micro level. Snippet A, however, suggests that decomposing and interpreting the collected content back in the classroom takes significantly longer, which resulted in students not managing to go through all their collected items during the post-visit lesson. This is an example of how omitting to resonate the micro and meso levels might lead to problems in the meso level. Possible fixes to this problem can be placed at any of the three levels. At the micro level, we can enforce an upper limit on the number of items a student can collect; or we might want to simplify the web interface so that online processing of items takes less time. At the meso level, we might try to educate students in regulating their collecting practices. At the macro level, we might try to influence the teachers' practice so that they include more than one post-visit lesson in their planning.

Discussion and Conclusion

M3 provides a structured format to assess usability, educational and organisational impact, and their inter-relationships. • The focus on interaction puts equal emphasis on the learners and the technology P3. Look beyond cognitive gains into changes in the learning process and practice

• Relates the intended learning processes and outcomes to observed activities and examines the gaps between expectation and reality P4. Consider organisational issues in the adoption of mobile learning practice and its integration with existing practices and understand how this integration affects attributes of in/formality

• Can analyse individual interactions, educational processes and organisational change • Can be applied across formal and informal settings P5. Span the lifecycle of the mobile learning innovation that is evaluated, from conception to full deployment and beyond

• Integrates with a Lifecycle [START_REF] Meek | Adopting a Lifecycle Approach to the Evaluation of Computers and Information Technology[END_REF]) approach to evaluation The six challenges in mobile learning evaluation identified in this paper are a direct consequence of the complex nature of mobile learning as we have come to understand it, as a social rather than technical phenomenon of people on the move constructing spontaneous learning contexts and advancing through everyday life by negotiating knowledge and meanings through interactions with settings, people and technology. In this paper we construed these challenges into precepts for evaluation and presented M3 as the implementation of one interpretation of these precepts; other frameworks previously proposed in the literature for the design [START_REF] Ryu | Designing Learning Activities with Mobile Technologies[END_REF] and/or evaluation of mobile learning [START_REF] Taylor | A task-centred approach to evaluating a mobile learning environment for pedagogical soundness[END_REF][START_REF] Taylor | Towards a Task Model for Mobile Learning: a Dialectical Approach[END_REF][START_REF] Traxler | Evaluating Mobile Learning: Reflections on Current Practice[END_REF] can be seen as different interpretations of the same precepts. We view M3's main contributions to this growing body of knowledge and experience in mobile learning design and evaluation to be (a) its multi-and cross-level focus on individual interactions, educational processes and organisation change; (b) the way it combines with Lifecycle evaluation approaches to weave requirements analysis and evaluation into the whole development cycle, thereby emphasising experience-centred over technology-centred development; and (c) its focus on experience gains over cognitive gains alone. These qualities of M3 allude to (Traxler forthcoming) proposal for alignment of our modernist conceptions of evaluation with the postmodern reality of mobile technologies and learning.

The application of M3 in the context of Myartspace was successful and offered valuable insights to the project. Although we believe the framework is transferable to other mobile learning contexts, it needs further development to address, for example, contexts with higher ethical concerns, or contexts where it is challenging to align the requirements analysis and evaluation activities with the objectives and ethos of the project. The outcomes of an evaluation based on this framework can feed directly into system design, as has happened in the case of Myartspace. Perhaps with suitable extensions the framework could serve the design process more directly, guiding mobile learning designers to interpret and implement requirements for learning across self-constructed contexts.

would like to thank the Department for Culture, Media and Sport for the funding of Myartspace through Culture Online. Myartspace was designed and developed by The Sea (http://www.thesea.com); the service was branded as OOKL in 2007 and is now commercially available (www.ookl.org.uk) and used in a number of sites, including the Kew Gardens in London. Last, but certainly not least, we would like to thank all the students, teachers, and museum educators who took part in our trials, with special thanks to Bryony Kelly from the D-Day museum for her enthusiasm about Myartspace and her continuous support during the user trials.

Figure 1 :

 1 Figure 1: Evaluation activities at the three levels over the project phases.

  Published inVavoula, G. & Sharples, M. (2009) Meeting the Challenges in Evaluating Mobile Learning: a 3-level Evaluation Framework. International Journal ofMobile and Blended Learning, 1,2,

Figure

  Figure 2: Data snippets gathered through observations (C, D), interviews (A, B), and system logs (1, 2)

--________________________________vagueness__________________________ ++ Classroom School museum visit or field trip Personal or family visit Personal mobile learning

  Published inVavoula, G. & Sharples, M. (2009) Meeting the Challenges in Evaluating Mobile Learning: a 3-level Evaluation Framework. International Journal ofMobile and Blended Learning, 1,2,

	Physical setting	 Conventional	 Moving around	 Moving around	 Unpredictable &
		and static	a fixed location	a fixed location	changing
	Social setting	 Fixed	 Pre-arranged	 Pre-arranged	 Unpredictable
					and changing
	Learning objectives	 Externally set	 Externally set	 Personally set or	 Personally set or
	and outcomes			contingent	contingent
	Learning method	 Pre-determined	 Pre-determined	 Pre-determined	 Pre-determined
	and activities			or contingent	or contingent
	Learning progress	 Pre-determined	 Pre-determined	 Mostly	 Contingent
	and history		or contingent	contingent
	Learning tools	 Provided	 Provided by	 Provided &	 Personal &
			school or museum	personally owned	serendipitous

Table 1 . Context elements relevant to the learning researcher.

 1

Table 2

 2 summarises requirements analysis and evaluation activities and the respective data collection methods in the Myartspace project at each level of M3, for all project phases.

	Data collection for requirements and evaluation activities	Level Phase
	Requirements analysis
	Stage 1: 'expectations' data collection
	• Scoping study of previous projects and related recommendations	All	Requirements
	• Consultation workshop on 'User Experience' to establish requirements
	Stage 2: 'reality' data collection
	• Data supplied by evaluation analysis	All	Requirements
	Heuristic Evaluations (examining how system designs compare to expectations
	re established design heuristics)
	Stage 1: 'expectations' data collection
	• Established design heuristics	Micro All
	Stage 2: 'reality' data collection
	• Experts undertaking heuristic evaluation	Micro All
	Technical testing prior to trials
	Stage 1: 'expectations' data collection
	• Data supplied by system requirements	Micro Implement
	Stage 2: 'reality' data collection
	• System performance tests	Micro Implement
	Full scale user trial	All	Implement/Deploy
	(Key Stage 2 class visits Myartspace museum)
	Stage 1: 'expectations' data collection
	• Examine system documentation (Teacher's Pack and Lesson Plans, online	Micro Implement/Deploy
	help) for descriptions of functionality
	• Interview teacher prior to lesson to assess level of knowledge and
	expectations for functionality
	• Observe training sessions at museum and school to document how
	functionality is described to teachers/students.
	• Student questionnaires regarding expectations of system functionality in
	forthcoming lesson
	• Analyse description of educational experience based on Teacher's Pack	Meso Implement/Deploy
	and Lesson Plans
	• Interview teachers and museum educators prior to lessons about what they
	have planned for the students' learning experience
	• Observe teachers and museum educators while presenting learning
	experience to students in the classroom/museum
	• Student questionnaires regarding expectations of learning experience in
	forthcoming lesson
	• Analyse descriptions in service promotion materials, original proposal,	Macro Deploy
	minutes of early project meetings
	• Interviews with stakeholders to elicit initial expectations for impact of
	service
	Stage 2: 'reality' data collection
	• Observe lesson to establish actual teacher and student experience of	Micro Implement/Deploy
	functionality
	• Interview teacher after the lesson to clarify experience of functionality
	• Questionnaire and focus groups with students after the lesson to capture
	experience of functionality
	• Observe educational experience in museum/classroom	Meso Implement/Deploy
	o Note critical incidents that show new forms of learning or
	educational interaction
	o Note breakdowns
	• Interviews/focus groups with teachers, museum educators, students on
	educational experience in museum/classroom
	• Review of press coverage and interviews with stakeholders to document	Macro Deploy

Table 2 : Methods used for data collection during requirements analysis and evaluation activities at each level, for each project phase Table 3: Methods used for data analysis during requirements and evaluation activities at each level, for each project phase

 2

  TeacherPublished inVavoula, G. & Sharples, M. (2009) Meeting the Challenges in Evaluating Mobile Learning: a 3-level Evaluation Framework. International Journal ofMobile and Blended Learning, 1,2,

	Data analysis for requirements evaluation activities	Level Phase
	Requirements analysis
	Stage 3: data analysis
	• Workshop to finalise educational and user requirements	All	Requirements
	• Revisions of requirements in light of evaluation findings
	Heuristic Evaluations
	Stage 3: data analysis
	• Analysis of expert reports and production of (re)design recommendations Micro All
	Technical testing prior to trials
	Stage 3: data analysis
	• Analysis of performance data against requirements	Micro Implement
	Full scale user trial			All	Implement/Deploy
	Stage 3: data analysis
	• Capture expectations-reality gaps in terms of user experience of	Micro Implement/Deploy
	functionality through reflective interpretation of documentation analysis
	in the light of observations; through interviews and focus groups with
	teachers/students; and through critical incident analysis with students.
	• Capture expectations-reality gaps in terms of educational experience	Meso Implement/Deploy
	through reflective interpretation of documentation analysis and
	observations; through interviews/focus groups with teachers, students,
	museum educators; and through critical incident analysis with students.
	• Reflective analysis of expectations-reality gaps in terms of service	Macro Deploy
	impact
			Class
	Photographs	33	364	"A student can effectively
	Sounds	11	121	process 5-10 items during a single post-visit lesson"
	Written comments	7	77
	Collected objects	7	75
	TOTAL	58	637
	Objects created / collected during museum visit
				"How will I know what
	"I expect to be able to record what pictures are of"	"-It has a code -I want to take my own picture"	this photo is about?" Student
	Student		Students dialogue

  Table 4 describes how it follows the precepts for mobile learning evaluation presented earlier in the paper. Published in Vavoula, G. & Sharples, M. (2009) Meeting the Challenges in Evaluating Mobile Learning: a 3-level Evaluation Framework. International Journal of Mobile and Blended Learning, 1,2, 54-75.

	Precept	Framework qualities
	P1. Capture and analyse learning in	• Illuminates learning activities and contexts at different
	context; with consideration of	levels of detail
	learner privacy	• Involves learners and teachers as informed participants in
		the evaluation process
	P2. Assess the usability of the
	technology and how it affects
	the learning experience

• Micro-level (usability) evaluation activities are linked with evaluation activities at the meso and macro levels (educational effectiveness and institutional adoption)

Table 4 : Methods used for data analysis during requirements and evaluation activities at each level, for each project phase

 4

Published inVavoula, G. & Sharples, M. (2009) Meeting the Challenges in Evaluating Mobile Learning: a 3-level Evaluation Framework. International Journal ofMobile and Blended Learning, 1,2,

We are grateful to a reviewer for bringing this aspect of evaluation to our attention.

Published in Vavoula, G. & Sharples, M. (2009) Meeting the Challenges in Evaluating Mobile Learning: a 3-level Evaluation Framework. International Journal of Mobile and Blended Learning, 1,2, 54-75.

Acknowledgements

We are grateful to Peter Lonsdale, Julia Meek and Paul Rudman for their invaluable contribution in the design of the evaluation of Myartspace and the development of the evaluation framework. We


Daniel J Van Rooijen
email: dvanrooijen@hotmail.com

Urbanization, water demand and sanitation in large cities of the developing world: an introduction to studies carried out in Accra, Addis Ababa and Hyderabad

Keywords: Urban water, sanitation, water resources, sustainable development, developing countries

Urbanization, water demand and sanitation in large cities of the developing world: an introduction to studies carried out in Accra, Addis Ababa and Hyderabad

INTRODUCTION

Urban population numbers and water use patterns mainly determine the management of water supply and wastewater disposal and thereby contribute to the ecological footprint of a city. In many developing countries in Africa and Asia, urban water agencies have great difficulties to provide proper water supply access and sanitation due to rapid urbanization and insufficient capacity and resources (UN HABITAT 2001). A complex of technical, institutional, economic, social and environmental problems can be noted in many cases [START_REF] Stephenson | Problems of developing countries[END_REF]. The challenges to meet water demand are much bigger, as compared to cities in the developing world, due to a much higher urban growth rate and lower average income [START_REF] Biswas | Water management for large cities[END_REF]. Both low income and rapid urban growth are found in countries in Africa and Asia, justifying the narrowing of focus to these continents. Proper water management is often lacking, although allocation plans of water to a large city can be ambitious in terms of added water volumes and far reaching as they cross basins borders ( [START_REF] Van Rooijen | Sponge City: Water Balance of Megacity water use and wastewater use in Hyderabad, India[END_REF].

A sound assessment of the current watsan situation in cities is limited by unreliable governmental data (due to its political dimension) and their possible use of inappropriate criteria that define what is 'safe supply' and 'adequate sanitation'. The implications of (proposed) water and sanitation development for urban water demand and wastewater generation is not sufficiently researched. It is important that the impact of cities on the rural environment should be researched and translated into useful tools for planners and policy makers.

Findings from urban hydrological studies based on collection of hydrological data, calculations and modeling constitute a necessary fundamental for meaningful water management not only in urban areas but also in entire river basins [START_REF] Niemczynowicz | Urban hydrology and water management -present and future challenges[END_REF]. City case studies that assess the current water supply and sanitation situation and model scenarios of future water demand and wastewater disposal, contribute towards this end and support integrated water management in an urban watershed context. Based upon the above problem analysis, the following research question, objectives, hypotheses and goals where developed;

Research question

What implications do different urban water and sanitation development scenarios have for the future characteristics of urban water demand and wastewater generation across the urban-rural gradient?

Overall objective

To assess the impact of urban water and sanitation development scenarios on the future characteristics of urban water demand and wastewater generation across the urban-rural gradient.

Sub-objectives 1. To assess the impact of urban water demand scenarios on existing agricultural uses. 2. To assess the current and future urban and industrial water demand in a closed basin, and its impact on future water availability for agriculture. 3. To assess the impact of urban water supply and sanitation investment scenarios on urban water demand, water flows and water resource availability. 4. To assess the potential of urban domestic wastewater re-use in agriculture for different urban water supply and sanitation scenarios. 5. To define a logical framework for future water demand scenarios for cities in developing countries in varying institutional and climatological settings.

Hypotheses

1. Urban growth will lead to considerable changes in irrigated agriculture in the future, in terms of wastewater irrigated area and water resource availability of existing systems. 2. The share of non-agricultural water demand in basin water demand will become significant, with implications for water availability for agriculture. 3. Any likely investment scenario will cause a rise in average per capita water demand with likely stress on water resources and agricultural water demands. 4. Urban domestic wastewater will contribute significantly to water resource availability in (peri-) urban agriculture, given that wastewater irrigation is at least acknowledged and permitted by governmental authorities. 5. Future urban water availability can be modeled with current data on urban water and the institutional and physical environment. Cities in a similar development stage and with a similar institutional and physical setting are likely to follow a similar path of change in urban water demand.

Goals

Urbanization, water demand & sanitation in large cities of the developing world -Daniel VAN ROOIJEN 3 / 6 To contribute to useful research in the area of integrated urban water management in developing countries. To contribute to improvement of the management of water resources in developing countries. To stimulate further thinking on the increasing importance of large cities in integrated water management.

Objective

To get insight into the implications of water and sanitation development scenarios for urban water demand and wastewater generation in fast growing cities across the urban-rural gradient in developing countries and develop an easy-to-use urban water model adapted to use by planners and policy makers.

MATERIAL & METHODS

The research consists of four phases. Part of the analyses and writing will be done at the university. Most of the data analyses and modeling will be done at the International Water Management Institute in Accra. Data collection, consultation of local stakeholders and field visits will take place in the selected cities; Accra, Addis Ababa and Hyderabad. The proposed time frame will be two years of field work and one year of analyses and writing at either university or research institute (IWMI).

Phase 1: Selection of cities

The analyses of two cities in Africa (Accra and Addis Ababa) and one in Asia (Hyderabad) is considered to be representative to some extent, for these continents and main characteristic of cities (coastal/inland, dry/wet climate). Criteria for selection were the present or potential severity of urban water issues, representation of a geographical context, local demand, donor preference and the advantage of present data from previous work. Also, synergy with other projects was a criterion. Selection of all cities was also influenced by the presence of office infrastructure and facilities.

Phase 2: Data collection

On site (city) data collection was done during about 3 months after literature review and the setting up of professional connections. Local expertise from persons or institutes was sought and a local research assistant was used in some cases. The use of existing contacts was considered to be advantageous. In this period, discussion with stakeholders took place to make them aware of the research and to include their knowledge.

Quantitative data is still being collected from secondary sources for use with the model. For scenario development, projections will be made based on growth patterns and future city plans. Data are population data (census), time series of urban water flows (water supply, wastewater generation, treatment, re-use), rainfall and river flow patterns into and out of the city (if required). Maps of land cover and use within the urban boundaries are required. Also statistics on urban water use patterns, from information on water supply and sanitation facilities. If statistics on water and sanitation cover seem unreliable or incomplete, then surveys can serve to get similar information. Data for the model will be compared with and supported by literature review, estimations and assumptions.

Phase 3: Urban water balance and modeling

Developing countries typically have poor existing data sets that limit the use of sophisticated hydrological models in any context. Easy-to-use software to model urban water flows provide local water planners with an effective tool to get insight in interactions between urban water management and the environment outside the urban boundaries. This can imply withdrawal from upstream reservoir and use of wastewater in agriculture downstream. For the two reasons mentioned, a simple model is effective and appropriate both for use by researchers and local planners.

Hydrological data will be presented schematically as a water balance (for general impression of city). Data will then be processed in the model. The choice of scenarios may differ by city, but same scenarios should be there for comparative analyses. Potential scenarios can be; doubling of connections to supply system, stepwise increase in coverage of conventional wastewater treatment, and installation of flush toilet and septic tank at households. Spatial boundary of the model will be the official or 'real' (peri) urban area, depending on the difference between the two and data availability. Time frame of the model will be long term (up to 30 years). An appropriate and easy-to-use model like VENSIM or UrbanCycle will be used (See [START_REF] Van Rooijen | Sponge City: Water Balance of Megacity water use and wastewater use in Hyderabad, India[END_REF][START_REF] Hardy | Integrated urban water cycle management: the UrbanCycle model[END_REF]. The model can be built manually (like VENSIM for Hyderabad, see layout fig. 1), or an existing structure can be used and adopted to the specific environment (like UrbanCycle for Sydney). Model inputs are the data collected. In the absence of local data, assumptions will be made based on other comparable cities and information from literature. For example, the per capita water use is often unreliable because of poor measurements. A realistic value can be based upon a combination of municipality reports, corresponding literature, and assumptions. The model is based on a set of relatively simple calculations that provide numbers as output (see for example fig. 2).

GW Change

Urbanization, water demand & sanitation in large cities of the developing world -Daniel VAN ROOIJEN 5 / 6 In a project following up on the PhD, the model will be adjusted and disseminated as a useable tool for local urban water planners and policy makers in any city in a comparable setting.

Phase 4: Analyses and writing

Model results of each city will be analyzed separately and compared eventually. Articles that focus on each city as well as all cities (comparative) will be published. Findings from cities and major similarities and differences between them will be compared with literature to confirm or challenge existing theories and approaches in this research domain. Research findings will be combined with research from a previous study, on the implications of city water use for water allocation and basin water management which was conducted in the Krishna Basin in India (see VAN ROOIJEN et al., 2008). This will also become part of the PhD thesis.

PRELIMINARY RESULTS AND DISCUSSION

This paper is describing ongoing PhD research. Therefore results are still in preparation and have not yet been published. An introduction to the water situation can be found in [START_REF] Van Rooijen | Exploring implications of urban growth scenarios and investments for water supply, sanitation, wastewater generation and use in Accra[END_REF] for Accra and VAN ROOIJEN & TADDESSE (forthcoming) for Addis and [START_REF] Van Rooijen | Sponge City: Water Balance of Megacity water use and wastewater use in Hyderabad, India[END_REF] for Hyderabad.

CONCLUSIONS

The aim of this research is to provide persons involved in urban water management or research with decision support and insight into urban water flows and its impact on water resources and wastewater disposal. Also, the importance of urban water management across the urban-rural gradient has not been studied much previously and this is a new research area being pioneered. Results expected from this research will therefore contribute to a better understanding of the dynamics of urban water systems and its management in developing countries.

Figure 1 .

 1 Figure 1. Copy of the VENSIM model as used for Hyderabad. Boxes display input or output components and arrows display mathematical relations.

[ 1 ]Figure 2 :

 12 Figure 2: Formulation of domestic water use [1] and wastewater generation [2].


Elizabeth Brown

Michael Sharples

Mike Sharples

Education in the Wild: contextual and location-based mobile learning in action

Scope of the workshop

The workshop explored recent innovations into location-based, or geospatially-informed, contextual mobile learning, and issues arising from them. Location-based technologies offer opportunities for new forms of learning that engage more deeply with physical surroundings and support continuity of understanding across settings; they also pose technical difficulties of modelling and maintaining continuity of context, and ethical challenges including the right to privacy of location and escape from continual monitoring. The workshop brought together experts in mobile learning from educational and technical backgrounds, with newcomers (including PhD students) across Europe and North America. The goals of the workshop were threefold:

1. to share good practice of research innovations and case studies 2. to engage in debate and discussion of critical issues surrounding contextual and location-based mobile learning both currently and in the future 3. to conduct future-scanning activities in contextual and location-based learning

Actual schedule/procedure of the workshop; main events

The structure of the workshop was as follows (for detailed time schedule and the participant list see appendix 3): Day 1: 2 Dec 09: Introduce and map the scope of 'Contextual and location-based learning' An initial introductory talk was given, from the workshop proposers, presenting the need and scope of the field, and its basis in education theory and in technical developments, followed by group discussion. Presentations of exemplar research projects from participants to indicate the spread and depth of current work in contextual and location-based learning. Each project was in the form of a structured session, where the contributors presented demonstrations of their systems and/or a video of a TEL intervention, along with a presentation of the educational aims of the project, its technical execution, and evaluation of usability and educational use. There was a mixture of mature projects and new work in progress. Group discussions, addressing the question 'What is the educational problem that contextual and location-based mobile learning is trying to solve?' followed by a plenary discussion.

Day 2: 3 Dec 09: Explore opportunities, possibilities and issues for the future A half day Future Technology Workshop (FTW, a structured workshop to 'design the future' of learning technology and activity) was carried out. Three teams with mixed education and technology expertise were set the challenge to design an innovative and educationally valuable contextual learning system of the future. The design process was captured in video and still images, as part of the FTW approach (including creation of designs, physical models, and educational scenarios) and the outcomes from each group resulted in an envisioning of technology and activity for future contextual learning.

A semi-formal debate was held, on an issue of concern in location-based technology. The issue was decided by participants during the workshop and the title was 'the delivery of contextual and location-based education results in shallower learning strategies'.

Results of the workshop a)

What has been learned -especially in respect to the Grand Challenges 'Connecting learners', 'Orchestration' and 'Contextualisation'? Connecting learners: The workshop examined ways of connecting learning across formal and non-formal settings, such as carrying out work outdoors and later reflecting upon it in the classroom. We also discussed the opportunities for lifelong learning and the fact that mobile learning creates many opportunities for informal learning, such as that in the workplace or for learners who are on the move (on public transport, for example). In this way, mobile and location-based learning provides settings in which new paradigms of education can be explored. This could possibly mean the separation of 'schooling' (which seems to be assessment-driven) and 'education' (as a more holistic endeavour). We also talked about the role of learner preferences for context-based and classroom-based learning. Another aspect is the technological opportunities for connecting learners in location, such as augmented reality platforms (e.g. Layar; Wikitude) to bring together people and artefacts from their environment, and including contexts such as history as an additional aspect. We also discussed the ethical implications of mobile learning, such as the inherent problems in being able to track users: this is a potentially more sinister aspect of 'connecting learners' and one that requires much more research. Are people happy to compromise on the availability/use of their private data if they are to benefit from this?

Orchestration: Internet users have changed from being mere receivers of web-based content to become that of 'prosumers' -both producers and consumers. The quantity of data generated each day by the general public is vast, leading to information overload and difficulties in managing such information. A key challenge will be the appropriate selection of information and the technologies that generate it, so that learners are provided with appropriate content generation and filtering mechanisms.

Contextualisation: Technology has changed dramatically in the last decade and GPS is now standard. Technologically, the world has changed, but we feel that the scientific questions haven't. Context can be created through interaction between people, settings, and artefacts. A central issue is how to create micro-sites for learning. The environment can be an active agent and resource: we can learn in environments (seizing opportunities for learning), through the environment (employing aspects of the current surroundings to structure and enrich learning), and about the environment (in environmental studies and field trips). However, we also need to look beyond just the physical environment and look at trajectories of context and how we enable transitions between contexts. In relation to the 'orchestration' theme, where we discussed information management, it is important to realise that data can change context and so it is also crucial to consider versioning and preservation of data. b) Which are some new questions and issues with respect to the Grand Challenges 'Connecting learners', 'Orchestration' and 'Contextualisation'? Connecting learners: -As society becomes more consumer-driven, how can we support a growing divergence between formal and informal education? How can we address this conflict (if one exists)?

-How can we connect formal and informal learning experiences? -What learning connections might there be, between cultures? How might educational innovations connect learners on a global scale?

Orchestration:

-Will new opportunities for personal and mobile learning prompt a transformation of schooling, or will learning in and beyond school be reconciled without any fundamental change to our education system? -How can learning outside a formal setting be managed and supported? -How do we decide what information from the WWW should be used for learning? How do we manage this data? How do we preserve it -or how do we decide what to lose? Should we be able to go back and change data? Does this mean we are altering history, and if so, should we try and capture this? How can we capture/revisit rich aspects from the past? -What are the potential of mobile devices to support assessment, both in formal and nonformal settings? -In what ways do we need to be mindful of ethical issues relevant to the changing contexts of the mobile learner such as context appropriate behaviours and responsibilities, privacy and informed consent?

Contextualisation:

-Should we model context to enable more adaptive contextual learning, or alternatively should we design rich tools to support awareness and continuity of learning across contexts? If both, then how can they be combined? -How does adding artefacts from the physical or virtual world enhance learning? c) Network building among workshop participants Expertise (both technical and methodological) was shared between participants at the workshop, including the use of video capture from a head-mounted position and also the logistics (cost/sourcing) of various sensors for location detection. Also there seemed to be research groupings that resulted from areas of common interest (mathematical location-based work; field trips; geocaching and treasure hunts), that formed organically during the workshop. Several newcomers to the area also benefitted from being introduced to both the local workshop participants and also the wider STELLAR network, throughout the Alpine Rendez-Vous event.

It is anticipated that a theme team proposal, surrounding the issue of mobile learning, will result from this workshop.


J Dupuis

E Fourmond

O Nichiporuk

F Gibaja

M Lemiti

REAR PASSIVATION SCHEMES FOR INDUSTRIAL SILICON SOLAR CELLS

Keywords: Rear side passivation, PECVD, Silicon Nitride

This article deals with the problem of the contact between dielectric layers and aluminium on the rear side of solar cells and its impact on the passivation effect. Several dielectric layers and stacks of PECVD silicon nitride and silicon oxide have been deposited on monocrystalline and multicrystalline wafers to study their passivation behaviours. Solar cells were also elaborated with a dielectric/aluminium structure on the back side and a set of wafers was used to highlight the influence of aluminium capping dielectric layers on passivation. Although it is reported that all structures give similar passivations when composed of one layer of silicon nitride at least, capping them with aluminium degrades the passivation below the standard aluminium BSF except for the SiNr/SiN stacks which is only slightly degraded. Consequently, solar cells are not as good as full aluminium coverage cells and it is found that the minority carrier lifetime and the internal quantum efficiency results are in contradiction with the I (V) results. This fact entails the need of performing complete solar cells to study the rear passivation by dielectric layers.

INTRODUCTION

During the past few years, the impressive growth of the photovoltaic industry has caused a general silicon shortage. Many solutions are brought by research laboratories and manufacturers, like the development of thin films solar cells or the reduction of thickness in standard silicon solar cells [START_REF] Agostinelli | 17 th NREL Workshop on Crystalline Silicon Technology[END_REF]. With the last solution, several problems appear, dealing particularly with the possible presence of aluminium on the rear side. In order to solve this problem, some photovoltaic laboratories developed new structures to limit the two major problems due to aluminium: low surface passivation and high bowing of the cell [START_REF] Agostinelli | 17 th NREL Workshop on Crystalline Silicon Technology[END_REF][START_REF] Schneider | 29 th IEEE Photovoltaic Specialists Conference[END_REF][START_REF] Schneiderlöchner | [END_REF][START_REF] Agostinelli | 20 th EUPVSEC[END_REF][START_REF] Janβen | [END_REF].

The common point in all these new structures is the need of a good passivation layer on the rear side to keep the efficiency similar or better to standard cells. To fulfil this aspect, deposited dielectric layers from common CVD reactor is the most promising solution, as it is already performed on the front side of industrial solar cells by PECVD for the SiN x :H antireflection coating. Unfortunately, except on bifacial solar cells [START_REF] Janβen | [END_REF][6], a single layer of SiN x :H does not work very well and alternative dielectric structures have to be developed [7][START_REF] Hofmann | 22 nd EUPVSEC[END_REF].

In this paper, we concentrated on studying rear structures containing SiO x :H (SiO) and SiN x :H (SiN) with different stoichiometries (nitrogen or silicon-rich) to investigate their passivation behaviours with and without capping aluminium. All these layers have been elaborated by Low Frequency Plasma Enhanced Chemical Vapour Deposition with ammonia (NH 3 ), silane (SiH 4 ) and nitrogen protoxide (N 2 O) as precursor gases.

EXPERIMENTAL

In order to separate the surface and the bulk influences, three sorts of substrates have been used.

•

One set of FZ-monocrystalline silicon.

•

One set of multicrystalline silicon, textured with NaOH.

• One set of multicrystalline silicon which has undergone a phosphorous diffusion followed by emitter etching. Resistivity and thickness are given in Table I. In order to study the influence of passivation layers with aluminium, we have also carried out five sets of six multicrystalline silicon solar cells of 156 cm 2 area with five different rear structures. For our structures, we have chosen single, double and triple layers based and on previous tests and on the literature. Three PECVD dielectric layers have been used: SiO, SiN and silicon-rich SiN (noted SiNr afterwards). Nine different stacks were processed, and three simple layers act as reference. Table II The total thickness of deposited layers is contained between 100 and 130nm with roughly 20 -25 nm for the first layer and 85 -105 nm for the total of the other layers in the case of multilayer stacks. These values have been confirmed after the deposition with spectroscopic ellipsometry. All the depositions have been made on both sides of the samples to allow the measurement of the minority carrier lifetime (MCL) by the photoconductance decay method. MCL measurements were performed in transient and quasi steady state mode. All results are given for an injection level of 7.10 14 cm -3 .

Finally, after the measurement, all the samples have been annealed during several seconds (RTA) at 800°C in a lamp furnace to simulate the contact-firing step and then MCL measurements were carried out again.

RESULTS FOR SILICON WAFERS

To discuss minority carrier lifetime results, we first consider the results for the monocrystalline samples and then for the multicrystalline samples to discriminate surface and bulk passivation.

Monocrystalline samples

Float zone samples lead directly to the surface passivation because of their high bulk lifetime (superior to 2ms with iodine methanol surface passivation [START_REF] Schönecker | 14 th EUPVSEC[END_REF]). Concerning single layers, SiN and SiNr have a better passivation effect than SiO. SiNr leads to the best passivation results among these layers with or without the rapid thermal anneal (RTA). For double layers, the best results are obtained with SiNr as the first layer and show comparable values to the silicon nitride single layer. The other stacks with SiO as first layer have intermediate results between SiN and SiO alone. As observed for single layers, the RTA is still beneficent for the passivation quality for all stacks. Finally, the triple layers show the same trend than the double layers with a slightly better result for the structure containing SiNr.

PECVD hydrogenated silicon nitride is well known to be a great passivation layer, contrary to PECVD silicon oxide. Moreover, the better passivation of SiNr has been already demonstrated by a lot of authors [START_REF] Aberle | [END_REF][START_REF] Charifi | 21 st EUPVSEC[END_REF][START_REF] Lelièvre | th IPSEC[END_REF].

The field effect created by the SiN layer is always strong whatever the stoichiometry of the film is (Qf in the layer is reported to be higher than 10 12 cm -2 , sufficient to put the silicon surface in weak inversion) [START_REF] Lelièvre | th IPSEC[END_REF]. Thus, the better passivation must come from a smaller interface states density at the SiNr/Si interface due to higher Si-H bonds than in N-rich SiN. In fact these bonds are strongly correlated to the passivation quality and can be used as surface passivation indicators [START_REF] Mäckel | [END_REF]. We can also expect fewer constraints and consequently fewer defects, at the interface between a Si-rich SiN layer and a silicon substrate [15].

Lastly, the RTA gives the opportunity for the SiN layers to free a part of their atomic hydrogen which can bond to some remaining interface defects. But this release does not happen necessarily in the same way for both layers. In the case of SiNr, the hydrogen will be released in high quantity due to the high number of easily break-able Si-H bonds. However, the low density of the film will allow more H 2 molecules to form and might be destructive for the layer itself if these molecules accumulate and pierce the film (also called blistering effect [START_REF] Fourmond | [END_REF]).

Concerning the SiO single layer, the passivation remains weak. Moreover, contrary to our previous work on hydrogenated silicon oxynitride [16], the RTA does not improve the minority carrier lifetime. This fact leads us to conclude that the RTA profile (different from [16]) is a critical parameter for the passivation effect of deposited SiO.

The excellent surface passivation of the double SiNr/SiN layer is explained by what we have developed for single layer: the high surface passivation by SiNr and the high hydrogenation during the layers deposition and the RTA which results in a passivation as good as SiN single layer. The SiNr/SiO layers bring a reasonable MCL with only 20nm of SiNr that appears to be sufficient for surface passivation.

For the two stacks layers with SiO as first layer (SiO/SiN and SiO/SiNr), the passivation effect comes mainly from the diffused hydrogen of the SiN layer during the deposition and during the annealing [START_REF] Agostinelli | 17 th NREL Workshop on Crystalline Silicon Technology[END_REF][START_REF] Hofmann | 22 nd EUPVSEC[END_REF]. Thus, a large improvement with the RTA is expected but it is not clearly evident on Figure I. The low MCL value of SiO/SiN might suggest a problem during the wafer's cleaning before the deposition. Although we can observe an improvement after the RTA, the final passivation remains low. Conversely, in the case of SiO/SiNr, MCL of the as-deposited layer is quite high but stays constant after the RTA. These results would suggest no more surface hydrogenation during the RTA due probably to the hydrogen exodiffusion.

Finally, both triple layers confirm the tendency observed on the double layers with slightly higher passivation results. This time, the MCL results seem to confirm a better hydrogenation of the interface during the deposition of SiNr layer and more atomic hydrogen diffusion from the SiN layer through the SiO layer during anneal. The capping of the silicon nitride layers by SiO seems to improve slightly the surface passivation probably by limiting the hydrogen exodiffusion out of the layer.

In a nutshell, the double and the triple layers are less efficient than the single silicon nitride layer for surface passivation. But the situation becomes more complex on the multicrystalline samples with the necessity of improving the bulk passivation by hydrogen diffusion. This passivation is also completed by the gettering effect during the phosphorous diffusion step. We have thus to discriminate both contributions in our experiments by studying samples with and without gettering.

Multicrystalline samples

The comparisons and discussions between the samples with and without gettering have to be made carefully because of they are not neighbouring wafers.

The minority carrier lifetime results are inferior to the monocrystalline samples but the differences between the various passivation schemes tested remain the same before RTA. On the other hand, the impact of the annealing step is a major benefit for the multicrystalline samples especially when the first layer is SiO. Thus, except the SiO single layer, the other structures show good passiva-tions after RTA with slightly superior results for the all silicon nitride stacks.

For the samples with gettering the tendency is globally the same. The gettering effect seems to be independent of the hydrogenation effect except, maybe, for the SiNr layer. After annealing, the improvement is in the same order as samples without gettering. The slightly identical behaviour before annealing can be explained in the same way as for the monocrystalline wafers. After deposition, the surface passivation enhancement seems to be the main contribution to the effective MCL: the structures with silicon nitride at the interface exhibit the best results.

The strong MCL improvement after the RTA points out an effective bulk passivation due to the hydrogen diffusion from SiN (or SiNr) layers into the bulk. The rear passivation structure needs only one layer of silicon nitride for the bulk hydrogenation to happen. This fact is highlighted with SiO/SiN: a poor MCL before annealing (bad surface passivation) but similar results as the other structures after the RTA (bulk enhancement).

Identical tendency between the samples with and without gettering were expected. The hydrogen will essentially help to fill dangling bonds or lattice dislocations whereas the gettering will contribute to reduce recombinations created by metallic impurities [17]. Thus, both treatments have complementary actions.

To sum up, it seems that for the multicrystalline samples, all studied stacks give less contrasted minority carrier lifetime results (except SiO) due to a predominant bulk passivation by hydrogen. Thus, it may be better to choose a structure in order to optimise another parameter of the back side (reflection for example).

To complete these results, we have studied the influence of the aluminium deposition on these structures to look for the degradation problem due to the formed Metal-Insulator-Semiconductor (MIS) structure.

RESULTS FOR MIS STRUCTURES

Several sets of 156 cm 2 multicrystalline silicon substrates with no emitter were used for minority carrier lifetime purpose. This time, dielectric layers and screen printed aluminium were applied on different half on the back side as shown in Figure IV for the case of the SiN layer. The same dielectric layers were also deposited on the front side. Next, all these structures have been annealed in a belt furnace from Photowatt. The temperature profile for this annealing step differs from the one used at INL. Finally MCL were performed using µPCD.

Solar cells with dielectric stacks passivation were also elaborated for spectral response and I (V) measurements purpose. A set of cells with a full aluminium coverage stand as references with also a thick SiO/SiN structure which has already given good solar cells results [START_REF] Agostinelli | 21 st EUPVSEC[END_REF]. The passivation results and the normalized internal quantum efficiency (IQE) are plotted in Figure V The dielectric layers without aluminium achieve the best passivation in all structures compared to the aluminium back surface field (BSF). After aluminium capping, all MCL decrease and become inferiors to the aluminium BSF except for the SiNr/SiN stacks. The SiNr/SiN structure gives also the best passivation and stays unaffected, contrary to the triple layers where SiNr has been degraded by the RTA. It is probably due to a thinner SiNr layer and a thicker capping layer in the SiNr/SiN structure.

As it was already mentioned by IMEC [START_REF] Agostinelli | 4 th IEEE WCPEC[END_REF], capping the dielectric layer by aluminium creates a MIS structure and consequently might affect the SRH recombination by inducing a weak inversion at the surface of the substrate. Although this explanation might work, it is insufficient to explain why SiNr/SiN is so good compared to SiN with aluminium coverage (Figure V). Moreover, if we consider the IQE results (Figure VI) we can see that, surprisingly, every passivation schemes give clearly worse results than standard full-coverage aluminium even for the SiNr/SiN stacks. V oc remain strongly below the two references with fullcoverage aluminium and thick dielectric stacks. Thus they are in agreement with the poor rear side passivation indicated by the IQE except for the stacks with thick SiO/SiN layers. In fact this structure gives a I cc and a V oc nearly as good as aluminium BSF, but leads to poor IQE results. This contradiction remains hard to explain; it might be cause by the measurements done at different injection levels in each case. In any case these characterizations seem unable to give correct information about the final solar cells behaviours. Thus, these results point out the necessity of processing complete solar cells to performed any passivation comparison.

Moreover, as it was already mentioned, we can conclude that the aluminium seems to play an important role in the efficiencies degradations by creating a MIS structure. The interaction between the MIS and the density of positive dielectric charges are probably other factors which have to be taken into consideration to explain correctly the I (V) results.

SUMMARY

In the present article, we have studied different rear passivation schemes on monocrystalline and multicrystalline substrates in order to choose the most promising structures. Although the all silicon nitride layers give slightly best results, capping them with aluminium entails inferior results than standard aluminium BSF, except for the SiNr/SiN stacks which is only slightly affected by the aluminium. The contradictions between the MCL results, the IQE and the I (V) measurements lead to the need of studying the rear structure using complete solar cells. Lastly, the most promising structure on the final solar cell seems to be a combination of thick layers as it was already mentioned by the IMEC.

Figure I :

 I Figure I: Minority carrier lifetime on monocrystalline samples of the nine rear passivation schemes tested. N, Nr and O stand for SiN, SiNr and SiO.

Figure II :

 II Figure II: Minority carrier lifetime on multicrystalline samples without gettering of the nine rear passivation schemes tested.

Figure III :

 III Figure III: Minority carrier lifetime on multicrystalline samples with gettering of the nine rear passivation schemes tested.

  and Figure VI.

Figure IV :

 IV Figure IV: Cartography of minority carrier lifetime for rear passivation with a single SiN layer.

Figure V :

 V Figure V: Minority carrier lifetime for different rear passivation state.

Figure VI :

 VI Figure VI: Normalized IQE for the five rear passivation schemes, for the standard aluminium BSF and for a thick O/N passivation stacks (Ot/Nt). The normalization has been made with the calibration sample of the quantum efficiency device.

Figure VII :

 VII Figure VII: The I cc and The V oc of the final cells performed with a dielectric rear side passivation. To settle the argument, I (V) measurements have been performed on final cells. As shown in Figure VII the short-circuit current I cc and the open circuit voltageV oc remain strongly below the two references with fullcoverage aluminium and thick dielectric stacks. Thus they are in agreement with the poor rear side passivation indicated by the IQE except for the stacks with thick SiO/SiN layers. In fact this structure gives a I cc and a V oc nearly as good as aluminium BSF, but leads to poor IQE results. This contradiction remains hard to explain; it might be cause by the measurements done at different injection levels in each case. In any case these characterizations seem unable to give correct information about the final solar cells behaviours. Thus, these results point out the necessity of processing complete solar cells to performed any passivation comparison.Moreover, as it was already mentioned, we can conclude that the aluminium seems to play an important role in the efficiencies degradations by creating a MIS structure. The interaction between the MIS and the density of positive dielectric charges are probably other factors which have to be taken into consideration to explain correctly the I (V) results.

Table I :

 I Characteristics of samples used in this study.

	Properties	Monocrystalline Multicrystalline
	Size	2 inches	62.5x62.5 mm 2
	Type	P	P
	Resistivity	4-7 ohm.cm	1 ohm.cm
	Thickness	280µm	300µm

Table II :

 II Rear structures tested.

			summarizes these rear
	passivation schemes.
	Single	Double	Triple
	SiN	SiNr/SiN	SiO/SiN/SiO
	SiNr	SiNr/SiO	SiO/SiNr/SiO
	SiO	SiO/SiN
		SiO/SiNr

ACKNOWLEDGEMENTS

The authors want to thanks the entire Nanolyon platform staff for its technical contribution and the National Research Agency (ANR) with the French Agency for Environment and Energy Management (ADEME) for their financial contributions through the national projects PHARE and REDUCOP II.