Stavros Demetriadis

Antonios Barbas

Anastasios Molohides

George Palaigeorgiou

Dimitris Psillos

Ioannis Vlahavas

Ioannis Tsoukalas

Andreas Pombortsis

"Cultures in negotiation": teachers' acceptance/resistance attitudes considering the infusion of technology into schools

Keywords: Media in education, country-specific developments, secondary education, improving classroom teaching, learning communities

come   L'archive ouverte pluridisciplinaire

In many countries the introduction of Information and Communication Technology (ICT) into schools has been praised as the necessary (although not without problems) course of action for the qualitative improvement of teaching and learning methodology (OECD, 2001;OECD, 1999[START_REF] Sinko | The challenge of ICT in Finnish education[END_REF][START_REF] Pedretti | Teaming technology enhanced instruction in the science classroom and teacher professional development[END_REF]. Introduction of ICT in school is considered a necessity premised on economic, social and pedagogical rationales (OECD, 2001) and many governments have launched major programs and invest substantial capitals to support ICT-in-Education projects [START_REF] Pelgrum | Obstacles to the integration of ICT in education: results from a worldwide educational assessment[END_REF]. According to OECD (1999) this was estimated at an annual figure of US$16 billion for OECD countries across primary, secondary and tertiary education, which could well have grown significantly since. These efforts are expected to foster the important educational reform that ICT based teaching can hopefully support by transforming students into active knowledge constructors, a more appropriate profile for the citizens of the Information Society. It is expected that "an educated citizen in the year 2020 will be more valuable as an employee because he or she will be able to produce more builders of theory, synthesizers, and inventors of strategy than valuable as an employee who manages facts" (Di Sessa, 1998 cited by [START_REF] Hamza | Technology and Education: Between Chaos & Order[END_REF].

At the same time though a dissatisfying phenomenon is recorded: teachers in schools do not appear to make effective use of information technology for teaching [START_REF] Zhao | Teacher adoption of technology: a perceptual control theory perspective[END_REF]Stetson & Bagwell, 1999;[START_REF] Pedretti | Teaming technology enhanced instruction in the science classroom and teacher professional development[END_REF][START_REF] Ely | Computers in schools and universities in the United States of America[END_REF]OECD, 2001). It seems that teachers' attitudes regarding ICT use in schools not only pose difficulties in the use of technology per se but also cancel the learning benefits expected to spring from the instructional reform. Teachers are characterized as being 'technophobic' about using ICT [START_REF] Rosen | Computer availability, computer experience and technophobia among public school teachers[END_REF][START_REF] Brosnan | The fourth 'R': Are teachers hindering computer literacy in school children?[END_REF]cited in Selwyn, Dawes & Mercer, 2001). They are also blamed of resisting instructional reform being rooted in improper pedagogical beliefs [START_REF] Gillman | Change in public education: A technologicl perspective[END_REF][START_REF] Finlayson | Turning skeptics into missionaries: The case for compulsory information technology courses[END_REF]cited in Selwyn, Dawes & Mercer, 2001), which leads to a considerable variation of the supposed uniformly designed instructional reform efforts. The overall picture seems to be that introduction of ICT in schools, although long awaited and strongly supported, encounters significant problems related to the attitudes of the people who are responsible for its use in the classroom.

In this paper we present and discuss our experiences from the effort to train teachers in the Greek secondary schools to use ICT as a means for teaching broader subjects of the curriculum. In the Greek context this program is innovative in two aspects. First, it is the first time ever that teachers are offered training regarding the use of specific educational software, thus advancing the use of ICT in the curriculum. Second, teachers' training is being delivered in school by teacher-mentors, i.e. their colleagues who, after attending a postgraduate specialization course, administer training sessions in schools appropriately equipped with computer laboratories.

Our work focuses on the acceptance and/or resistance attitudes that teachers develop regarding the training efforts and their initial attitudes towards incorporating ICT in their teaching practice. In the following we first outline the general picture concerning the use of ICT in schools worldwide as documented in the literature. We then present our recorded data and try to reach a deeper understanding by connecting to a consistent theoretical framework emerging primarily from the application of a model premised on Perceptual Control Theory. This framework also comprises our supported view that the infusion of ICT into school can be conceptualized as initializing a negotiation process between cultures.

ICT in schools

How is ICT used in schools?

Research data indicate that the introductory step for computers in school is using them in administrative tasks and not as part of the learning process (McCannon & Crews, 2000, p. 111). There is also indication that teachers proceed to adopt ICT in stages. [START_REF] Wells | Teachers' stages of concern towards Internet integration[END_REF] (cited in [START_REF] Myhre | I think this will keep them busy: computers in a teacher's thought and practice[END_REF] report that teachers initially focus on their own interaction with the new medium and as they gradually become comfortable with the technology they start deliberating upon potential learning benefits that would result from the use of the computer. [START_REF] Myhre | I think this will keep them busy: computers in a teacher's thought and practice[END_REF] concludes that this increased familiarity with computers allows teachers to turn their interest in the pedagogical use of technology (rather than its operational issues) but also emphasizes that such change processes do not occur rapidly and are not easily achieved.

Two important questions are relevant at this point: (a) "Under what conditions are teachers motivated to use technology in classroom?" and (b) "What kind of instructional methodology do teachers employ after deciding to use technology?". [START_REF] Zhao | Teacher adoption of technology: a perceptual control theory perspective[END_REF] formulate an answer to the first question, employing a Perceptual Control Theory (PCT) based perspective. According to PCT [START_REF] Powers | Behavior: The control of perception[END_REF][START_REF] Mcclelland | The collective control of perceptions: constructing order from conflict[END_REF] the activation of purposeful agents (such as human beings) towards success of their objectives occurs when a discrepancy is detected between what is perceived and what is internally established as a standard of reference. Discrepancies could emerge either as a result of changing perceptions about conditions of external environment or as an alteration of internal reference condition. As a result the individual begins to vary his/her behavior trying to minimize the discrepancy and its consequences. These internal reference conditions should be conceptualised as parts of a hierarchy where higher-level conditions (goals) are the most important to maintain. Many times people accept to alter lower level goals in order to succeed stability in higher levels. An interesting element of the PCT approach is that human perception about environmental conditions (and not human behaviour) is considered as the independent variable that defines further human actions which have as objective to minimize detected discrepancies. Following this perspective three conditions are identified as necessary for teachers to use technology (Zhao & Cziko, 2001, p. 27):

1. Teachers must believe that technology can more effectively achieve or maintain a higher-level goal than what has been used ("effectiveness"). 2. Teachers must believe that using technology will not cause disturbances to other higher-level goals that they evaluate as more important than the one being maintained ("disturbances") 3. Teachers must believe that they have the ability and resources to use technology ("control").

It is therefore supposed that if these conditions are met, teachers will introduce ICT in the classroom. But how do they actually teach when using ICT? Available data are rather disappointing. Overall there is evidence that ICT is assimilated into the established way of teaching without teachers making effective use of the new medium to create innovative learning experiences. "Technology use reflects traditional classroom methodology, though affording some increased attention to the individual learner. It still depends too much on highly motivated pioneering principals and teachers" (OECD 2001, p. 67). Also "information technology in the classroom is used in an ineffective way and it has been proved difficult to integrate within traditional curriculum settings" (Jules Van Belle & Soetaert, 2001, p. 38).

What are the obstacles for using ICT in schools?

There are many elements identified as obstacles in the way of introducing ICT in schools. [START_REF] Pelgrum | Obstacles to the integration of ICT in education: results from a worldwide educational assessment[END_REF] presents a list of ten such issues that educational practitioners perceive as serious impediments for realizing their ICT related goals.

From this group of ten we isolate the three major ones: (1) insufficient number of computers, (2) teachers' lack of knowledge/skills, and (3) difficult to integrate in instruction. [START_REF] Ely | Computers in schools and universities in the United States of America[END_REF] similarly distinguishes as major conditions, relevant to ICT implementation, the following: (1) dissatisfaction with the status quo, (2) existence of knowledge and skills, and (3) availability of resources.

The two taxonomies identify, more or less, the same issues: Ely's "existence of knowledge and skills" relates to Pelgrum's "teachers lack knowledge/skills". Also Ely's "availability of resources" is relative to Pelgrum's "insufficient number of computers". Finally Ely's "dissatisfaction with the status quo" is directly related to what [START_REF] Zhao | Teacher adoption of technology: a perceptual control theory perspective[END_REF] describe as "discrepancies that activate individual". The issue of teachers' confidence in their ICT competence as a major factor for integrating technology in teaching is reported in other studies as well. [START_REF] Mooij | Modelling and supporting ICT implementation in secondary schools[END_REF] state that "if teachers are not confident in their ability or competence to handle computers this may hamper their willingness to introduce technology in their classroom". In an international study [START_REF] Smeets | The impact of information and communication technology on the teacher[END_REF]cited in Mooij & Smeets, 2001) it is also reported that the most important reason teachers mention for not using ICT is that they are not familiar with ICT or they feel unsure about it. This ICT competence factor is the same that [START_REF] Zhao | Teacher adoption of technology: a perceptual control theory perspective[END_REF] refer to as "Control Principle".

Some other important factors are also recorded as significantly influencing ICT use in schools. Teachers claiming to follow more innovative educational practices (use of inquiry, project-oriented work, hands-on activities) are more likely to use new technologies than those who stick to the more traditional instructional approaches. [START_REF] Honey | Teachers' beliefs and technology integration: Different values, different understandings[END_REF]cited in Myhre, 1998). According to [START_REF] Mooij | Modelling and supporting ICT implementation in secondary schools[END_REF] school manager's policy and budgetary decisions and in general the attitude of the school manager (their commitment and decisions) are expected to be relevant to the ICT innovation process.

As a conclusion, there are three major issues repeatedly identified by research as important for introducing ICT into the classroom. Namely: 1. "Control" (possessing working knowledge of ICT, being confident, having control over technology) as an enabling and psychologically reassuring factor. 2. "Resources" (number of available computers) as an enabling factor. 3. "Inner dissatisfaction" (dissatisfaction with the current status) as a motivating -activating factor. technology studies and pedagogical studies. In these lessons teachers attended lectures and participated in laboratory sessions, group discussions and team project works. Academics and other highly specialized educators delivered these lessons emphasizing ICT supported didactic approaches (interactive learning environments, project based learning, collaborative learning). Didactic phase was followed by an apprenticeship phase. Teacher-mentors visited selected schools in the area of Thessaloniki and, supervised by their course tutors, delivered initial training sessions to the school teachers. This phase lasted 4 weeks and teacher-mentors worked in small groups of mixed specialties. So they were introduced to the practical aspects of their mission and gained hands-on experience. During next semester (spring semester) teachers-mentors were assigned practice work. Each one of them was responsible for three of four selected schools (in various places of Northern Greece) and administered in-school training sessions for the respective school teachers. Their main objective was twofold: (a) training of teachers of the same specialization (e.g. Physicists) to the use of domain specific educational software including off-line and on-line resources (we call these sessions "vertical training" since it addresses the needs of teachers in specific domain), and (b) the training of every interested teacher to general ICT tools (i.e. word processors, electronic presentations, Internet communication, etc.) ("horizontal" training sessions). During their practice teachermentors were supported by their PGS course tutors using Web based communication facilities (discussion forum and email). Teacher-mentors were regularly reporting to their tutors on the progress of their training efforts and discussed (both with tutors and other colleague mentors) the problems encountered and any possible solutions. Teacher-mentors are currently being involved into supporting in-school training projects and transmitting their ICT experience to their colleagues.

Introduction of technology into Greek secondary schools

This model of peer to peer training has been reported to be followed in other cases as well resulting to positive outcomes [START_REF] Trushell | Insights into ICT capability on a teachermentored PGCE course[END_REF]OECD, 2001). It is worth noticing that a question related to this model is whether teacher-mentors will be removed from their school didactical duties in part or completely. In countries such as Canada and Norway mentors are assigned a reduced teaching load while they go on supporting their colleagues (OECD, 2001). Contact with classroom offers them the chance to increase their experience, deepen their understanding and develop new methodologies. At the moment teacher-mentors in Greece are only assigned to offer training to their colleagues. However they still get the chance of using ICT in classroom teaching in cooperation with their trainees.

Research Data

Selection of Schools and their characteristics

Data presented in this study come from the schools where teacher-mentors delivered training sessions in their apprenticeship and practice work. Schools were selected depending on their infrastructure availability. Apprenticeship took place in 5 Gymnasiums in the area of Thessaloniki, which were selected by PGS course administration. Practice lasted three months and schools were selected by the Ministry and the CTI. Overall 39 Gymnasiums (including those in apprenticeship), 17 Lyceums, and 4 Technical Institutions participated and accepted teacher-mentors. Some of these schools were also in the area of Thessaloniki but many others were located in other municipalities of Northern Greece as well.

All training took place in school computer laboratories. Such a typical laboratory includes a number (12) of personal computers connected in local network with educational software preinstalled. The local network is connected to the Internet by means of a router and a proxy server. Printer and a video projector are also available.

The voluntary basis of the whole project should be emphasized. Teacher-mentors were volunteers and so were their trainees in the schools. Teachers who attended training sessions had no further obligation to include ICT in classroom and apply any proposed methodologies. Only voluntarily (and experimenting) some of them brought their students into the laboratory and (in cooperation with teacher-mentors) delivered ICT supported lessons.

Data collection methodology

Data were collected employing a threefold methodological approach: (a) naturalistic inquiry, (b) teacher-mentors' reports, and (c) teacher-mentors' interviews. In detail: a) Naturalistic inquiry: While in apprenticeship teacher-mentors visited the selected schools and organized initial training sessions, supervised by their PGS course tutors. Supervisors recorded and processed data that emerged from this early training experience.

b) Reporting: During their practice teacher-mentors were regularly reporting to supervisors about their on-going training efforts in the schools of their responsibility. These reports were analyzed and offered substantial data for the overall picture of teachers' acceptance/resistance attitudes. c) Interviewing: After completing their practice work teacher-mentors were interviewed to record available information regarding their experiences in schools. Many of these interviews were videotaped and in depth processed (i.e. categorizing experiences and interpretations thus resulting in the identification of common themes). These data being correlated to previously recorded data offered opportunity for confirmation and fine-tuning of resulting conclusions.

Recorded Data 2

• Teachers' participation and training schedules Table 1 presents the number of teachers who attended "horizontal" training sessions (addressing every interested teacher).

{insert Table 1 here}

Table 2 presents the number of Physics and/or Literature teachers who attended "vertical" training sessions (domain related educational software presented to teachers of these two specialties).

{insert Table 2 here} Table 3 presents the weekly mean workload per teacher-mentor.

{insert Table 3 here} In the last column "ICT in classroom" denotes ICT supported teaching sessions for students, administered cooperatively by teacher-mentor and teacher-trainee. There were totally 73 such sessions delivered by 10 teacher-mentors and trainees within a time interval of 9 weeks.

An important task for teacher-mentors was to find the necessary time in schools to organize training sessions. Here are the solutions they followed: a) In 25 schools: Teachers attended in groups whenever their lesson timetable permitted, one day per week or two days every second week. b) In 14 schools: Teachers attended in groups, in the afternoons, after school was over. c) In 9 schools: Teachers attended in groups in three-hour shift (after rearranging the lesson timetable), one or two days per week. d) Finally in 12 schools training sessions were organized in some other way (e.g.: individual training whenever teacher was available or after school was over).

• School headmasters' attitudes Perhaps the major problem that teacher-mentors encountered was to be accepted by regular members of school life and find necessary resources for their training efforts (taking into account that the whole training project was on a voluntary basis).

Teacher-mentors unequivocally agreed that the role of school principal had been very important to this aspect. There are however both positive and negative attitudes recorded regarding their behavior. Positively acting headmasters intervened and offered solutions to problems or newly encountered situations (e.g. allocated appropriate time for training sessions, encouraged school teachers to attend, enhanced perceived importance of the project) (E1). On the other hand some of them exhibited a bureaucratic (and even possessive) attitude in the management of school resources, especially in relation to the availability of computer laboratory and they were generally unwilling to help and facilitate teacher-mentors (E2).

One could identify three distinct attitudes in school headmasters' behavior (E3): (a) Supporting (mainly in Gymnasiums and in Lyceums with former experience in innovative programs), (b) reluctant to support (mainly in Lyceums for the reason of not disturbing school timetable and students' preparation for their examinations) (c) initially negative turning afterwards to positive (mainly in Gymnasiums).

• Teachers' interest

Many teachers did care to learn how to use computers (especially in schools with experience in innovative projects) and were very positive towards training (E4). As documented in Table 1 more than half of the teachers working in selected schools attended the training sessions. In some cases teachers from neighboring schools (initially not included in the training program) asked the teacher-mentor to accept them for training as well. According to teacher-mentors the great majority of teachers were completely novices and their interest focused primarily on the use of commonly used software tools (E5) (word processor, spreadsheet, Web browsers, search machines). A great part of the training sessions addressed these initial needs as shown in Table 3 ("horizontal" sessions). Comparatively less teachers attended the "vertical" sessions (see percentages in 'total' rows of Tables 1 &2) where the use of domain specific educational software was presented (e.g. "Interactive Physics" for Physicists). (E6)

However some teachers (usually carrying negative experience from previous rather unsuccessful training efforts) tended to perceive this new training as just another Informatics seminar or computer use workshop (E7). According to teacher-mentors it had been necessary to better inform teachers about the objectives of such a training program aiming to introduce ICT in the curriculum.

Teachers were sometimes reluctant to participate in seminars outside of their timetable, as this would be a significant additional workload for them (E8). Some (including those who were about to get retired) were not interested and did not feel that ICT had anything to do with them (E9). In cases it was recorded that teachers were gradually losing their initial interest and withdrew from training sessions, somehow developing a "that's all" feeling about ICT and believing that they had mastered everything there was to (E10).

It is characteristic that many teachers asked for an official acknowledgment of their learning efforts, some kind of a certificate for their personal files. Teacher-mentors also expressed the opinion that a more official program profile could greatly enhance perceived importance of the training, participation and commitment (E11).

What should not go without noticing is that there was no ideological opposition to ICT school introduction. Teachers seem to have totally accepted the necessity of ICT use in teaching (only some of them expressed worries about possible Internet misuse by students). This thesis though is a tentative one since many other teachers did not attend training sessions and they might have expressed different opinions if given the chance (E12).

Finally it is worth mentioning the role of Informatics teachers. They often offered technical support (E13) but sometimes problems arose in the sharing of resources (e.g. timetable of computer laboratories).

• Lower vs. Upper Secondary Schools

Teachers in Lyceums (upper secondary) appear to be higher motivated than their Gymnasium colleagues, to explore instructional benefits that ICT could offer. Students' examinations for entering tertiary education is an important objective in Lyceums and imposes greater teaching demands which in turn activate teachers to explore ICT based teaching. In Gymnasiums teachers' interest in ICT seems to depend much more on personal motivation. However Lyceum teachers in several occasions expressed their doubtfulness as to the extent and the mode of ICT use that could better support their teaching objective. Overall there seems to be a selective and adaptive process in action, i.e. teachers trying to identify modes of ICT use compatible to the established teaching and learning methodologies (which are valued as better accommodating the already set instructional target) (E14) (we extensively comment on this point at the "Discussion" section of this paper).

Moreover Lyceum teachers are assessed by teacher-mentors as hard workers provided they are convinced for the necessity and the benefits of ICT based teaching, exactly because they are accustomed to higher teaching demands and greater workloads (E15).

• Lack of ICT competence -Mode of use Teachers in many cases expressed their lack of confidence when confronting the new teaching demands imposed by ICT. They avoided implementing by themselves pilot ICT teaching but several tried and delivered lessons in cooperation with their teachermentors (E16). We briefly present and comment on two representative cases.

In a most interesting case (E17) a Lyceum literature teacher along with her teachermentor delivered an ICT supported lesson where students located literature texts in Web sites, processed them and emailed authors their questions and opinions (one of the authors was invited and joined the classroom in a later session). The whole event was evaluated as innovative, significant and satisfying by teachers and students and was even supervised by the literature school advisor of the prefecture. The teachermentor expressed the opinion that interweaving of medium and content helps novice users develop more easily ICT competences while working in the context of specific problems. She also noticed that her objective was to create a kind of "literature workshop" that would engage students into various activities using ICT as a communication and information processing tool. What should also be emphasized is that ICT was used to support the already established school objectives (i.e. engaging students into literature text processing and production of their own original text).

In another case (E18) the rather restrained use of ICT was demonstrated in the teaching of Gymnasium Physics. The teacher had his students seated in groups in front of the computers and presented "Interactive Physics" software using a video projector. He asked students to follow instructions in their worksheets to set up a simple motion experiment. Students were not given the time to discuss in their group, experiment and formulate their own explanations. Whenever a question came up they turned to the teacher to receive explanations and answers. It was a kind of ICT supported teacher-centered teaching; an adaptation of ICT tools to the traditional mode of school teaching directly connected to established teacher's conceptions of how students should learn.

• Technical support

The project initially employed a technical support scheme provided by independent cooperating vendors. However there was no technical support whenever needed. This caused significant problems to teacher-mentors (E19). A teacher-mentor's statement: "technical problems made me follow another route" indicates flexibility but also possible disorientation of training objectives. Teacher-mentors proposed that trained informatics teachers could possibly offer technical support.

• Acceptance -Adaptation Schools where apprenticeship took place were more easily and efficiently adapted to the needs of the practice phase. Teacher's prior experience helped developing a positive attitude towards the further acceptance of the training program. This "acceptance -adaptation" effect had been observed in another instance as well: in schools (Gymnasiums and/or Lyceums) having participated in other innovative educational projects teachers were reportedly more open to accept and participate in their in-school training programs (E20).

Teachers also consistently tried to identify (by asking teacher-mentors or searching in software tutorials) in what degree the presented educational software was compatible to the existing curriculum. This seems to have been their first attempt to accept and adapt software usage in terms of the school culture (E21).

• Support

Teacher-mentors expressed in many cases their need for continuous and multifaceted support (continuous training on various issues of educational technology, information about new software, communication, meetings and conference attending, accessibility to data bases of didactic proposals and successful approaches sorted according to curriculum contents) (E22).

Another proposal by teacher-mentors was that their job would have been more effective if they could become members of a specific school community, being responsible for ICT training and support for this specific school instead of many (something that worn them out by imposing on them too many time consuming tasks) (E23). This exclusive connection between teacher-mentor and school has been supported in many cases by the strategy followed in various countries, which "encourage the designation of one teacher within the school as ICT coordinator, to act as the central reference point for ICT issues and support colleagues on technical and pedagogical issues" (OECD, 2001, p. 94)"

• Flexibility

Teacher-mentors acknowledged that a considerable degree of flexibility was necessary for dealing with various problems resulting from the more rigid form of traditional school structure as well as other unexpected situations that eventually came up. Adapting the training schedule, forming teacher groups, addressing the needs of each specific trainee, even dealing with hardware failures, were few of the resource management issues that teacher-mentors found themselves deeply involved in (E24).

Discussion

Theme I: "Necessary Conditions for using ICT"

To better understand teachers' behavior we refer to the conditions set by the model by [START_REF] Zhao | Teacher adoption of technology: a perceptual control theory perspective[END_REF]. The model is premised on the notion of a perceived reality dynamically evolving due to actions of purposeful human agents who aim to satisfy their need for balance between perceptions of outside world and inner hierarchy of goals. In order for teachers to introduce ICT in classroom they must feel that: a) Using ICT maximizes effectiveness for achievement of higher level goals, b) Using ICT does not cause disturbances to other higher order goals, and c) They have the ability to control technology.

a) "Effectiveness": What are the goals that teachers do care to achieve when using ICT?

An important higher order goal for all professionals is competence in their job. Teachers seem to have totally accepted the rationale for using ICT in school (see E12), realizing that technology slowly but steadily finds its way into the educational setting. They are interested therefore in attending training programs (E4) for attaining a more competent level for themselves and they wish to certify this knowledge (E11) in order to present a better professional image in any future teacher assessment. In case this competence factor was valued insignificant (E9, E10) their interest in training was minimized. Enhancing, therefore, their professional status is an important motivation for teachers to learn about ICT. It is worth noticing that many more teachers expressed initial interest to receive introductory training on general ICT tools as opposed to domain specific training on the use of educational software (E5, E6). This and other recorded facts (E7, E8) indicate that for many teachers "ICT competency" means primarily to know the basics about the computer. Since for the moment it remains unclear how exactly ICT is going to be used for teaching and what kind of skills and knowledge will be necessary, many teachers set accordingly as main objective what is already clearly identifiable (i.e. technology knowledge and skills at a basic level) and leave the rest for when they feel as more appropriate. It should also be noticed how teachers' willingness to receive training was influenced (positively or negatively) by in-school factors, such as headmasters' attitude (E1, E2, E3) or other introduced disturbances (E7, E8).

Of course another important goal that teachers are interested to achieve when using ICT is enhancing the quality of their teaching. This is more clear in the case of Lyceum teachers where teaching demands are higher (E14, E15). However "quality of teaching" is something evaluated in relation to the established school objectives.

Teachers' effort, in principle, focuses on producing the kind of learning compatible to these objectives. This eventually results to an effort of adapting ICT to the traditional teacher-centered methodology which is perceived as successful at producing the expected learning outcomes. This brings us to the second condition of the model, namely teachers' efforts to minimize any "disturbances" resulting from the use of ICT.

b) "Side effects": What are the ICT caused disturbances that teachers do care to minimize?

Higher secondary teachers appear to be more explorative and selective, compared to their lower secondary colleagues, as to how they are going to use ICT in classroom (E14). These efforts aim at identifying those qualities and modes of use of educational software that would enhance students' learning according to school perspective. So, teachers being strongly oriented to fulfilling the already established school instructional targets tend to ignore any innovative learning activities (e.g. collaborative learning, team project work) that would seem to "disturb" this course of action. Is this attitude simply a teachers' false judgment or is it justified by research evidence? We try in the following to formulate an answer.

Students' examination system is a socially constructed assessment methodology for evaluating students' learning acquisition and expertise development. In Greek school system it is students' performance at written examinations which determines in a great degree their studying at tertiary education institutions. Is expertise something developing out of the context of the learning setting or does it incorporate situational characteristics? If expertise is a decontextualized form of knowledge then its practice (i.e. students' performance in examinations) would be successful no matter what the situational characteristics of the learning environment might be. If the opposite is true and expertise is modified by contextual characteristics then performance varies according to environmental conditions and it is valid to suppose that a learning environment exhibiting the same situational characteristics as the practicing environment, would better accommodate successful performance.

Billet ( 2001) offers significant evidence that strongly support the latter option. He states that "…the conception of expertise founded in mainstream American cognitive psychology based on individual acquisition and organization of domain-specific knowledge in memory sits uneasily with emerging views that include, emphasize or give primacy to the social and cultural contributions and their relationships to thinking, acting and knowing" (Billet, 2001, p. 432).

Billet underlines the fact that the specific socially imposed requirements of a particular vocational practice influence the way that the abstracted knowledge of the occupation is manifested in practice. Situational characteristics of expertise are illustrated in the research results of a study by [START_REF] Voss | Individual differences in the solving of social sciences problems[END_REF], cited in Billet, 2001) where college students and their workplace-prepared colleagues participated in different constructed tests. The college students performed better when answering questions about economics, but failed when faced with real work tasks.

The workplace-prepared subjects outperformed the first team in these latter tasks.

Although Billet refers to "vocational training" we believe that these considerations apply to school learning as well, e.g. in a traditional curriculum subject like Physics.

Learning Physics in the school perspective means that students should become experts able to manifest their expertise under specific circumstances (mode of testing). Teachers traditionally follow a teaching methodology utilizing problem situations similar to those expected to be encountered in the final test. When ICT enters the learning setting they try to adapt its use in order to serve the established target. The core of their deliberation is that the situational characteristics of any innovative learning methodology employing ICT tools (e.g. higher degree of interaction, experimenting, students working in teams and cooperating) might hinder the development of the kind of expertise that would be useful for dealing with the specific mode of testing (individual ability to solve problems of specific type). One could say that the "language" to be learnt (abstract principles of Physics) may be the same in both cases of learning but the "dialect" to be mastered (way of applying this knowledge) is different since the situational characteristics vary. In the light of the aforementioned research evidence teachers' choice is justified.

Such incompatibilities between ICT supported modes of learning and traditional assessment methodologies have been recorded as a significant obstacle when introducing ICT into schools (OECD, 2001, p. 15). Situational nature of expertise is an impediment to teachers following innovative learning methods as long as assessment methodologies continue to focus on students' isolated skills and knowledge.

c) "Control" of technology

Most teachers are far from being ICT competent. Naturally they worry about their ability to overcome possible technical problems and to handle educational software effectively which, in turn, makes them feel insecure and reluctant to implement by themselves ICT supported teaching (E16). Technical support is crucial to overcome these initial drawbacks. Any lack of it may significantly decrease teachers' motivation to work with computers. Technical support may be well be offered by insiders (e.g. school teachers properly trained (E13)) and it should always be on time, for otherwise it results to redesigning (E19) which may be interpreted as evidence of flexibility but it is also a possible failure indication, in case the redesign eventually canceled the initially set training objectives.

Theme II: "When school met ICT: cultures in negotiation"

People in an educational setting are agents participating in the act of meaning development and sharing, in the perspective of culture continuation. Technology by permeating through every important aspect of contemporary social and economic life is not simply a medium for processing and transmitting information but a prominent cultural characteristic. Users of ICT belong to ICT culture, i.e. they experience a world where ICT multiplies interaction, mediates human communication, is incorporated in procedures of knowledge production and management (OECD, 2000, p. 56). However using ICT is not a trait of the school culture. "ICT has arisen outside the world of education, but with an irresistible case for adoption within schools" (OECD, 2001) and when ICT enters the school culture inconveniences arise. Many teachers find themselves in the difficult position of using technology products without being members of the technology culture. Using a computer is out of their everyday routine, many of them lack even the introductory technological knowledge (E5). School traditional mode of function hinders instead of facilitating the needs of effective ICT use (E2, E3b, E24). Being familiar with other innovative programs enhances acceptance (E20) but consistent support and extensive training (E22, E23, E19) is crucial for the productive use of technology. Teachers accept ICT (E12) but they initially conceptualize the use of ICT within the understandings they have developed in school culture (E17, E18, E21). They seem to be in doubt on how to proceed with the use of ICT tools and what exactly might be what ICT has to offer to school. [START_REF] Pelgrum | Obstacles to the integration of ICT in education: results from a worldwide educational assessment[END_REF] underlines this point: "there are notions that students should trained to learn more autonomously and to get access to and digest information more independently than has been the case so far. However what this means for the educational process is still rather unclear".

These evidence along with what has already been discussed (Theme I-b, I-c) support the view that school and ICT are cultures initially in tension. How could we describe the way things evolve at the meeting point? [START_REF] Salomon | It's not just the tool, but the educational rationale that counts[END_REF] points out that there is "a consistent tendency of the educational system to preserve itself and its practices by the assimilation of new technologies into existing instructional practices". Papert (1997) also emphasizes that "school reform is impossible", meaning that any centrally administered effort to change a complex system is bound to fail. "ICT assimilation" and "school reform" (or "non-reform") are two commonly used concepts relative to this process. One might describe this school transformation process as a "paradigm shift". A paradigm is understood as "the basic way of perceiving, thinking, valuing, and doing associated with a particular vision of reality" [START_REF] Harmon | An incomplete guide to the future[END_REF]. So a paradigm shift can be conceptualized as an evolutionary process that affects the fundamental beliefs, principles and practices of a community, leading gradually to a new collective understanding, which better accommodates its needs. In the core of a paradigm shift lies the cognitive conflict that the members of the community experience when the existing paradigm fails to explain newly encountered phenomena. These difficulties give rise to a state of crisis and a new paradigm is gradually formulated to offer new insights and resolve the conflict.

However, in the case of ICT entering school community it is rather an externally originated demand for change rather than an internally generated state of crisis and a seek for a new paradigm. School, if seen as an isolated learning community, can continue employ traditional teaching methodologies and produce "satisfactory" (within the established assessment paradigm) learning outcomes. The inconvenience arises when the different knowledge production and management skills that broader society demands are taken into account. So, unlikely the paradigm shift process where the affected community experiences the old paradigm inadequacy and strives for a better understanding, in this process the added problem exists that the essence of the necessity for change is still to be understood and accepted by practitioners within the school community. Teachers are asked to adapt their teaching to the needs of another culture, the cognitive traits of which they experience mainly in a theoretical way. We propose that this complex sociocognitive process can be described as a negotiation process between the two cultures. The "negotiation" concept implies that when ICT enters school a "give and take" process is initiated for the establishment of an accepted mode of use of the innovation by accepting and/or adapting and/or rejecting the various possibilities offered. The "invaded" culture (school) has to redefine its boundaries, meaning that its members have to decide what changes and to what extend they are going to implement considering the new practices and relationships that these changes are going to introduce. The dynamics of this process are outlined by the model we use: teachers accept to alter their practices at lower level aiming (a) to better accommodate higher order goals and (b) to minimize detected disturbances. This negotiation takes place not only between teachers and external agents supporting ICT introduction (e.g. governmental services) but also within teachers themselves, forcing them to decisions that accept/adapt/reject modes of ICT based teaching. Our data reveal how teachers welcome training to attain a higher professional and teaching status, support a communicative mode of ICT-use when a "consonance" exists with the established school learning objectives (E17) and assimilate the use of ICT into a mode of teaching compatible to these objectives (E18), to avoid any possible "dissonance". We see all these as evidence of the negotiation process.

Within this framework of understanding it seems reasonable to hypothesize that the implementation of innovative teaching methodologies would be facilitated by emphasizing to teachers the "higher -lower" level dualism of conceptualizing about their teaching: "teaching for success in testing" is a lower level dispensable goal, a minor case resulting from the higher level goal "teaching for the development of socially useful skills and knowledge" that should be preserved. In this perspective, teaching and learning are not procedures divorced from socially imposed requirements and using ICT to support innovative modes of learning is not solely a cognitive matter. What is characterized as innovative, constructive and open use of ICT and the respective learning quality has to be reflected in the socially accepted instructional objectives, in teachers' epistemological assumptions and of course in assessment methodologies.

We feel that at this point the following two research questions emerge: (a) to identify and describe the dynamics of this negotiated use of ICT in schools, and (b) to propose and implement appropriate course of action that would enhance the adoption of teaching methodologies which, tailored to ICT tools, can foster the desired learning qualities of students in the information society. What we described in the previous paragraph as "emphasizing the higher -lower level dualism" might be a possible candidate.

Conclusions

In this paper we presented and discussed our first observations regarding Greek secondary school teachers' attitudes towards the introduction of ICT in the curriculum. Training efforts are generally welcomed by teachers but consistent support and extensive training is necessary in order to consider themselves able for integrating ICT in their teaching methodologies. Teachers are interested in using ICT (a) to attain a better professional profile, and (b) to take advantage of any possible learning benefits offered by ICT but always within the context of the school culture. So we experienced some open and communicative modes of ICT based teaching where school objectives encouraged, but also the assimilation of ICT tools into a traditional teacher-centered mode of teaching. Teachers' attitude to adapt ICT mode of use is justified by research evidence that emphasize the situational character of knowledge and expertise. Introducing ICT into schools is seen as initiating a "negotiation" process where lower level goals may be altered to preserve what are perceived as goals of higher order. These higher order goals however are socially constructed and established. While we agree with [START_REF] Norum | Healing the universe is an inside job: teachers' views on integrating technology[END_REF] who, emphasizing the role of the individual in this process, state that "healing the universe is an inside job", we would like to add that "healing" prescriptions are negotiated and formulated by interacting social agents as a whole.

Table 1 .

 1 Attendance of "horizontal" training sessions

	Tables
		Teachers who attended	Total number of teachers in
			participating schools (100%)
	Gymnasiums	430 (51,9%)	828
	Lyceums	260 (63,1%)	412
	Technical	110 (62,5%)	176
	Total	800 (56,5%)	1416

Table 2 .

 2 Attendance of "vertical" training sessions (offered only to Physics and Literature teachers)

		Training sessions per week (total: 9 weeks)
		"Horizontal"	"Vertical"	ICT in classroom
	Number	3.2	1.2	0.6
	Hours	9.3	2.4	not available
	Teacher-trainees	25.7	5.0
	Teacher-trainees	8.0	4.3
	per training session

Table 3 .

 3 Weekly mean workload per teacher-mentor.

Elements of data are presented accompanied by a code in Italics, e.g. "E4". These codes are used later in the "Discussion" section for reference to each specific element.

Acknowledgments

The postgraduate course described in this paper has been funded by the Greek Ministry of Education and Religious Affairs.


Stavros Demetriadis

Athanasios Karoulis

Andreas Pombortsis

"Graphical" Jogthrough: Expert Based Methodology for User Interface Evaluation, Applied in the Case of an Educational Simulation Interface

"Graphical" Jogthrough: expert based methodology for user interface evaluation, applied in the case of an educational simulation interface

Stavros Demetriadis, Athanasios Karoulis, Andreas Pombortsis

INTRODUCTION

The term "evaluation" generally refers to the process of "gathering data about the usability of a design or product by a specified group of users for a particular activity within a specified environment or work context" [7, p. 602]. Evaluation of user interface design is of special importance in the overall software evaluation plan, for two major reasons: first because it concerns exactly that part of the software product which enables users to communicate their instructions to the machine. Evaluation should verify that the interface design delivers to end-users a friendly, intuitive and transparent yet powerful environment for the accomplishment of their goals. Second because evaluation of user interface should be carried out at the right time; early enough to offer designers the chance of getting valuable feedback about their design ideas and possibly proceed to interface redesign, while all important interface characteristics have been designed and are included for evaluation.

Having designed and developed a first prototype of an educational simulation program that we call "Network Simulator", we wanted to obtain early evaluation data concerning the design principles that we followed. We chose to apply the Cognitive Jogthrough method [START_REF] Rowley | The Cognitive Jogthrouth: A Fast-Paced User Interface Evaluation Procedure[END_REF], an expert based evaluation method intended for evaluation in the design or early development phase. The method has been reported to be a valuable information source during a system design process [START_REF] Aedo | The Evaluation of a Hypermedia Learning Environment: The CESAR Experience[END_REF], is relatively cheap to apply and has already been applied for the evaluation of educational interfaces of a certain kind [START_REF] Catenazzi | The Evaluation of Electronic Book Guidelines from two Practical Experiences[END_REF]. Since our main purpose was to produce evidence of interface quality that would take into account the user's gradual familiarization with the interface tasks, we proceeded to modify the experts' questionnaire used in a typical Jogthrough method by introducing a special graph for recording and presenting evaluators' ratings. We call this modified version "Graphical" Jogthrough.

In this paper we try to formulate an answer to the question: "What are the possible benefits and limitations of the Graphical Jogthrough method when applied for the evaluation of educational This is a pre-print version of the article: Demetriadis, S., Karoulis, A. & Pombortsis, A. (1999). "Graphical" jogthrough: expert based methodology for user interface evaluation, applied in the case of an educational simulation interface. Computers & Education, 32(4), 285-299. software interface design as in the case of Network Simulator?". We describe the characteristic features of Network Simulator, the Graphical Jogthrough method along with the results of our session and we conclude discussing the usefulness and limitations of the method.

WHAT IS THE "NETWORK SIMULATOR"

User Interface Qualities: Intuitiveness and Transparency

Any simulation is a software medium that utilizes the interactive capabilities of the computer and delivers to learner a properly structured environment where user -system interaction becomes the means for knowledge acquisition. A simulation holds a model of some real system and learners generally are expected to discover and understand the properties of the model by giving and modifying input parameters and analyzing the output. The actions that users are allowed to perform and the experiences they are guided to have create a virtual cognitive world which is materialized not only in the internally used model of the simulated phenomenon, but also in the external component of the user interface. The user interface therefore of a simulation software becomes a highly critical element of the overall design not only as far as its perceptual characteristics are concerned but also (and more important) in relation to the cognitive functions that it supports.

There are two (among others) important user interface qualities that designers always should try to keep at high levels:

• intuitiveness (using proper and easily understandable metaphors),

• transparency (not interfering with the learning procedure) An intuitive interface minimizes "the mismatch between the user's expectation of what an interface object should and should not do, based on their previous knowledge, and what the interface object actually does" [7, p.147]. Selecting suitable metaphors allows users to quickly become accustomed with the way of accomplishing tasks, easily remembering and effectively using interface options in order to reach their goals. A transparent interface respectively would help learners concentrate on "what to do" (accomplishment of the educational tasks) and not "how to do" it (trying to figure out how the interface functions) [START_REF] Roth | Phenomenology, Cognition, and the Design of Interactive Learning Environments[END_REF]). In a simulation program, where the majority of available interactions are not merely navigational (guiding users to browse through content) but learning oriented (offering user the ability to alter the system state and learn by observing the subsequent system response), the above mentioned characteristics are of paramount importance when quality of learning experiences is considered.

Users working in a transparent and intuitive interface are generally expected to quickly become effective users of it.

So the core idea of the usage of the graph that we are introducing, is that if we have a tool to record estimated pace of user familiarization with a specific interface then these estimations (being simultaneously evidence concerning the interface intuitiveness and transparency) could help us draw conclusions about software usability and make comparisons between various interface design approaches.

The Network Simulator Interface

Network Simulator is an educational simulation software that enables users to build a computer network, make the necessary hardware and software installations and test the network functionality. This is a pre-print version of the article: Demetriadis, S., Karoulis, A. & Pombortsis, A. (1999). "Graphical" jogthrough: expert based methodology for user interface evaluation, applied in the case of an educational simulation interface. Computers & Education, 32(4), 285-299. These networks may vary from a small typical LAN (Local Area Network) to an Intranet or a WAN (Wide Area Network). The program has been designed for educational use and it may operate in either of two modes: (a) stand-alone or (b) in communication with ISTOS, an interactive learning environment. Details about ISTOS have been published elsewhere [START_REF] Demetriadis | Design Issues for Hypermedia Educational Environments: The Case of "ISTOS[END_REF], but for the scope of this paper it is enough to say that ISTOS is a hypermedia educational program for instruction in the Computer Networking domain, utilizing a case based instructional approach, following primarily Cognitive Flexibility instructional prescriptions [START_REF] Jacobson | Learning with Hypertext Learning Environments : Theory, Design, and Research[END_REF] [START_REF] Spiro | Cognitive Flexibility and Hypertext: Theory and Technology for the Nonlinear and Multidimensional Traversal of Complex Subject Matter[END_REF]. Students, among other activities, have to cope with real world problem scenarios concerning the installation of a computer network under specific conditions and needs. They use Network Simulator in order to design their proposed networking solutions and they submit this solution to ISTOS for further assessment and feedback. In Fig. 1 a general view of Network Simulator interface is presented. Users of the simulation manipulate a network much as word processor users manipulate a written document. All common user interface actions (such as "New", Open", "Save", "Save as…", "Rename", "Close", "Cut", "Copy, "Paste") are supported, along with others that are specially designed for the Network Simulator. Users select the "New" menu command to activate on the screen a specific topology or network architecture selecting from the various supported in the simulation. An activated topology is a properly structured grid consisted by a number of cells where the hardware components of the network may be placed. Users have to place the appropriate computer machines, communication media and other elements of the network (network cards, cables, routers, etc.) and proceed later on to install and setup the necessary software so that the initially empty grid is transformed into a fully functional computer network. Network Simulator extensively utilizes the Select -Carry -Install metaphor. Users may select the component they need from the toolbox (a thematically organized inventory), carry it to the desired cell (cursor changes to denote that an item is being carried) and install it by clicking on the empty cell (Fig. 2). When they finish configuring the network they may ask for OSI control, a network functionality control based on the OSI (Open Systems Interconnection) model. OSI control will inform users on any mismatch present in the network at any level of the OSI model, explain the reasons for this mismatch and prompt them to correct it in order to have the network function properly. Fig. 3 presents a computer in the way that it appears after there have been installed on it a network interface card (NIC) and the Windows 95 software. Student-users may click on NIC and/or software icon and proceed to set the necessary settings for these hardware and software components (such as connecting NIC to cable and/or setting the desired networking protocols for the operational system) so that computer is fully functional in the network.

USER INTERFACE EVALUATION METHODOLOGIES

Walkthroughs and the Jogthrough Version

Interface evaluation of a software system is a procedure intended to identify and propose solutions for usability problems caused by the specific software design. A usability problem may be defined as "anything that interferes with user's ability to efficiently and effectively complete tasks" [START_REF] Karat | Comparison of Emperical Testing and Walkthrough Methods in User Interface Evaluation[END_REF]. Generally speaking an evaluation can be either formative or summative. In a formative evaluation the product is evaluated before it reaches the end user and the objective is to adjust the design to the user's need according to what is already known about the end user profile. A summative evaluation on the contrary involves usage of the software by the end user and tries to identify the level of accomplishment of the targets set in the design phase. Expert based methodologies offer formative evaluation and are reported to be efficient methods that could be applied on system prototypes or even early design specifications [START_REF] Aedo | The Evaluation of a Hypermedia Learning Environment: The CESAR Experience[END_REF] [2] [10] [12]. This is an additional advantage of the method since experts are expected to be able to understand interface functionality without necessarily seeing the product completed. Reeves [START_REF] Reeves | Evaluating technology-based learning[END_REF] points out that "expert review may be the most frequently used evaluation strategy ... different types of experts can provide different perspectives on the critical aspects of your product". The main idea in expert based evaluation is to present the interface supported tasks to a group of experts who will play the role of would be users and try to identify possible deficiencies in the interface design. Designers guide expert evaluators to a walk through the tasks and offer them an appropriately structured This is a pre-print version of the article: Demetriadis, S., Karoulis, A. & Pombortsis, A. (1999). "Graphical" jogthrough: expert based methodology for user interface evaluation, applied in the case of an educational simulation interface. Computers & Education, 32(4), 285-299. questionnaire to record their ratings. The original Walkthrough method [START_REF] Lewis | Testing a Wlakthrough Methodology for Theory-Based Design of Walk-Up-and-Use Interfaces[END_REF] is a substantially slower proceeding method since the entire recording is done manually. In the Jogthrough version [START_REF] Rowley | The Cognitive Jogthrouth: A Fast-Paced User Interface Evaluation Procedure[END_REF] instead a video camera is used for recording evaluators' comments and proposals, thus speeding up the pace of the session. The method is characterized as "cognitive" to denote that the focus is on the cognitive dimension of the user -interface interaction and special care should be given to understand the tasks in terms of user defined goals and not just as actions on the interface (click, drag, etc.). The words of C. Lewis (one of the Cognitive Walkthrough originators) that "the Walkthrough does not identify problems with an interface; it identifies mismatches between system affordances and user goals" [cited in 12] is a clear indication of the cognitive orientation embedded in the method. One should also have in mind that Cognitive Walkthrough is a method originally designed for the evaluation of "Walk Up and Use" interfaces. Users therefore are supposed to be basically novices or infrequent users working with relatively simpler interfaces (e.g. information kiosks). The method though has been applied to more complex interfaces [START_REF] Wharton | Applying Cognitive Walkthroughs to More Complex Interfaces: Experiences, Issues and Recommendations[END_REF] in an attempt to identify needed refinements and augmentations that could turn it into a useful evaluation methodology even under these circumstances. The Jogthrough version has also been implemented for educational software evaluation [START_REF] Aedo | The Evaluation of a Hypermedia Learning Environment: The CESAR Experience[END_REF], the kind of software that users may gradually become familiar with through repetitive usage.

A typical Jogthrough session setup

Fig. 4 presents a typical setup of a Jogthrough session. Software to be evaluated is presented on a computer screen (or it could be a simulated software behavior using ordinary slides). The presenter presents the pending task and action to be evaluated. The moderator helps to keep up with the timetable and generally to maintain the pace of the session and the recorder manipulates all recording media and does the note taking. Evaluators finally record their opinions on questionnaires and discuss with the design team issues of interface design Why "Graphical" Jogthrough?

The original Jogthrough method asks the evaluators to estimate proportion of users that understand and work efficiently with the interface only the first time they work with it. So it is intended for the kind of "walk up and use" interfaces, those that are going to be used once by perspective users (e.g. in an information kiosk). In a different case though, such as an educational application, learners are going to repeatedly use interface supported tasks in various phases of instruction. A properly structured interface being intuitive and transparent enough will enable users to quickly understand how to do things in the virtual environment and allow them to focus only on the learning aspects of their actions. On the other hand a complex or badly designed interface may still offer the means for doing things but users will take longer to understand interface functionality and to remember how to make effective use of it. The novice-becoming-expert pace will be different in the two instances. It is exactly this pace that we wanted to capture in our evaluation and proposed a slightly modified Jogthrough version in order: a) to introduce time variable in terms of user familiarization with the interface and b) to allow evaluators to record quantitatively their opinion. We introduced time variable by designing a graph based answer sheet for the evaluators and asked them to estimate proportions of successfully working users in four distinct phases of user familiarization with the interface: a) the first time users try to perform a task action, b) after having performed it few (2 or 3) times, c) after enough times (4 to 6) and d) after more than 6 times (see Fig. 5).

These numbers of repetitions were arbitrarily set, aiming simply to offer evaluators a reasonable quantitative representation of what we mean when we use the terms "few", "enough" and "more". In this way evaluators' estimates may be presented in the form of a graph denoting number of users able to This is a pre-print version of the article: Demetriadis, S., Karoulis, A. & Pombortsis, A. (1999). "Graphical" jogthrough: expert based methodology for user interface evaluation, applied in the case of an educational simulation interface. Computers & Education, 32(4), 285-299. successfully perform interface tasks after some repetitions. The faster this curve reaches the high-end limit, the more intuitive and transparent the interface is evaluated to be.

CONDUCTING THE EVALUATION SESSION The Graphical Jogthrough Evaluation Session

Our session lasted about 4 hours and in this time interval 6 tasks (analyzed into 20 actions) were evaluated. Initially we gave evaluators a small presentation of the overall interface design and allowed them to have hands-on experience working a little time with it. Would be users of the simulator were described as students who have enough working experience with the "Microsoft Windows 95 ™" environment, know the basics of a word processor and a spreadsheet, have introductory knowledge of basic programming languages such as BASIC and/or Pascal and are familiar with some popular computer games. We invited four experts to act as evaluators in the session. The general advice when selecting evaluators is to form an interdisciplinary team and try to enlist people with Cognitive Science or HCI background [START_REF] Wharton | Applying Cognitive Walkthroughs to More Complex Interfaces: Experiences, Issues and Recommendations[END_REF]. Our evaluators were very experienced in multimedia software programming and development and one of them in the subject matter. Two of them had important educational experience but none of them possessed any special cognitive science or HCI background. The tasks selected for evaluation were representative of the complete set of tasks available in Network Simulator, presenting to evaluators all the major interaction mechanisms available in Network Simulator. Each task was analyzed into a series of actions that the user had to perform for completing it and these actions were defined both in terms of user goals and interface actions that had to be executed (Fig. 6). So we tried to define user goals at the granularity of individual interface actions although this is reported to cause problems in certain cases [START_REF] Wharton | Applying Cognitive Walkthroughs to More Complex Interfaces: Experiences, Issues and Recommendations[END_REF]. The reason for doing this was the limited Cognitive Science background of our evaluators and the fact that they were unfamiliar with the method. We believed it would be helpful to see user goals and interface actions tightly bound, avoiding thus getting involved in considerations that might lead to session delay and failure of completing the evaluation in the time available.

Recording Evaluators' Ratings

Evaluators were handed a booklet that contained the forms to record their ratings. The header of each form included the task and the pending action. The form displayed the five basic questions that the evaluators had to answer. These questions were: (After Moderator described the action) a) How many users will think this action is available? b) How many users will think this action is appropriate? c) How many users will know how to perform the action? (After Moderator executed the action) d) Is the system response obvious? YES NO e) How many users will think that the system reaction brings them closer to their goal? When the task was finally completed evaluators were prompted by question (f) to generally comment on the evaluated task. f) Do you find the task useful? YES NO If NO then how can it be modified to become useful? Comments / Opinions

The above questionnaire had to be answered for every action of each evaluated task. No alternative or auxiliary actions were considered for evaluation. In a typical Jogthrough session such as those described in [START_REF] Aedo | The Evaluation of a Hypermedia Learning Environment: The CESAR Experience[END_REF] and [START_REF] Rowley | The Cognitive Jogthrouth: A Fast-Paced User Interface Evaluation Procedure[END_REF], evaluators answer questions a, b, c, and e by using a simple arithmetic code: 0=very few, 1=less than half, 2=more than half, 3=nearly all, NS/NC=Doesn't Know/No answer. Their ratings define a number of potential users without considering the process of user becoming experienced with the interface after repeatedly having completed the various tasks. In our session instead each time the evaluators had to answer one of the questions a, b, c, or e, they put a check mark This is a pre-print version of the article: Demetriadis, S., Karoulis, A. & Pombortsis, A. (1999). "Graphical" jogthrough: expert based methodology for user interface evaluation, applied in the case of an educational simulation interface. Computers & Education, 32(4), 285-299. in every column of the graph presented in Fig. 5, declaring the number of estimated users that could successfully understand interface functionality.

RESULTS

The graph presented in Fig. 5 was the main instrument for recording evaluators' ratings in our session. The hypothesis when using such a tool is that evaluators will be (or at least feel) capable of estimating user proportions in the four distinct phases that appear in the graph. In our case there was no comment or any other indication against this hypothesis, from the evaluators themselves. After a short introduction by the moderator, they were able to comfortably use the graph and denote their ratings by checking on the proper boxes. It soon became a completely transparent recording tool.

In Fig. 7 evaluators' ratings are presented after they have been coded (1=None, 2=Few, 3=About Half, 4=Most, 5=All) and statistically elaborated to obtain the mean scores across all the evaluated tasks. We interpret this final graph as a kind of quality "fingerprint" displaying visually an estimate of how fast the majority of the users would be effectively working with the specific interface. The four curves of the graph focus on four cognitive dimensions of the interaction between human and computer: a) "available" curve: understanding the availability of a specific interface option, b) "appropriate" curve: understanding that a certain option is the appropriate for the action in mind, c) "know to perform" curve: knowing how to interact with the interface in order to perform the specific action, and d) "closer to goal" curve: understanding that the interface response (after user's action) brings user closer to his/her goal. These curves present the expected proportions of users who could successfully understand actions as available, appropriate, getting them closer to their goal or know how to perform them versus the times they would actually have to perform the tasks in order to achieve this level of understanding.

A first observation is that a great number of users (~ 60%) is expected to effectively understand the availability, appropriateness and way to perform the actions even from the first time. Even more users (~ 90%) will understand the first time that system responses are getting them closer to their goals. The overall result is very positive for the Network Simulator since it indicates that evaluators expect the majority of users (90%) to be able to completely understand the task performance after they actually perform the tasks for a few (2 or 3) times. One can see that "Closer to Goal" curve is distinctly higher than the three others. We believe that this might be an indication of evaluators having connected strongly the user understanding of getting closer to his/her goal (and hence their ratings) to the perceptual alteration of the system state (obvious when an action was executed) and not so much to the cognitive process underlying the task (something that would demand them to closer focus on the cognitive aspects of system state alteration).

Apart from these quantitative results, what we see as equally important is that the graph structured evaluators' thought towards the process of user gradually understanding the interface functionality and effectively working with it. This perspective became the cognitive lens through which they tried to formulate their evaluation. Experts, in other words, tried to understand user-system interaction as timeextended and repeated and not only as quick "walk up and use" experience. As a result they were more willing to accept apparent task complexity provided that a) they felt that the proposed design adequately met the educational needs of the simulation and b) the design of the task was evaluated as easily understood after few times of performance.

DISCUSSION

Quantitative Description

This is a pre-print version of the article: Demetriadis, S., Karoulis, A. & Pombortsis, A. (1999). "Graphical" jogthrough: expert based methodology for user interface evaluation, applied in the case of an educational simulation interface. Computers & Education, 32(4), 285-299. We believe that an important aspect of the method is that it presents recorded data as a graphical representation of novice-becoming-expert pace of would be users. This form allows one to easily make concluding remarks that correlate percentages of effectively working users and effort needed to reach certain ability levels. Statements such as "about 90% of target group users are estimated to work comfortably with the software after they have completed tasks for 2 or 3 times" or "the majority of users are expected to understand availability and appropriateness of interface options even from the first time they use it", could be conclusions based on the graph results. In a case where a group of experts evaluates a series of interfaces then the resulting graphs might offer the basis for reaching a final decision.

Employing therefore the Graphical Jogthrough approach allows evaluation to include the time variable and offer more appropriate quantitative results in cases where gradual user familiarization with the interface is an important factor of the overall user -machine interactivity.

Structuring Experts' Review

Reeves [START_REF] Reeves | Evaluating technology-based learning[END_REF] stresses the fact that "it is often useful to structure the expert's review so that you are assured of getting the types and depth of information you desire". This is what we have accomplished by using the graph based answer sheet. But as we mentioned previously, the graph did not only offer the chance for recording quantitative data, but also set the general framework for the evaluators to follow in the evaluation process. So their comments and proposals were based on this approach, i.e. taking into account users' repetitive usage of the interface and therefore their gradual familiarization with it.

We believe that this approach is much more appropriate when design of educational interface is concerned. Especially in the case of an educational simulation interface, one can not possibly follow the same evaluation approach as in the case of a "walk up and use" interface, exactly because the interface tasks that are necessary for the integration of the simulation, may be initially experienced by users as complex but after a few times of performance as reasonably well designed and structured.

The Decisive Role of Evaluators' Expertise and Evaluated Tasks

It is clear of course that the curves of the graph are the product of the interaction between evaluator's expertise and tasks evaluated. More experienced evaluators examining a broader set of available tasks might locate interface deficiencies and result into a graph with slower ascending curves related to those presented in a graph by less experienced evaluators who would examine only a small and possibly not representative portion of the interface tasks. Evaluators' specific expertise may lead them to focus on characteristics they are more familiarized with. As Preece et al. point out "experts are often renowned for their strong views and preferences, in other words, biases. They may concentrate on certain features and virtually ignore others" [7, p. 675]. It is a common advice [START_REF] Preece | Human -Computer Interaction[END_REF] [12] that care should be taken so that evaluators' team should be as interdisciplinary as possible and that the evaluated tasks constitute indeed a representative group of the overall available tasks. In our case it was clear from the evaluators' observations that multimedia experts focused more on perceptual and organizational characteristics of interface (e.g. appearance and functional characteristics of buttons) while the content expert addressed more subjects related to the simulated characteristics of the domain. Similarly, lack of educational interface design experience might mislead evaluators to propose tentative redesign approaches, something that might also affect their ratings. We came across such a case when the content expert proposed that selection of a specific network interface card might well be unavailable when user is working with a network type where this card is not used, so that interface could support users' decisions by limiting the available options only to the accepted ones. This suggestion was discussed and rejected since it was in contradiction with our pedagogical approach to allow users make mistakes that in the real world would be possible for novices to do. This is a pre-print version of the article: Demetriadis, S., Karoulis, A. & Pombortsis, A. (1999). "Graphical" jogthrough: expert based methodology for user interface evaluation, applied in the case of an educational simulation interface. Computers & Education, 32(4), 285-299. So although our experience is limited to only one evaluation session we support the view that the resulting curves are certainly biased both by evaluators' expertise and tasks presented and that in order for the graph to be useful for reviewers, it should be accompanied by a description of evaluators' background and evaluated system tasks. This description would set the frame for reviewers for better understanding the kind of the interacting factors that produced the specific graph.

Points of Concern

We believe that one major point of criticism against the approach we described is that experts are capable of producing only rough estimations about the users proportions that the graph asks for and this kind of information does not convey the details that designers need in order to better understand shortcomings of their interface design. Moreover different interface designs could not possibly produce distinctly different curves unless someone compares the curves produced by an exceptionally good and a poor interface. Resourcing to our experience from the session we conducted, we comment this line of thought as follows: certainly the graph asks only for rough estimations since it is based on the questionnaire used in original Jogthrouh method, which too allows for the recording of rough estimations. Nobody, we believe, is expecting that evaluators in their role play could give more precise quantitative measures. But as already underlined, we see as equally important the fact that the graph frames evaluators' point of view towards the process of user getting gradually familiar with the interface. This perspective reflected into evaluators' comments, proposals and ratings is what designers gain (apart from the final graph) by employing the Graphical Jogthrough approach. As far as the second part of the argument is concerned our opinion is that these rough estimations could classify evaluated interfaces into one of three general categories: "successful design" (curves climb fast in 90 -100 percent area), "some redesign suggested" (curves exhibit a more or less 45 angle slope), and "poor design" (curves remain in the low user percentage area). In certain cases this classification might be enough for decisions to be taken (accept or revise an interface design). It is a matter of further research and application of the method to produce evidence for supporting (or rejecting) the above expressed opinion.

One might of course ask "how do we possibly know that evaluators (whatever their expertise might be) are capable of estimating user proportions that the graph asks for?" Well, obviously we do not. This is only a hypothesis. But what we could do is to cross-examine results of expert session with those produced in an empirical evaluation session by a sample of end users. If the results of the two sessions conclude then this would be a strong indication that Graphical Jogthrough can really produce trustworthy evidence.

CONCLUSIONS

Concluding from our experience we argue that the Graphical Jogthrough method, in spite of the limitations discussed, can offer useful quantitative and qualitative feedback to designers because the introduced graph guides evaluators' review to focus (as intended) on the process of user gradually getting familiar with the interface tasks. It is also advisable that designers should try to understand in depth the interaction between the two main factors of the session (evaluators' expertise and selected tasks), and present a description of them along with the produced results.

Further research could verify the results of such a session by comparing them to the results produced by end users themselves and could also examine other possible forms of the graph for enabling more reliable and useful data recording.

Acknowledgments

Authors wish to thank Panagiotis Tzounakis, George Papapanagiotou, Vangelis Triantafyllou and Asterios Chatzicharistos for their valued contribution in the evaluation process described in this paper. We would also like to thank Co-Editor of "Computers & Education" Journal Prof. R. S. Heller and the anonymous referee for useful comments and suggestions. This is a pre-print version of the article: Demetriadis, S., Karoulis, A. & Pombortsis, A. (1999). "Graphical" jogthrough: expert based methodology for user interface evaluation, applied in the case of an educational simulation interface. Computers & Education, 32(4), 285-299.

Author Note

This work has been presented (in a more concise form) in ED-MEDIA & ED-TELECOM 98 Conference, AACE, Freiburg, Germany, June 1998.