Characterizing the 'experimental procedure' in science laboratories: a preliminary step toward students experimental

Keywords: experimental procedure, laboratory work, experimental design

Many studies have stressed students' lack of understanding of experiments in laboratories. Some researchers suggest that if students design parts of, or entire experiments, as part of an inquirybased approach, it would overcome certain difficulties. Experimental design requires to write a procedure. The aim of this paper is to describe the characteristics of a procedure in science laboratories, in an educational context. As a starting point, this paper proposes a model in the form of a hierarchical task diagram that gives the general structure of any procedure. This model allows both the analysis of existing procedures and the design of a new inquiry-based approach. The obtained characteristics are further organized into criteria that can help both teachers and students to assess a procedure during and after its writing. These results are obtained through two different sets of data. First, the characteristics of procedures are established by analysing laboratory manuals. This allows the organization and type of information in procedures to be defined. This analysis reveals that students are seldom asked to write a full procedure, but sometimes have to specify tasks within a procedure. Secondly, iterative interviews are undertaken with teachers. This leads to the list of criteria to evaluate the procedure.

1-Introduction

An experimental procedure describes an experiment. Starting from this broad definition, the work presented here intends to describe more finely the nature and the place of experimental procedures in an educational context. It seems very important to address this question since labwork, hands-on activities and Inquiry-Based-Learning are worldwide promoted in education from school to university. In particular, the laboratory is a good place for inquiry (2003) or other activities in which students have to design and carry out their own experiments in order to answer a scientific question.

Inquiry and experimental design in labworks

Labwork is essential for the learner in experimental sciences because some specific learning goals can only be achieved in this context. The goals of labworks are available in official curricula, in laboratory instructions or from actual practises studies (See e.g. [START_REF] Tiberghien | An analysis of labwork tasks used in science teaching at upper secondary school and university levels in[END_REF]. Among these goals are 'physical manipulations of real world substances or systems, interactions with simulations, interactions with data drawn from the real world, access to databases or remote access to scientific instruments and observations' (Committee on High School Science Laboratories, 2006). According to this committee, labworks should help students to develop an understanding of the complexity and ambiguity of empirical work, as well as the skills to calibrate and troubleshoot equipment used to make observations. However, students encounter several difficulties with labwork, such as understanding the goal of an experiment [START_REF] Keys | Revitalizing instruction in scientific genres: Connecting knowledge production with writing to learn in science[END_REF] or interpreting data [START_REF] Millar | The role of practical work in the teaching and learning of science, High School Science Laboratories: Role and Vision[END_REF].

To overcome some of these difficulties, inquiry-based approaches have been suggested. Recent reports from the European Commission (2007) as well as from OECD (2006) with science, especially at a time of decline in interest in sciences and technologies. Rocard (European Commission, 2007) claims that 'a reversal of schools' science-teaching pedagogy from mainly deductive to inquiry-based methods provides the means to increase interest in science'. In the USA, inquiry based learning in laboratories was reintroduced in the programmes around 1996 [START_REF] Hofstein | The laboratory in science education: foundations for the twentyfirst century[END_REF]. They call this 'a shift from teacher-directed to purposeful-inquiry'. An important aim of inquiry in the laboratory is to help students understand the link between theory and experimental activities. This comes back to the difficulties students often have in making the link between scientific concepts and the experiment to be done, or later between the experimental data and the conclusion to be drawn. As [START_REF] Millar | The role of practical work in the teaching and learning of science, High School Science Laboratories: Role and Vision[END_REF] states, 'the aim is to develop a link between an observation and a way of thinking about it -between the world and a mental representation of the world'. The task of experimental design, where learners conceive and describe their own experiment, can be a good opportunity for learners to make these links [START_REF] Karelina | Acting like a physicist: Student approach study to experimental design[END_REF].

Inquiry-Based pedagogy approach during laboratory sessions often implies that the learners design their own experiments in order to answer a question. Many authors have described the process of inquiry as a set of sub-processes including the experimental design, or, more generally, 'planning investigation'. For example, [START_REF] Möller | Defining levels of scientific inquiry skills in lower secondary biology education[END_REF] use the following steps for describing an inquiry: 'formulating questions', 'generating hypotheses', 'planning investigation', and 'interpreting data'.

Experimental design has also been mentioned (e.g. [START_REF] Dunbar | How scientists build models: InVivo science as a window on the scientific mind[END_REF][START_REF] Lewis | Design and inquiry: bases for an accomodation between science and technology education in the curriculum[END_REF] as an important part of the discovery process followed by researchers when conducting experiments in science. The activity of experimental design has the characteristics of the general activities of design. It embodies three different features, as reported by de [START_REF] De Vries | Students' construction of external representations in design-based learning situations[END_REF]: 'design is a creative activity'; 'the future artefact has to fulfil needs'; and 'a plan or model or something has to be formulated before the artefact is made'. In the case of experimental design, the experimental procedure is the written plan while the experiment is the artefact.

Different studies emphasize the importance of experimental design in an educational context. [START_REF] Koretsky | Enhancement of Student Learning in Experimental Design Using a Virtual Laboratory[END_REF][START_REF] Koretsky | Enhancement of Student Learning in Experimental Design Using a Virtual Laboratory[END_REF][START_REF] Neber | Promoting pre-experimental activities in high-school chemistry: focusing on the role of students' epistemic questions[END_REF][START_REF] Neber | Promoting pre-experimental activities in high-school chemistry: focusing on the role of students' epistemic questions[END_REF] observe higher-order cognitive activities of students facing such a task. [START_REF] Apedoe | The empirical attitude, material practice and design activities[END_REF] stress the importance to help students acquire an empirical attitude by making them design experiments. [START_REF] Karelina | Acting like a physicist: Student approach study to experimental design[END_REF] find that, when students design their own experiments, they engage in behaviours that are much closer to the ones of scientists than did students working in traditional laboratories, because they spent more time 'making sense', i.e. in discussions about physics concepts, experimental design, and data analysis. [START_REF] Arce | Student-designed experiments in scientific lab instruction[END_REF] find that students showed a better understanding, in the exam, of concepts related to the experiments they designed themselves, while [START_REF] Séré | Towards renewed research questions from the outcomes of the European project labwork in science education[END_REF] suggests that experimental designs might be helpful to acquire procedural knowledge. Etkina, Karelina and Ruibal-Villasenor) (2010) found that when students are used to design experiments, they perform similarly on exams than students who did not design experiments.

However, the development of students' scientific abilities (i.e. the most important procedures, processes, and methods that scientists use when constructing knowledge and solving experimental problems) is fostered through design labs.

Despite these benefits, teachers rarely let the students design their own experiment [START_REF] Author | Papers presented at the ESERA conference[END_REF]. [START_REF] Tiberghien | An analysis of labwork tasks used in science teaching at upper secondary school and university levels in[END_REF] find that, in high schools of five European countries (Denmark, England, France, Germany, and Spain), experiments are fully specified in 80 to 95% (depending on the discipline) of the laboratory manuals. This trends is confirmed by other studies in the US [START_REF] Fuhrman | Do secondary school laboratory texts reflect the goals of the "new" science curricula[END_REF] and in Australia [START_REF] Fischer | Laboratory Learning Environments and Practical tasks in Senior Secondary Sciences Classes[END_REF].

Experimental design is a difficult task for students [START_REF] Séré | Le fonctionnement intellectuel d'étudiants réalisant des experiences : observation de séances de travaux pratiques en premier cycle universitaire scientifique[END_REF], which may be part of the reason why it is difficult for a teacher to let students carry on such tasks [START_REF] Author | Papers presented at the ESERA conference[END_REF]. Several difficulties encountered by students have been reported, including correctly analyzing the issue, putting the experimental procedure into words which relates to difficulties in writing a text [START_REF] Marzin | How can we take into account student conceptions of the facial angle in a palaeontology laboratory work[END_REF], taking into account the question of measurement accuracy (Author a, 2007), and using the necessary conceptual knowledge they should master [START_REF] Laugier | Résolution de problème et pratique expérimentale : analyse du comportement des élèves en début de seconde[END_REF]. [START_REF] Lyons | Design of a laboratory to teach design of Experiments[END_REF] ended problem, students first engage in several modules to establish a level of competence in experimental design, in a scaffolding process. For [START_REF] Laugier | Résolution de problème et pratique expérimentale : analyse du comportement des élèves en début de seconde[END_REF], students need to be able to take the initiative and engage in discussion, with careful guidance from the teacher. Therefore, experimental design appears as a promising and challenging activity for learning. The experimental procedure, as the plan of the experiment, occupies a central position in the experimental process, as an essential input for the execution of the experiment, but also as the main outcome of experimental design. Furthermore, one can expect that the experimental procedure plays an important role while building and scaffolding an experimental activity for students, be it with design or not. In this context, a preliminary but fundamental question arises: What is the procedure of a scientific experiment? How can we describe and characterize it? This question has to be considered with an epistemological point of view as well as in an educational context.

First, one may look for some definition or description of experimental procedures. One problem is that the experimental procedure in itself is not part of established knowledge at the level we focus on. The few references we found in the literature about the experimental procedure are books dedicated to biology students learning about experimental design and statistical analysis. For instance, according to [START_REF] Dean | Design and analysis of experiments[END_REF], when planning an experiment, it is necessary to 'specify the measurements to be made, the experimental procedure and the anticipated difficulties'. This reference, like many others, does not go into details about the nature and the content of procedures. It appears that there is a lack of explicit knowledge on this topic. In this paper, we aim at filling this gap by providing a formal description of experimental procedures for educational contexts.

In the following, the word procedure will refer to the experimental procedure or the procedure of a scientific experiment. Our aim is to build a model of procedures for an educational context. This model, based on theoretical and empirical approaches is divided in two parts. The first part is descriptive and accounts for the operational nature of a procedure; the second part is a list of criteria that helps to evaluate the properties and the validity of a procedure. First, our point of view on procedures is derived from the activity theory that provides tools to analyse activities. This allows the descriptive part of the model of experimental procedures to be built (see Section 1.3). We used this model to analyse the laboratory manuals written by teachers in order to characterize the procedures experienced by the learners during their school practices. The literature results mentioned above suggest that, in cookbook manuals, the procedures are rather complete [START_REF] Lunetta | The school science laboratory: Historical perspectives and context for contemporary teaching[END_REF]. Our empirical work was guided by the hypothesis that procedures are traditional objects used during labworks, but are seldom written by learners. In other words, the place for experimental design is very small. It is not sufficient to analyse procedures, especially when dealing with the question of scaffolding the students' activity of conceiving an experiment or evaluating the resulting procedure. This requires epistemological information about procedures. Our first part of the model of experimental procedures described in Section 1.3 is descriptive and does not account for such properties. We conducted a literature search on evaluation criteria of procedures in science research (see Section 1.4) in order to complete the model. However, as our focus is on procedures in an educational context for which such analysis are not available, we turned to teachers. An experimental study was conducted to find how teachers would define the evaluation criteria for procedures designed by students.

The use of hierarchical task diagrams for modelling procedures

The first contribution of this study is the proposal of a descriptive model for procedures. We define a procedure as a description of the manipulation of data and real-world objects, in the aim of collecting and processing experimental data and/or building new objects. Thus, a procedure describes an activity. For this reason, we chose to use different tools and models aiming at analysing the activity of a subject: hierarchical task analysis and Leont'ev's model of activity. It should be emphasized here that the activity we intend to describe with our model is not the activity of experimental design, but the outcome of experimental design, i.e. the written procedure.

We propose to describe procedures with hierarchical task diagrams (HTD) [START_REF] Author | Paper presented at the ESERA Conference[END_REF]. The principle of task analysis with HTD is to decompose the main task into sub-tasks (expressed by their goal) following a downward analysis: the lower the task in the HTD, the more detailed is the description. The default timeline used for reading such a diagram is from left to right: tasks on the left should be carried out before tasks on the right of the diagram. Figure 1 presents an example of procedure (take an absorbance spectrum) in the shape of a hierarchical task diagram.

[Insert Figure 1 about here] The hierarchical decomposition of experiments proved to be useful in artificial intelligence research for automating the experimental design in a restricted domain [START_REF] Friedland | The concept and implementation of skeletal plans[END_REF], and HTDs are widely used for task analysis in the field of Human-Computer Interaction [START_REF] Dix | Human-Computer Interaction[END_REF][START_REF] Mori | CTTE/ support for developing and analyzing task models for interactive system design[END_REF]. In our case, the use of HTDs had several interests: the hierarchical structuring of tasks at different levels and the characterization of the tasks by their goal bring out the strategy employed in the experiment, and thus the meaning of the procedure. The information that is contained in the procedure and cannot be described with the HTD has a special status because it does not concern, neither task description, nor strategy considerations. We called this information, the 'non-technical information' and the use of our model helped to detect it. For example, the task 'heat to a maximum of 80°C', could be completed by the non-technical information 'otherwise the molecules will be destroyed' to justify the parameter value of the task. Another interest of HTDs is that the level of detail of the procedure's description is easily visualized as the last level of decomposition of the task. This level is important to detect as it informs us on the teacher practise: it is the role of teachers to adapt the level of details of the procedure to the ability of the learners who will execute the experiment.

Our description of a procedure is also based on Leont'ev's model of 'activity' (Leont'ev, 1978).

According to this model, an activity (in our case 'execute an experiment') can be broken down into actions, which are further subdivided into operations. In a context of experimental design, using these categories can provide the designer with an understanding of the steps necessary for a user to carry out the task [START_REF] Nardi | Context and Consciousness: Activity Theory and Human-computer Interaction[END_REF]. The difference between actions and operations is the level of consciousness of these processes: while actions are connected to conscious goals, operations are related to routine behaviours performed automatically [START_REF] Freire | A Socio-Cultural/Semiotic Interpretation of Computer-Mediated Communication[END_REF]. Another difference between actions and operations is that actions are related to goals while operations are related to conditions: operations could be different, depending on the context of the activity [START_REF] Leont'ev | Activity, Consciousness, and Personality[END_REF].

[Insert Figure 2 about here]

In our hierarchical task diagrams (see Figure 2), we use the three levels 'activity / action / operation'. We add a hierarchical decomposition in the level of actions, as suggested by [START_REF] Leont'ev | Activity, Consciousness, and Personality[END_REF]: 'In the course of achieving an isolated general goal, there may occur a separation of intermediate goals as a result of which the whole action is divided into a series of separate sequential actions'. The root of the HTD (upper box) concerns the activity of 'executing an experiment' related to the scientific problem to be solved. From this root, we have named different levels in the structure. Beneath the root are the structuring tasks: they reveal the logical organization of the procedure, related to the strategy. There can be multiple levels of structuring tasks, depending on the hierarchical organization of the procedure. The level of actions represents the effective part of the procedure in reference to the experimenter's skills. As the default direction of the timeline is from left to right, the effective procedure for the HTD described in Figure 2 would be 'A111, A112, A11, A211, A212, A213, A221, A222'. The last level is the level of operations that are related to unconscious activities of the experimenter, like cognitive operations or routine and simple gestures.

As the operations do not need to be explicitly described, this level is usually not expressed in procedures. For example, in the diagram presented in Figure 1, operations such as 'rinse the Finally, in order to represent a complete procedure, the operations must be described with parameters that will finely define the experiment (like the material to use, the quantities of substances...). If the operations are not written in the procedure, these parameters can be found in the level of actions (following the principle that each task can be defined by the parameters of its subtasks).

Therefore, a complete procedure will include two types of tasks: structuring tasks and actions (with attached parameters).

As a tool in experimental research, a procedure is essentially an explicit discretization and organization of the activity. For this reason a description model was needed, in order to provide means to analyse the procedure. Moreover, given the design nature of the construction of a procedure, using HTD was quite natural since it works as an analytical instrument, structuring in time and space the procedure at all the needed level of granularity. The choice of activity theory imposed itself in this context (activity meant both as an intellectual and manual one, and based on the use of devices).

The validity aspect of procedures

We have considered the results issued from diverse fields of research that proposed some generic criteria for evaluating experiments. [START_REF] Kerlinger | Foundations of behavioral research[END_REF] in a study about research design in the field of behavioural research, gives two basic goals corresponding to experiments designed by researchers:

'answer the research question' and 'control all sources of variance'. [START_REF] Friedland | The concept and implementation of skeletal plans[END_REF], who modelled the control structure of experimental design for artificial intelligence treatments, used three classes of criteria to compare experimental techniques: 'whether the technique serves to meet the experimental goal', 'whether it will be successfully applied to the given sample under the given laboratory conditions', 'whether it will be optimal in terms of reliability, convenience, accuracy, cost and time'. [START_REF] Baker | Constraints on the experimental design process in real-word science[END_REF], who studied the 'InVivo' process of experimental design in an immunology laboratory, found that scientists use four sets of criteria to evaluate their experiments: 'ensuring a robust internal structure to the experiment', 'optimizing the likelihood experiments will work', 'performing cost/benefits analysis on possible design components', and 'ensuring acceptance of results by the scientific community'. These different criteria have been proposed in different contexts such as guidelines for graduate students, criteria considered by researches in science laboratories, or criteria used by artificial intelligence treatments. Our aim is to determine if teachers would hold similar criteria when they evaluate the experiments designed by learners, and if they would go to more detailed criteria or remain at a coarse grain level.

Research questions

The goal of this paper is to investigate the nature of a procedure in educational context. Two aspects are studied: (i) manuals where procedures are presented to students in written form and (ii) teachers' evaluation of procedures written by students.

As a result, this paper answers the following questions:

1. How is the procedure presented to students in laboratory manuals? What type of information is given in the procedures? 2. Are there any missing tasks in the procedures of the manuals? If yes, what types of tasks are missing?

3. Under which conditions the missing tasks of the procedures can lead to an activity of experimental design by the learners? 4. According to which criteria can teachers evaluate the validity of procedures written by students?

We study a variety of typical laboratory manuals from various disciplines (biology, chemistry, geosciences, and physics). They give some insight into the ways students are confronted to procedures when starting scientific studies. We focus on the end of high school and the first years of university. To uncover teacher's criteria when evaluating students' procedures, we undertook a series of situational interviews with teachers at universities and high schools. Finally, based on this study and the literature, we were able to propose a complete list of criteria.

2-Method

Laboratory manuals analysis

Choice of the laboratory manuals

In many countries, the laboratory manual is the traditional document given to students to define the work they have to do during labworks, and to guide them through it. We use laboratory manuals as the materials for analysis because they are representative of usual practises in science education and play a central role in labwork in science teaching. They have a strong influence on students' activities during labworks [START_REF] Tiberghien | An analysis of labwork tasks used in science teaching at upper secondary school and university levels in[END_REF]. As [START_REF] Tiberghien | An analysis of labwork tasks used in science teaching at upper secondary school and university levels in[END_REF] reported in their study, 'analysing a labsheet can provide information about the main features of labwork activities that the teacher makes explicit'. Both cookbooks and laboratory manuals are a priori supposed to contain a complete procedure. Therefore, traditional laboratory manuals should provide an abundant and varied collection of procedures in an educational context. Our hypothesis is that studying laboratory manuals helps us to learn more about the procedure, such as its organization and the tasks really dedicated to students.

We analysed laboratory manuals in four main experimental science disciplines (biology, chemistry, geosciences, and physics), at two teaching levels: The last year of secondary school and the first and second years of university (analysed together as university level). We selected 39 laboratory manuals that have been in use for several years in four secondary schools and two universities.

These manuals include a variety of content and pedagogical practises. The LSE project [START_REF] Tiberghien | An analysis of labwork tasks used in science teaching at upper secondary school and university levels in[END_REF] states strong similarities between most educational systems when analysing labsheets chosen as representative of regular teaching practises in several European countries: 'This similarity leads us to wonder if there is an implicit international paradigm of labwork in science education'. So that we 'need to be aware of the tacit institutional "norms" which may be operating across national boundaries'.

With a sample relatively small per discipline in each level, our objective is to identify patterns and trends per level or per discipline, rather than doing statistics on large samples. Following [START_REF] Tiberghien | An analysis of labwork tasks used in science teaching at upper secondary school and university levels in[END_REF], we believe that it is more likely to pick up features that are shared rather than being specific to unique local practises. Firstly, we selected manuals considered by teachers as representative of regular teaching practises: our selection is typical of what is usually done at secondary school or university level in France. Even if limited in geographic location and in number, the manuals are collected from a variety of contents, schools, disciplines, and levels. All manuals have been designed by small groups of teachers and used by whole teams of teachers for years. Secondly, we used an appropriate analysis grid, which is not intended to capture fine details, but focuses on general trends over the variety of laboratory manuals.

Description of the analysis grid

The previous description of a procedure is used to build an analysis grid for laboratory manuals. We designed a sequence of questions (Table 2) in order to get elements to answer our research question.

Each of the 39 laboratory manuals was analysed separately by two evaluators, among a team of eight evaluators. One of the evaluators is a teacher from the discipline while the other is a researcher in didactics of the discipline. Both evaluators compared their answers and produced a joint analysis of the laboratory manual. The questions of the analysis grid (Table 2, see appendix) often rely on issues previously discussed in the literature studies section; they are supported by the HTD procedure model. There are two main directions:

-How explicit are the elements that may support students in giving sense to the procedure: the scientific question (referred to as the 'problem' in the grid), the structuring tasks, the completeness of the procedure, and the explanations?

-Are there missing tasks in the procedure? At which level in the HTD (from general strategy to details)? What is the degree of freedom in such activities? Can it be considered as a design activity?

We will further see that these questions echo with the teachers' interviews, being consistent or contradictory.

Teacher interviews

In order to elaborate the evaluation criteria, the aim of the second study is to understand what is a 'good' procedure for teachers. Two types of interviews were conducted with teachers who have different profiles.

• Interview A: science teachers at university level, who all teach and do some research. They were asked to do a teacher job (they evaluated students' production). This results in a first list of criteria proposed by these teachers to evaluate a procedure.

• Interview B: in a second stage, other teachers were involved. They are upper secondary school science teachers who have been working on this project. This means that they are 'experienced' teachers who teach laboratories in which students designed (part of) a procedure. They were asked to refine and complete the list of criteria proposed by the first pool of teachers during a focus group session.

Interview A

The first list of criteria is the outcome of a situational interview: a three-step process involving six teachers. The teachers were exposed to a real class situation where students had to write a procedure. The experiment involved the titration of iron in iron-rich water using the spectrophotometric method. This experiment is part of a multidisciplinary first year course at the university. Six university teachers agreed to be involved in our research. Their dual position as teacher and researcher is interesting since they have already dealt with procedures, both as teachers (to produce laboratory manuals to be used by their students) and as researchers. Nevertheless, it is not obvious that the teachers have explicitly previously formulated what is important in a procedure. This explains why we put the teachers in situation of evaluating students' work before defining a list of criteria. The three steps are:

I.

Six teachers were individually asked to write down all the potential errors that the students may make when designing this experiment. They had to classify them as errors related to, either chemistry misconceptions, or the lack of knowledge about what is expected in procedures (e.g. the procedure is not complete, there are missing parameters). The teachers did this work individually and used the laboratory manual as a basis for defining their lists of errors.

II.

One week later, each teacher met individually with a researcher for a 90 minutes semistructured interview about his/her list of errors to explain these criteria verbally, and to clarify them where necessary. The teachers were then asked to assess three students' written experimental procedures corresponding to the iron titration laboratory, and to identify any errors they could find. Again, we asked the teachers to distinguish errors related to chemistry versus errors related to the procedure by itself.

III.

Two weeks later, three among the six previous teachers worked together for two hours. We asked them to produce, by consensus, a list of criteria to evaluate the procedures written by students, based both on the example of the iron titration and on their personal experience.

These criteria had to be general enough to evaluate iron titration, as well as other procedures. These criteria were written on a blackboard. We then gave them a list of errors representative of all the errors that the six teachers had previously identified in the steps I and II of the research. The teachers were asked to verify if these errors correspond to the list of criteria they had established collectively and to modify the criteria if necessary. The list of criteria collectively generated by the teachers is provided in the result part.

Interview B

We interviewed four other teachers (two of them teach physics and chemistry while the two others teach biology and geosciences) after showing them the list generated by the first group of teachers.

We have been working with them for two years on the development of labworks that include the design of experiments. The teachers have tested these activities with their students. We discuss the criteria proposed by the first teachers with these 'experienced teachers', in order to improve them if necessary. This second list is further described as the 'extended list of criteria'.

3-Results and discussion

The first three parts of our results relate to the procedure as it appears in laboratory manuals. It deals with the 'given tasks', i.e. the tasks that are already written by teachers in the procedure and the 'missing tasks', tasks that are not written in the procedure but are necessary to organize the procedure ('structuring tasks level'), or to detail it ('action level'). The second section corresponds to the elaboration of criteria with interviewed teachers to characterize a procedure.

Presentation of the procedure in laboratory manuals

This analysis aims at answering the first research question about the way the procedure is presented to student in laboratory manuals. Table 3 shows the results for the questions Q1 to Q3. The answers are given for each discipline that was studied at both high school and university levels.

[Insert Table 3 about here]

Our analysis reveals that out of a total number of 39 manuals across all five disciplines, 25 manuals (64%) give explicitly the problem to be solved (Q1). In the other manuals, the problem to be solved has to be deduced from the title or subtitles of the laboratory manual, or is given step by step throughout the document. The analysis also reveals that the problems are better explained at high school level. This shows that in laboratory manuals, the teachers often do not favour the comprehension of the problem to be solved by the students, and therefore the meaning of the procedure. Similar results have been found in the literature where problems to be solved are not always given to students. According to [START_REF] Gomes | Students' Performance in Investigative Activity and their Understanding of Activity Aims[END_REF], it is important that student know the aim of an experiment before doing it. For this purpose, first-year chemistry practical manuals from 17 universities in England and Wales were analysed [START_REF] Meester | First-year chemistry practicals at universities in England and Wales: aims and the scientific level of the experiments[END_REF].

They found that the aims of experiments were mostly hidden in the introductory sessions of the experimental descriptions and that students often had to work hard to discover them.

The results of Q2 shows that the procedure is provided in its entirety by teachers in 33% of the studied laboratory manuals (13/39). In 20% of the laboratory manuals (8/39), there are missing tasks in comparison to a complete procedure that would include all the tasks: its structure (structuring tasks), as well as actions (the lower level tasks) (see Figure 2). For example, students are asked to titrate 10 mL of benzoic acid. No details are given about how to proceed and which indicator or titrator to use. Our data suggest a strong disparity between disciplines: biology laboratory manuals contain the greatest proportion of missing tasks (30% or 3/10), while chemistry manuals have far fewer (13% or 2/15), specially at university level (0/5).

Question 3 (Q3) deals with the presence in the procedure of gestures (usually operations in the HTD, but it seldom can be an action when there is an emphasis on the gesture), as well as other information that is different from the tasks described in the HTD (see Figure 2).

A low proportion of laboratory manuals contain information about gesture (13% or 5/39). It is interesting to notice that this kind of information is mainly found in the biology procedures (40% or 4/10). There is no gesture information in the physics and geosciences procedures. Indeed, the gestures usually do not play an important role in physics. We need to further analyse the laboratory manuals to understand if one of the patterns described above would favour an activity of experimental design.

F

Missing tasks and experimental design

In the laboratory manuals, when there are missing tasks, we would like to understand if this could correspond to an activity of experimental design by the students (our third research question). To be able to answer this question, we have been looking for indicators in the laboratory manuals that could be in favour of experimental design:

• The procedure includes missing tasks at structuring level (see Q5 in Table 4).

• The students know what is the question to be solved (see Q1 in Table 3).

• The teachers make explicit the activity of completing the procedure (Q6 in Table 5).

• A high degree of freedom is given to the students when completing the procedure (Q7 in Table 5).

Following de Vries ( 2006) about creativity, needs, and planning in design activities, these indicators state that a missing task cannot be identified as a design activity in several cases: first if students do not know that they have to do something and the goal of this activity; second if they cannot not get involve, at least partly, in the strategy, and if the problem to be solved has a unique solution. We will give below the data corresponding to Q6 and Q7 (see Table 5) and will give a cross analysis of the four indicators mentioned above (Table 6).

[Insert Table 5 about here]

Are students explicitly asked to complete the procedure when there are missing tasks? When only studying manuals, it is not clear whether students really have to design part of the experiment when manuals do not explicitly request this activity. It is likely that the answer depends on the teacher since he/she could give more or less verbal indications during the session. From the results of Question 6, we see that in 46% of the procedures (12/26) it is explained to students that they must design all or part of the experiment (and eventually write the procedure). This average does not reflect the diversity, because the design activity seems to be rarely explicit in geosciences (0/2) or physics (13% or 1/8), while it seems to be more frequently explicit in chemistry (63% or 7/11) and in biology (80% or 4/5).

Most of the laboratory manuals show that students have little freedom when completing the procedure (see Question 7): the freedom was estimated to be small in 81% of the manuals (21/26).

This means that a unique procedure is expected from the teachers, with possibly very few adaptations from one student to another (difference in parameters values for example).

The next step of our analysis is the cross analysis of Q5 (highest level of missing tasks) with Q1, Q6 and Q7 per laboratory manual (Table 6), to understand the learner's experience about experimental design.

[Insert Table 6 about here]

Among the twelve experiments showing missing tasks at the structuring level (Q5 in Table 4), the question to be solved was made totally explicit in most of them (83% or 10/12). The two other laboratory manuals corresponded to manuals in physics (university level), where only some of the structuring tasks were missing.

The completion of the procedure was explicit for 7 of them (33% or 4/12 in biology and 25% or 3/12 in chemistry at high school level). These 7 laboratory manuals are also part of the 10 previous ones that have an explicit question to be solved. The completion of the procedure was not explicit for 41% of the laboratory manuals (5/12, all in physics). In these physics labs, the part of the procedure that needed to be completed corresponds to an adaptation of the procedure given in another part of the laboratory manual, or the reuse of a procedure given in a previous lab. In both cases, students are intended to remind and/or to look for previous laboratory procedures, and this cannot be considered as a design activity. refer to previously defined criteria when they evaluate possible solutions in design-based learning situations. In fact, the criteria given in the literature are of epistemological nature and are stating fundamental properties of procedures. In this way, these criteria constitute a complement of our procedure model.

To answer our fourth research question: 'upon which criteria can teachers evaluate the validity of procedures written by students?', we asked university level teachers, involved in research activity and in the teaching of labworks, to produce criteria to evaluate procedures written by students (interview A). They were asked to analyse procedures written by students to titrate iron in water.

Then, they had to think about criteria that would be applicable to other laboratory procedures.

During the third step of our methodology, the teachers formulated together the following criteria, that we present here as raw data:

a.

The procedure written by students must fit the laboratory objectives. The question to be solved needs to be explicit.

b.

The procedure must be reproducible: the experiment corresponding to the procedure can be repeated. The teachers developed this criterion into the three following ones: conciseness, preciseness, and concreteness.

b.1 The procedure must be concise.

This criterion includes two different ideas; the first one is to avoid long descriptions. For example, in order to obtain a calibration curve in chemistry and biology, different solutions are prepared. There is a common procedure for the preparation of these solutions that should be described only once. Furthermore, the use of a table can make the procedure more concise and clearer. The second idea is to write more or less details depending on the level of knowledge. When students are experienced with a given sub-procedure, they do not need to detail it. They can use an appropriate scientific term to describe a sub-procedure. For example, 'register the baseline' refers to different tasks that do not necessarily need to be a way that is easy to read for the student. For example, information can be given in an HTD with items and sub-items, in tables (repeating tasks), in drawings… Regarding the analysed laboratory manuals, we do not provide here a quantitative result but this criteria proposed by teachers is verified: we observe that, when it is relevant, the studied laboratory manuals' procedures always include tables and figures that avoid long texts.

• Presence of the adequate type of information. This last criterion evaluates the presence of non-technical information that is not strictly part of the procedure, but that may be important for the student. For example, this can be the theoretical justification of the procedure, the safety facts about the manipulated objects, the collection in a datasheet of all the data that will be acquired in the procedure, and so on. The types of information expected in the procedure strongly depend on the didactical contract that is stated between the teacher and the learners. During their interview, teachers talked about 'gestures': it appears that this kind of information should not be written in the procedure. The analysis of the laboratory manuals reveals that it is the case in most procedures, since 87% of them (34/39) have no gestures information. Teachers also suggested the presence of 'explanations' in the procedures: students have to write the justification of the chosen actions or the explanation of their calculations. However, when students are familiar enough with a task, teachers do not expect them to give explanations anymore. The laboratory manuals reveals that task's explanations and theoretical information are present in about 50% of the procedures. The relevance of separating this non-technical information from the technical information was raised during the discussion among teachers, and most teachers believed that it would be too complicated to read the procedure if they were separated. This echoes with an important issue of the manuals' analysis: a compromise is required between the need of non-technical information close enough to the related procedure task and the clarity and integrity of the procedure. they expected students to write in the procedure the main guideline of how the results will be treated in the procedure. The qualitative analysis of laboratory manuals (not shown earlier) reveals that, in physics and geosciences, the data's treatment is often a major part of the procedure. This is not the case in chemistry and biology, where usually only a few ideas about how to process data are given.

Summary: Upon which criteria can teachers evaluate the validity of procedures written by students?

The result is a list of criteria that is organised in 3 parts: Relevance (the function of the experiment),

Executability (the experiment in the laboratory conditions) and Communicability (the description of the experiment).

4-Conclusion

The proposals of this research emphasized that:

• In most cases, the procedures given in laboratory manuals are neither complete nor clearly shown, and some experimental tasks are implicitly devoted to students.

• Despite this fact, writing a procedure is not a usual and explicit activity for students.

• A procedure can be modelled as a Hierarchical Task Diagram (HTD) describing both the structure and the tasks of the procedure, completed by a set of criteria that define the properties of a procedure.

•

This set of criteria could be used to assist teachers when designing inquiry-based activities for their students. These criteria may be applied to the procedures in all the experimental sciences. It can also assist students during experimental design and give them autonomy regarding this task. in our study. Yet, it needs to be improved and validated in other contexts and by other users. We can report its use during the implementation of experimental design within labworks. This is not the purpose of this paper, and we present only the relevant outcomes.

• We actually used the HTD to design labworks that included experimental design. The HTD was a powerful tool for teachers to describe work expected from students during laboratories. Once all the tasks have been described, it was easier to evaluate the cognitive load for each task. Then the teacher could select the amount of tasks devoted to students and finally pre-structure the procedure if necessary [START_REF] Author | Papers presented at the ESERA conference[END_REF].

• The success of experimental design during labworks depended on students' awareness of the useful criteria to assess their procedure. We have conducted preliminary studies to monitor students' ability to deal with the criteria elaborated for the assessment of experimental design. One of them concerned a palaeontology laboratory in which the teacher gave students part of the previous criteria (Author b, 2007). One of the results of this study was the necessity for students to know who will carry out their procedure. When they had to write the procedures for another pair of students of the same school level, we observed that it was easier for them to give more details (cf. criteria of explicitness) compared to the case where the teacher was the only receiver of the procedure.

Our goal was to propose a model of procedures in an educational context, and to study the way the experiment is displayed in traditional laboratories manuals (its goals and means, its procedure, the presence and organization of non-technical information). Finally, we proposed a model using hierarchical tasks diagrams (HTD) for describing the organization of a procedure (Figure . 2) completed by a set of criteria that defines the properties of a procedure (Table 7).

First, we studied manuals and showed that what is expected from students in traditional labworks is not as simple as one could first imagine from previous studies (the so-called 'cookbook' form of labworks), and that the status of the procedure in such manuals is often complex as well. Despite a collection of manuals limited to a single country and limited in number, it appears that the ways The current educational context encourages teachers and students towards inquired based learning, which often includes experimental design. Therefore, tools are needed to help teachers and students in this task. Our purpose here is to underline the need for tools for supporting teachers and students to better handle the technical part of the experiment. We also make some propositions in this direction, since it appears that very little information is available in the literature from the epistemological point of view as well as from the education side. We show here that information is hard to extract from traditional laboratory manuals contrary to what was expected. The problem is partially explicit

F

The problem is implicit We also analysed the procedure for any missing task; these are tasks that are not given by the teacher in the document but that have to be executed during the experiment. It is difficult to identify such gaps in laboratory manuals without being familiar with the learning context in which students will use the manuals / carry out the experiment. For example what appear to be gaps may be operations that the students already know about and just need to be reminded of, but there is no new information they need to assimilate. The teacher may also give additional information during the course.

Q5

Highest level of missing tasks.

Structuring tasks

Actions

Parameters

This is another way of presenting results of Q4. We wanted to identify the level of any missing tasks in the hierarchical task diagram. This will show if students have to write the procedure at a structuring task level, or if the structuring tasks are given in the

  highlight the need to change the pedagogy in science education and emphasize the importance of a positive experience

  suggest that, before designing the full experiment of an open-and[START_REF] Arce | Student-designed experiments in scientific lab instruction[END_REF] find that students showed better understanding, in the exam, of concepts related to the experiments they designed themselves.

  wipe off the cuvette', or 'place the cuvette in the spectrophotometer' are left implicit.

  corresponds to either gestures (the gesture is part of the operation level, see Figure2), or theory, explanations given about the tasks (what happens when performing this task), display and interpretation of results.Q4 to Q7: these questions only concern the laboratory manuals that include missing tasks.

Table 1

 1

	shows the

[Insert Table 1 about here]

Table 1 :

 1 Distribution of analysed laboratory manuals with respect to level of study (secondary school or university) and discipline (biology, chemistry, geosciences and physics).

	o r
	P
	e
	e r
	R
	e
	v i e
	w
	O n l
	y

URL: http://mc.manuscriptcentral.com/tsed Email: editor_ijse@hotmail.co.

uk International Journal of Science Education

	F o r
	P
	e
	e r
	R
	e
	v i e
	w
	O n l
	y

Table 3 :

 3 Analysis of written procedures in 39 laboratory manuals.

					High school level		University level	Total
		Discipline			Biology	Chemistry	Physics	Geo sciences	Biology	Chemistry	Physics
	Questions	Number of	laboratory	manuals	4	10	5	4	6	5	5	39
	Q1-The problem is totally explicit

Table 4 :

 4 Analysis of the 26 laboratory manuals that include missing tasks.

	Page 43 of 49
				High school level		University level	Total
		Discipline		Biology	Chemistry	Physics	Geo sciences	Biology	Chemistry	Physics
		of
				2	8	4	2	3	3	4	26
	Questions	Number	laboratory	manuals
	Q4-Missing tasks in the procedure given in the manuals:
	Structuring tasks	None		0/2	5/8	1/4	2/2	1/3	3/3	2/4	14/26
		Some of them	0/2	2/8	3/4	0/2	1/3	0/3	2/4	8/26
		All of them	2/2	1/8	0/4	0/2	1/3	0/3	0/4	4/26
	Actions	None		1/2	0/8	0/4	0/2	0/3	1/3	1/4	3/26
		Some of them	0/2	5/8	3/4	2/2	2/3	2/3	3/4	17/26
		All of them	1/2	3/8	1/4	0/2	1/3	0/3	0/4	6/26
	Q5-Highest level of missing tasks:
	Structuring tasks			2/2	3/8	3/4	0/2	2/3	0/3	2/4	12/26
	Actions				5/8	1/4	2/2	1/3	2/3	1/4	12/26
	Parameters								1/3	1/4	2/26

URL: http://mc.manuscriptcentral.com/tsed Email: editor_ijse@hotmail.co.

uk International Journal of Science Education

	F o r F o r
	P P
	e e
	e r e r
	R R
	e e
	v i e v i e
	w w
	O n l O n l
	y y

Table 6 :

 6 Cross analysis associating several questions in the same laboratory manual. The results correspond to a number of laboratory manuals per 8 laboratory manuals corresponding to structuring task level level as the highest level of missing tasks.

	Page 45 of 49
		Number of laboratory manuals among the 12
		manuals where the highest level of missing
		tasks is at structuring task level (Q5)
	Q1: Question to be solved is explicit	10/12
	Q6: Completion of the procedure is explicit	7/12
	Q7: High degree of freedom	4/12

URL: http://mc.manuscriptcentral.com/tsed Email: editor_ijse@hotmail.co.

uk International Journal of Science Education

	information, apart from
	the structuring tasks and
	the actions?
	F o r
	P
	e
	e r
	R
	e
	v i e
	w
	O n l
	y

Isabelle/Girault 3/3/11 12:30 Isabelle/Girault 3/3/11 12:28 Isabelle/Girault 18/2/11 23:44 Isabelle/Girault 3/3/11 12:34 Isabelle/Girault 3/3/11 12:06 Isabelle/Girault 18/2/11 23:45 Isabelle/Girault 18/2/11 23:45 Isabelle/Girault 18/2/11 23:45 Isabelle/Girault 18/2/11 23:45 Isabelle/Girault 18/2/11 23:46 Isabelle/Girault 18/2/11 23:46 Isabelle/Girault 18/2/11 23:46 Isabelle/Girault 18/2/11 23:47 Isabelle/Girault 18/2/11 23:47 Isabelle/Girault 18/2/11 23:48 Isabelle/Girault 18/2/11 23:48 Isabelle/Girault 3/3/11 12:37 Isabelle/Girault 18/2/11 23:50 Isabelle/Girault 18/2/11 23:50 Isabelle/Girault 3/3/11 12:36 Isabelle/Girault 18/2/11 23:51 Isabelle/Girault 4/3/11 11:59 Isabelle/Girault 4/3/11 11:59 Isabelle/Girault 4/3/11 12:01 Isabelle/Girault 1/3/11 22:57 Isabelle/Girault 1/3/11 23:09 Isabelle/Girault 1/3/11 23:06 Isabelle/Girault 1/3/11 23:06 Isabelle/Girault 1/3/11 23:07 Isabelle/Girault 1/3/11 23:08 Isabelle/Girault 1/3/11 23:08 Isabelle/Girault 1/3/11 17:13 Isabelle/Girault 3/3/11 10:39 Isabelle/Girault 1/3/11 23:10 Isabelle/Girault 1/3/11 23:10 Isabelle/Girault 1/3/11 17:16 Isabelle/Girault 1/3/11 17:16 Isabelle/Girault 1/3/11 17:17 Isabelle/Girault 1/3/11 23:12 Isabelle/Girault 1/3/11 23:12 Isabelle/Girault 1/3/11 23:12 Isabelle/Girault 1/3/11 17:25 Isabelle/Girault 1/3/11 17:25 Isabelle/Girault 1/3/11 17:25 Isabelle/Girault 1/3/11 23:13 Isabelle/Girault 3/3/11 14:25 Isabelle/Girault 1/3/11 17:27 Isabelle/Girault 3/3/11 10:39 Isabelle/Girault 1/3/11 17:28 Isabelle/Girault 1/3/11 17:28 Isabelle/Girault 3/3/11 10:47 Isabelle/Girault 1/3/11 17:28 Isabelle/Girault 1/3/11 17:28 Isabelle/Girault 1/3/11 17:28 Isabelle/Girault 1/3/11 17:29 Isabelle/Girault 1/3/11 17:29 Isabelle/Girault 3/3/11 10:39 Isabelle/Girault 3/3/11 10:40 Isabelle/Girault 1/3/11 17:30 Isabelle/Girault 1/3/11 17:30 Isabelle/Girault 1/3/11 17:30 Isabelle/Girault 1/3/11 17:30 Isabelle/Girault 1/3/11 17:30 Isabelle/Girault 1/3/11 17:31 Isabelle/Girault 1/3/11 17:31 Isabelle/Girault 1/3/11 17:32 Isabelle/Girault 1/3/11 17:32 Isabelle/Girault 1/3/11 17:33 Isabelle/Girault 2/3/11 21:12 Isabelle/Girault 1/3/11 17:36 Isabelle/Girault 2/3/11 21:12 Isabelle/Girault 1/3/11 18:06 Isabelle/Girault 1/3/11 18:06 Isabelle/Girault 1/3/11 18:06 Isabelle/Girault 1/3/11 18:07 Isabelle/Girault 1/3/11 18:07 Isabelle/Girault 1/3/11 18:07 Isabelle/Girault 1/3/11 18:07 Isabelle/Girault 1/3/11 18:08 Isabelle/Girault 1/3/11 18:08 Isabelle/Girault 1/3/11 18:08 Isabelle/Girault 1/3/11 18:09 Isabelle/Girault 1/3/11 18:09 Isabelle/Girault 1/3/11 18:09 Isabelle/Girault 1/3/11 18:09 Isabelle/Girault 1/3/11 18:09 Isabelle/Girault 1/3/11 18:10 Isabelle/Girault 1/3/11 18:11 Isabelle/Girault 1/3/11 18:11 Isabelle/Girault 1/3/11 18:11 Isabelle/Girault 1/3/11 18:12 Isabelle/Girault 1/3/11 18:12 Isabelle/Girault 2/3/11 21:15 Isabelle/Girault 1/3/11 18:12 Isabelle/Girault 1/3/11 18:12 Isabelle/Girault 3/3/11 10:36 Isabelle/Girault 1/3/11 18:13 Isabelle/Girault 2/3/11 21:18 Isabelle/Girault 1/3/11 18:13 Isabelle/Girault 2/3/11 21:18 Isabelle/Girault 2/3/11 21:18 Isabelle/Girault 1/3/11 18:13 Isabelle/Girault 1/3/11 18:13 Isabelle/Girault 1/3/11 18:13 Isabelle/Girault 2/3/11 21:19 Isabelle/Girault 2/3/11 21:36 Isabelle/Girault 2/3/11 21:36 Isabelle/Girault 2/3/11 22:00 Isabelle/Girault 3/3/11 10:38 Isabelle/Girault 2/3/11 22:00 Isabelle/Girault 2/3/11 22:00 Isabelle/Girault 2/3/11 22:00 Isabelle/Girault 2/3/11 22:00 Isabelle/Girault 2/3/11 22:01 Isabelle/Girault 3/3/11 10:59 Isabelle/Girault 2/3/11 22:01 Isabelle/Girault 2/3/11 22:04 Isabelle/Girault 2/3/11 22:04 Isabelle/Girault 2/3/11 22:04 Isabelle/Girault 2/3/11 22:05 Isabelle/Girault 2/3/11 22:05 Isabelle/Girault 2/3/11 22:05

The teachers did not give any criteria about the results' presentation or interpretation. However, Isabelle/Girault 2/3/11 22:33 Isabelle/Girault 2/3/11 22:33 Isabelle/Girault 2/3/11 22:33 Isabelle/Girault 3/3/11 11:10 Isabelle/Girault 3/3/11 11:10 Isabelle/Girault 3/3/11 11:10 Isabelle/Girault 3/3/11 11:10 Isabelle/Girault 2/3/11 22:33 Isabelle/Girault 3/3/11 11:11 Isabelle/Girault 2/3/11 22:35 Isabelle/Girault 2/3/11 22:35 Isabelle/Girault 3/3/11 11:11 Isabelle/Girault 3/3/11 11:11 Isabelle/Girault 2/3/11 22:36

The model proposed in this paper for describing and evaluating procedures was useful and efficient Isabelle/Girault 3/3/11 11:12 Isabelle/Girault 2/3/11 22:49 Isabelle/Girault 2/3/11 21:38 Isabelle/Girault 2/3/11 21:38 Isabelle/Girault 2/3/11 21:38 Isabelle/Girault 2/3/11 21:39 Isabelle/Girault 2/3/11 21:38 Isabelle/Girault 2/3/11 21:39 Isabelle/Girault 2/3/11 21:39 Isabelle/Girault 2/3/11 21:39 Isabelle/Girault 2/3/11 21:39 Isabelle/Girault 2/3/11 21:39 Isabelle/Girault 2/3/11 21:39 Isabelle/Girault 2/3/11 21:42 Isabelle/Girault 2/3/11 21:42 Isabelle/Girault 2/3/11 21:42 Isabelle/Girault 2/3/11 21:43 Isabelle/Girault 3/3/11 11:30 Mis en forme: Paragraphe de liste, Avec puces + Niveau : 1 + Alignement : 0,63 cm + Retrait : 1,27 cm Isabelle/Girault 2/3/11 21:40 Isabelle/Girault 2/3/11 21:40 Isabelle/Girault 2/3/11 21:45 Isabelle/Girault 2/3/11 21:46 Isabelle/Girault 3/3/11 14:28 Isabelle/Girault 2/3/11 21:47 Isabelle/Girault 2/3/11 21:48 Isabelle/Girault 2/3/11 21:47 Isabelle/Girault 2/3/11 21:47 Isabelle/Girault 2/3/11 21:47

Acknowledgements

The authors would like to thank the French Ministry of Research for financial support, some colleagues (names not given during the submission process, for anonymity reasons) for their involvement with the project. We also thank the teachers involved in this work (the experienced teachers Gilles Baudrant, Martine Biau, Daniel Devallois, and Réjane Monod-Ansaldi, and the university teachers we interviewed: Bernard Bessieres, Herminia Bettega, Florence Courtois, Muriel Jourdan, William Moneta, and Sabine Chierici).

Isabelle/Girault 3/3/11 10:40 Supprimé: U

Supprimé: implies

Supprimé: An analysis of procedure content is needed to provide a descriptive model focused on the operational nature of a procedure. Furthermore, if one aims at fostering experimental design by learners, one has to determine how learners' procedures could be evaluated. A model of the validity of the procedure in educational context is needed. In order to build a model of content and validity of procedures we combine theoretical and empirical approaches. Different kinds of non-technical information could be found within the procedures. The most common one is theoretical information (44% or 17/39 of the laboratory manuals), followed by explanations about the task (36% or 14/39). There are more explanations in the chemistry procedures (44% or 7/15) than in the biology or physics ones (20% or 2/10 for both). An example in chemistry is (see the explanation in italics): 'the procedure requires heating but to a maximum of 80°C otherwise the molecules will be destroyed'.

Supprimé: content

Supprimé

There is also some information related to the results in about a quarter of the laboratory manuals: the presentation of results (23% or 9/39) and the interpretation of results (26% or 10/39). There is far more information about the presentation of results in biology (40% or 4/10) and geosciences (75% or 3/4) procedures than in chemistry (13% or 2/15) or physics ones (0/10). For example, in biology laboratory manuals, students are asked to draw a scheme or a table in order to present the results.

Summary:

How is the procedure presented to students in laboratory manuals?

• The procedure is not explicitly connected to a problem, since the problem to be solved is not made totally explicit in about a third of the analysed laboratory documents.

• The procedure is not very often shown completely to students: Teachers provide all the tasks in 13 (33%) of the manuals.

• Within the procedure, descriptions of the tasks are often accompanied by additional information, mainly theoretical information or explanations of tasks. This means that the procedure itself does not appear clearly to students in manuals due to missing information as well as to extra information.

The results show that 33% of these manuals (13/39) could be considered as strictly cookbook type.

This seems a priori contradictory to our hypothesis based on the literature review, suggesting that the procedures are quite complete in the manuals. It is necessary to know more about the missing From the perspective of students' experimental design, these results show that this kind of activity may have a place in the current practises, since the procedures are not always complete. To foster experimental design, it seems necessary to make explicit the problems that the experiments are intended to answer, which is not always the case in the current practises.

If students are supposed to complete the procedure, it necessarily stands in the procedure tasks that are missing in the manual. The abundance of missing tasks in the manuals is striking, but it is not obvious that a missing task corresponds to any design activity; there are several other possibilities.

We face here some difficulties because the raison d'être of a missing task is not easy to catch from the manuals. This explains why we need to collect more details about these missing tasks and afterward we will focus on identifying design activities.

Missing tasks in the laboratory manuals

We delve more deeply to understand what types of tasks are missing (our second research question).

Results of questions Q4 and Q5 of our analysis grid are summarized in Table 4. These questions concern the subset of manuals where the procedure is not completely specified, that is, 26 out of 39 manuals. In the following discussion, the percentages pertain to this subset. These two questions intend to specify what kind of information is missing, information being classified according to our HTD model, in structuring tasks, actions, and actions parameters.

[Insert Table 4 about here] Table 4 gives an overview of the amount of missing tasks by level of hierarchy in the procedure. Question 4 analyses the missing tasks in the laboratory manual procedures. The main results show that there are no 'missing tasks' at the 'structuring tasks' level in 54% of the studied laboratory manuals (14/26) and 15% of the procedures (4/26) have no task written at structuring tasks level. The latter were found in three biology laboratory manuals and one in chemistry. Missing information at this level means that the strategies of the experiment are not given to students.

If we examine the 'action' level (i.e. the effective part of the procedure in reference to the experimenter's skills), the result is very different. Among the 26 analysed procedures, only three of them included all the actions (12%, 3/26). In most cases, the actions were partially specified by the manuals (65% or 17/26). In the remaining 23% of the laboratory manuals (6/26), all the actions were missing: it concerns biology and chemistry at high school level (50% or 1/2 and 37% or 3/8 respectively) and biology at university level (33% or 1/3). This suggests that students have very often to complete the procedure (88 % or 23/26 laboratories).

Question 5 shows what is the highest level of tasks, written in the laboratory manuals. This is another way of presenting the results of Q4. 46% of the experiments (12/26) lack tasks up to the structuring level. This means that 54% of the experiments (14/26) need to be completed only at a lower level, actions mainly (and occasionally only parameters) while the experiment strategies are given in the laboratory manual. When there are missing tasks at structuring level, there are also missing tasks at the action level, except in one biology manual. In this manual, the students have to choose a type of the procedure, i.e. a strategy, and when it is chosen, the teachers give the detailed procedure.

Summary: What type of tasks is missing?

The structuring tasks are most of the time given by teachers in the laboratory manuals whereas the tasks at a lower level (actions) are most of the time (partly) missing in the manuals.

When there are missing tasks in the procedure, two patterns emerged:

• Missing tasks at the structuring level, mainly in biology and physics laboratory manuals ( 12procedures lack tasks at the structuring level),

• Missing tasks only at a lower level (actions), mainly in chemistry and geosciences (14 procedures). Among the 7 procedures that had an explicit demand for completion, we considered that 33% of them (4/12) allowed the students a high degree of freedom to complete the procedure.

Summary: Under which conditions missing tasks can be associated with experimental design?

The results showed that only 10% of the studied laboratory manuals (4/39) could correspond to an experimental design activity. This is based on the analysis of 4 indicators (missing tasks at structuring level, explicit request to complete these tasks, question to be solved explicit and freedom to complete the procedure).

There is a gap between the 'cookbook' labworks and experimental design. In our analysis, we face several situations when students do not have to choose a procedure, but rather adapt and instantiate one. This was often the case when 'missing tasks' corresponded to 'actions'. In addition, the question to be solved was not always given, since the students' activity was often not appointed as a design task and the degree of freedom students have in this activity was very low. We did not consider this as a design activity with creative input, such as those that can be related to inquiry based learning. However, this is a first step towards experimental design by learners and, indeed, it can be interesting to let learners set up the actions with adequate parameters, or even decompose actions in a set of subtasks. The creativity is very low, but it can still be an interesting exercise that requires a good comprehension of the part of the procedure to be completed, even if it did not require an overview of the whole procedure. It should be noted that our study of manuals is not able to lead to any conclusions or judgements concerning the pedagogy attached to these manuals.

Criteria for the evaluation of student-written procedures

In order to help teachers to design activities in which students are required to design an experiment and subsequently to evaluate the procedure written by students, there is a need to define evaluation criteria. These criteria are important for teachers but can also be given to students to evaluate their procedure. Indeed, [START_REF] Puntambekar | Toward implementing distributed scaffolding: helping students learn science from design[END_REF] emphasized the importance for learners to Here also, there are two ideas. First, the parameters and their values must be given. For example, one cannot write that the baseline is registered when performing spectrophotometer's measures, without writing the name of the solution used to register this baseline. Secondly, this criterion also refers to the precision of the expected results that will be obtained when executing the procedure written by students. The level of precision depends on the experiments. b.3 The procedure must be concrete.

The procedure includes all the necessary practical information to be able to execute the experiment. For example, all the necessary products must be written. Each task has to be written, unless it is a routine task.

c.

No gestures information is expected in the procedure. The gestures aspects are evaluated during the class, when the teacher observes the students.

d.

The procedure must include explanations that depend on the level and knowledge of students. Explanations include the justifications of the tasks they choose, and sometime preliminary calculations. For example, in the iron titration, they must prepare solutions that respect the linearity of the Beer-Lambert's law. This requires to calculate first the maximal concentration corresponding to the linear part of the curve.

e. General ideas about the results' treatment must be included in the procedure.

During the interview, the divergence among teachers mainly concerned the priority they gave to these criteria. We decided not to give any hierarchy in the list of criteria, since it seems very dependent on the experimental situations.

In order to improve the first set of criteria, we interviewed four 'experienced' teachers who practised experimental design with their students (interview B). This section is a discussion that presents the final list of criteria based on the interview of the experiences teachers and on the literature. We found that the initial criteria had to be reorganized, because similar meanings sometimes appear with different names, or different ideas appear with the same name. Few criteria were added to this list of the teachers in order to completely evaluate the experimental design. Some of them come from the literature (i.e. adequacy between the sample and the domain of validity and observation of material / temporal constraints), while the relevance criterion was refined based on discussions with teachers (interview B). We also related what the first set of teachers expected in the procedure written by students (i.e. the criteria described in Section 3.2.1) to what teachers really do themselves in the laboratory manuals (i.e. analysis of laboratory manuals described in Section 3.1). We found that the criteria proposed by the teachers showed common points with the previous analysis of laboratory manuals.

Before we discuss the fine-grained criteria, we propose to reorganize them into a coherent set of classes. For doing so, we use the results issued from the literature presented in the introduction of this paper and we propose a set of three headings that will hold our criteria.

Our first heading deals with the evaluation of the function of the experiment. It relates to the scientific validity of the experiment and corresponds to the two goals given by [START_REF] Kerlinger | Foundations of behavioral research[END_REF] and to the first goal given by [START_REF] Friedland | The concept and implementation of skeletal plans[END_REF]: (A) the relevance criteria evaluates if the experiment 'answers the research question', if it 'meets the experimental goal', and if 'control sources of variance' is taken care of.

The heading B refers to the setting up of the experiment in specific laboratory conditions: (B) the executability criteria examines if the experiment is appropriate to the laboratory, i.e. it can be executed with the objects of the real world, without considering its relevance toward the problem to be solved or its signification for the person who executes it. This heading is similar to the second class of criteria proposed by [START_REF] Friedland | The concept and implementation of skeletal plans[END_REF]: 'will the experiment be successfully applied to the given sample under the given laboratory conditions'. The last heading has not been expressed by other authors and does not evaluate the experiment in itself, but its description, i.e. the procedure as a written document to describe the experiment. This heading comes out from the teachers' criteria and seems to be important in a teaching context: (C) the communicability criteria evaluates to what extent the procedure is understandable by the person who will read it (for execution, evaluation...).

The teacher's criteria are distributed among these three headings and discussed, which yields our final extended list of criteria as presented in Table 7.

[Insert Table 7 about here] A.

Relevance is mainly related to the first criterion given by teachers (see criterion a. above):

the procedure written by students must describe an experiment that fits the laboratory objectives. If the teacher's objective is that learners provide a relevant procedure, the question must be explicit.

The laboratory manuals' analysis shows that most but not all of the studied laboratory manuals include laboratory's objectives. But if we consider the documents where tasks are dedicated to the students, in 27% of the manuals (7/26) the objectives are not totally explicit.

To detail this criterion, we further divide it into three sub-criteria:

• External relevance. In the task of problem solving, the first step is to state hypothesis and evaluate the observable consequences. The latest determines the data to be acquired during the experiment. The first criterion concerning the relevance of the procedure is related to the coherence between the stated hypothesis and the data that are targeted in the procedure. This evaluation of relevance is performed at the level of the link between the hypotheses and the experiment.

• Internal relevance. Once the choice of the data to be acquired is made, it is necessary to determine if these specific data can be acquired within the conditions described by the procedure. It is the strategy employed in the procedure that is questioned here, i.e. the choice of the methods and the main materials.

Isabelle/Girault 3/3/11 11:08 Supprimé: ed

• Quality of data acquisition. This refers to the second definition of 'precise' and also to the broader criterion of 'reproducibility' (sees points b. and b.2 above). It is related to the 'accuracy of the measurements', but it was renamed to satisfy a broader type of experiments (i.e. when the measurement is not a noticeable issue of the experiment). This point can be evaluated according to two aspects that are well described in manuals of metrology [START_REF] Bindi | French: Practical dictionary of metrology: AFNOR[END_REF]: trueness that characterize the distance between the value of the measured data to the real value that is targeted in the procedure and precision that evaluates the dispersion of the different values when measures are taken repeatedly according to the procedure.

B.

In executability, we add some criteria that are not proposed by the teachers during their interviews. These criteria have been extracted from the literature and from previous experiments made by our team where learners had to design experiments (author b, 2007).

• Adequacy between the samples and the domains of validity of the measurement methods and materials. This criterion raises the following question: will the procedure allow the acquisition of the targeted data with the specific samples that will be used during the experiment? In the study of [START_REF] Friedland | The concept and implementation of skeletal plans[END_REF], this criterion is the second in term of importance for choosing an adequate experimental technique.

• Observation of material constraints. The material chosen in the procedure must be available at the beginning of the experiment and its cost has to be examined. The availability of material can also be evaluated all along the experimental procedure: since the procedure describes an anticipated experiment, problems in the management of the material can easily occur. This may happen when the material and data used in a task must have been produced in a previous task (e.g. to use a dilute solution, one must first perform a dilution of the stock solution). Another aspect of material constraints deals with the feasibility: what is described in the procedure must be doable with the selected material (e.g. a flask of 100 mL cannot contain 150 mL of liquid). Finally, the control of hazard is particularly important in chemistry and biology, where potential hazard must be evaluated and the risk be maintained • Observation of temporal constraints. The manipulation described by the procedure must be executable in the amount of time available.

C.

The communicability criterion considers the target person who is going to read and/or execute the procedure (student, teacher…). In this respect, when evaluating the communicability of a procedure, this person must be determined, and his/her competencies in the experimental domain must be known.

• Completeness. This criterion examines whether the procedure gives all the details needed for executing the described experiment. Two ideas have been gathered in this criterion.

Firstly, the level of explicitness, which is given by the depth of the decomposition of the procedure in subtasks, should be appropriate to the target person. The teachers referred to this point with the term 'concise' (see paragraph b.1 above) that relates to the amount of information to be specified in the procedure, according to the level of knowledge of the procedure's user. In our analysis of laboratory manuals, this criterion was revealed by procedures with missing tasks at lower level. Secondly, when the level of explicitness is established, the parameters and materials of the elementary tasks should all be specified.

The first definition of 'precise' (see paragraph b.2 above) corresponds to missing parameters. The same idea is in the 'concrete' criterion (see paragraph b.3 above): all parameters and materials have to be specified in the procedure. An insufficiently explicit procedure could still be executable, as long as materials and parameters are specified at the action level. But it implies that the person who executes this procedure has an operational knowledge of the tasks that do not need to be decomposed into subtasks.

• Structuring. The procedure must be structured to ensure the determination of the sequence of actions that have to be executed. This structure can be temporal (default structure given by the reading) or logical with tests included in the procedure. Furthermore, as the teachers in their first definition of 'concise' have expressed it, the information should be organized in parts of the procedure to be identified. This underlines the fact that a manual is not reduced to a procedure, and that the organization of information in classical laboratories manuals is much more complex and heterogeneous than the cookbook analogy suggests a priori. Actually, procedures are rather difficult to characterize from manuals, because the information is often intricate and not complete. The variety of cases found in our corpus shows that students encounter procedures in a form that is neither stable nor well defined, mainly due to the manner the manuals are organized.

Secondly, we derived evaluation criteria from the literature and from teacher interviews. These criteria evaluate the procedure in itself and not the validity of the hypothesis from which the procedure rises. We believe that the criteria of our extended set (

A. Relevance: the function of the experiment

External relevance between the hypothesis and the quantity to measure

Appendix

Table 2: The analysis grid used to assess laboratory manuals.

Question number

Question

Categories of answers Explanations

Q1

In the laboratory manual, is the problem to be solved explicit?

The problem is totally explicit

The problem is partially explicit

The problem is implicit This question deals with the presence of statements about the experimental goal to achieve by students. Is the procedure related to an explicit problem? We assume that if the problem to be solved is stated explicitly, students have a better chance to understand what they are doing.

Q2

What is the amount of tasks given in the experimental procedure?

Amount of tasks given in the procedure (by teachers):

• Nothing

• A little or average

• A lot or everything

The analysis of laboratory manuals will also show if the procedure is completely given by a teacher to students or if the students have to write (part of) the procedure.

Q3

In the experimental procedure, is there any Presence of information concerning:

•

Gestures

We sought any information in the procedure that is different from the expected description of experimental tasks: structuring tasks or document and the students only need to define the actions.

Q6

When there are missing tasks, is the completion of the procedure explicitly required?

The completion of the procedure is explicit

The completion of the procedure is not explicit

If students are meant to know that a task is missing in the procedure, there should be an explicit request for completion of the procedure.

Q7

Estimation of the freedom given to the students for completing the procedure

Small

Great

We also wanted to evaluate the degree of freedom given to students to complete the procedure. This criterion was related to the variety of procedures that a student could imagine. If a unique procedure was expected from all the students, then the freedom was considered as small.

Page 49 of 49 URL: http://mc.manuscriptcentral.com/tsed Email: editor_ijse@hotmail.co.uk

International Journal of Science Education