Ard W Lazonder
email: a.w.lazonder@utwente.nl

Effects of student collaboration on Web search behavior and search outcomes

Do two heads search better than one? Effects of student collaboration on Web search behavior and search outcomes

Ard Lazonder

Introduction

About thirty years ago, information seeking was a specialism and librarians could call themselves information specialists for a reason. People in need of specific information generally consulted a librarian who had access to various information retrieval systems and was able to reveal its contents by developing sophisticated queries. The librarian would sit down with the information seeker to identify his/her information needs. Taylor (1968) classified this pre-search interview as a negotiation process in which the information specialist works with the inquirer back to his/her information needs and then translates these needs into a useful search strategy.

Nowadays most public and research library catalogues are available online, and the Internet hosts tons of information that is readily accessible to the general public. Internet technology has also found its way into schools where, among other things, the Web is being used as an arena for students to obtain information. Asking students to collect their own learning materials is consistent with contemporary notions of learning that advocate knowledge building through constructive, self-directed activities. The Web is therefore considered a potentially powerful means to mediate this new form of learning.

The advantages of online public access notwithstanding, most students are unlikely to locate information efficiently and effectively. One reason is that the Web suffers from a lack of indexing conventions. Controlled vocabularies, thesaurus classifications, and human indexing are uncommon for Web search engines. More importantly, the Web opens up so much information that designated skills to manage the information-flow are called for. Students generally lack these self-regulatory skills, which caused about half of the high school freshmen from various European countries to consider themselves incapable of searching the Web efficiently and effectively [START_REF] Pelgrum | Curriculum and pedagogy in Pelgrum W J and Anderson R E (eds) ICT and the emerging paradigm for lifelong learning IEA[END_REF]Ten Brummelhuis and Slotman, 2000). In-depth studies by [START_REF] Branch | Junior high students and thinks alouds: generating information-seeking process data using concurrent verbal protocols[END_REF] and [START_REF] Lazonder | Exploring novice users' training needs in searching information on the World Wide Web[END_REF] showed that teenagers are largely unable to select appropriate search strategies (planning), check their progress (monitoring) and assess the relevance of search outcomes (evaluating).

Improving searchers' abilities to self-regulate seems called for, especially because such regulatory skills are the key to successful Web searching. Yet there is little research on effective Websearching instruction. [START_REF] Colaric | Instruction for Web searching: an empirical study[END_REF] examined three instructional treatments to aid searchers in developing queries (ie, planning). The results were equivocal. Undergraduate students receiving worked examples yielded superior syntactic knowledge than students in the two conceptual model conditions. That is, students in the worked-example condition were more proficient in structuring a search query using terminology a search engine can interpret correctly. In contrast, conceptual model instruction was the most effective method for increasing the students' declarative understanding of how a search engine works. [START_REF] Lazonder | Minimalist instruction for learning to search the World Wide Web Education and Information Technologies[END_REF] compared the instructional efficacy of two versions of self-regulatory skill instruction to a control group. His instruction was more comprehensive than Colaric's in that it taught students to plan, monitor, and evaluate a search. Results indicated that students from both selfregulatory skill groups needed about 25% more practice time than students in the control group (who were merely taught procedural skills to operate the search engine). However, self-regulatory skill instruction did not enhance search performance on test tasks. A possible explanation for this finding is that students had developed the self-regulatory skills during practice, but failed to recognize when the acquired skills should be applied on the test tasks.

The work reported here explored whether collaboration might help overcome this inert knowledge problem. The basic assumption is that experienced Web users have at least some understanding of the self-regulatory skills entailed in efficient Web searching. The presence of an equally knowledgeable peer might encourage these users to articulate their thoughts. Such verbalizations might have a positive impact on problem solving by supporting a variety of selfregulating functions (Teasley, 1995). That is, peer interaction is assumed to incite users to negotiate the suitability of search strategies and the relevance of search outcomes. Another alleged advantage of collaborative information seeking is the opportunity for users to critically observe and monitor each other's actions, which may facilitate early detection and correction of errors. Additionally, higher levels of self-regulation are assumed to be associated with better search outcomes.

Research on collaborative Web searching has recently begun to emerge. Some studies have focused on peripheral issues such as the roles individuals can adopt in collaborative information seeking (Prekop, 2002), attitude changes in intergenerational collaboration [START_REF] Caskey | Using parent-student pairs for internet instruction[END_REF], and gender differences in group-based information seeking [START_REF] Large | Gender differences in collaborative Web searching behavior: an elementary school study Information[END_REF]. Another line of research is Collaborative Web searching directed towards designing interfaces and support tools to foster collaborative information seeking activities [START_REF] Chau | Design and development of a multiagent collaborative Web mining system[END_REF][START_REF] Druin | A collaborative digital library for children[END_REF]Twidale and Nichols, 1998). Yet none of these studies report how groups of users (as opposed to individuals) organize and perform their search process, nor do they provide empirical evidence on the alleged advantages of collaborative Web searching.

Evidence on the effects of collaboration on task outcomes can be gleaned from research in other domains. Collaboration has been used successfully to promote learning across a range of curriculum areas including mathematics, science, reading and language arts. Its benefits appear to be relatively independent of group size, student age and ability. The magnitude of these effects was shown in meta-analytical studies comparing collaborative learning with whole-class instruction or individual learning (eg, [START_REF] Cohen | Restructuring the classroom: conditions for productive small groups[END_REF][START_REF] Lou | Small group and individual learning with technology: a meta analysis[END_REF]. These studies report substantial facilitative effects of collaborative learning on both group task performance and achievement on individual posttests. Zimbardo et al (2003) further demonstrated that these findings extend to collaborative test taking. Students who prepared for a test individually performed significantly better when they took that test with a fellow student than on their own. This result is relevant to Web searching as teachers generally require students to utilize existing search expertise to find information on the Web. Practical constraints such as the availability of computers further necessitate teachers to let students perform these searches collaboratively.

Very few studies have investigated whether collaboration promotes self-regulation. To illustrate, in [START_REF] Lou | Small group and individual learning with technology: a meta analysis[END_REF] meta-analysis on the effects of small-group learning with technology, only 3 of the 122 studies addressed the use of self-regulating strategies. Although the results indicated that collaboration yielded higher frequency of using appropriate strategies, the nature of these strategies remains unknown. [START_REF] Fleming | The benefits of peer collaboration: a replication with a delayed posttest[END_REF] give a more detailed account of the benefits of collaboration. They studied fourth-graders' strategy use during memory tasks and found that triads used more sophisticated strategies and were more aware of the usefulness of these strategies than did children who worked individually. Despite these favorable results, several properties of this study constrain its generalization. These include the nature of the target audience, the experimental task and the use of question prompts to promote children's' understanding and application of appropriate strategies. Consequently, there is no guarantee from Fleming and Alexander's study that collaboration will foster adolescents' self-regulation during Web searching.

Research supports the assumption that self-regulation improves academic achievement.

Schauble et al (1991) found that higher levels of planning lead to higher achievement. Schraw (1994) reported similar effects for monitoring and evaluating. Other studies have found that self-regulation is a relatively general process that operates in the same fashion across different domains (eg, Veenman et al, 1997). Together these findings substantiate that planning, monitoring and evaluating can increase achievement on Web search tasks. However, as these studies were conducted on individual students, further research is needed to show whether their findings generalize to collaborative Web searching. In summary, the cited studies strongly suggest that both collaboration and self-regulation improve search outcomes. While it seems plausible that the benefits of collaboration are at least in part attributable to higher levels of self-regulation, there is insufficient evidence to substantiate this claim.

The present study therefore explored whether collaboration stimulates self-regulatory activities in students searching for information on the Web. It sought to answer this question by comparing pairs of students with individual students in Web search tasks. Pairs were expected to perform these tasks faster and better than single students. Pairs were also expected to show relatively higher proportions of self-regulatory activities such as planning, monitoring, and evaluating.

Method

Participants

Participants were first-year students in social sciences from a Dutch university. There were 9 men and 16 women with a mean age of 20 years (SD=1.99). Students could participate in the study as part of a self-chosen dyad or on an individual basis, leading to 9 Pairs and 7 Singles. Six of these Pairs were same-gender couples. Preliminary checks revealed no within-pair differences based on computer experience and Web searching experience.

Materials

A background questionnaire determined the participants' gender, age, computer experience and hours per week spent searching the Web. The Motivated Strategies for Learning Questionnaire (MSLQ) assessed different facets of self-regulation, including self-efficacy, task value, test anxiety, cognitive strategies, and metacognitive strategies (Pintrich et al, 1993). The MSLQ was preferred over other self-report questionnaires because its 53 items can be tailored to specific situations or content areas.

Participants judged each item on a Likert scale that ranged from 1 (not at all true of me) to 7 (very true of me). All scales demonstrated satisfactory psychometric properties with internal consistency reliabilities in excess of 0.79. Six fact-finding search tasks assessed the participants' Web searching abilities (see Table 1).

To preclude possible interfering effects of domain expertise, search tasks addressed a variety of topics.

All searches were conducted using Internet Explorer 6 to access the Google search engine. Google was chosen because it is the predominant search engine among undergraduate students. Search outcomes were written down in an MsWord file. Computers were equipped with a registration program that captured the action from screen and saved it in an AVI movie file.

- -------------------Insert Table 1 about here --------------------

Procedure

Pairs and Singles participated in the experiment one at a time, receiving the same instructions and following the same experimental procedures. All sessions took place in a quiet room equipped with one computer. One week prior to attending their session, students completed the background and MSLQ questionnaires. At the beginning of a session the experimenter introduced participants to the experimental procedures. Participants were then given the first search task, using a counterbalanced administration to anticipate order effects. Participants were allowed 15 minutes maximum to complete this task. If participants completed the task ahead of time, they called the experimenter to receive the second task. If participants exceeded the time limit, the experimenter interrupted task performance and handed out the second task. The other search tasks were administered similarly. Participants were not allowed to return to a previous task during the experiment.

Design and analyses

The study used a between-subjects design with collaboration (Pair, Single) as the independent variable and search outcomes and regulation of search performance as dependent variables. Search outcomes was indicated by performance success and time. Performance success was the number of successfully completed search tasks; time was defined as the mean time for completed tasks.

Three measures were taken to assess regulation of search performance. These measures concerned planning, monitoring and evaluation, and were computed for completed tasks only.

Planning was indicated by the number of times a new search strategy was adopted or an existing strategy was modified. Changing from entering a URL to a keyword search would be an example of adopting a new strategy. When keywords were changed or added but the strategy itself remained intact, this was scored as strategy modification. Monitoring was scored as (1) the proportion of relevant Web sites accessed through the search engine's hit list, (2) the proportion of correct initial answers derived from the target site, and (3) the proportion of incorrect initial answers taken from the target site. Evaluation was defined by (1) the proportion of initial answers that were checked, and (2) the number of times an initially incorrect answer was modified. Checking concerned returning to the same Web page to verify the answer; modifying involved making adjustments to the initial answer based on a culmination of searches.

Given the small sample size, Kolmogorov-Smirnov tests were performed to test the normality assumption. Levene's tests were used to check the homogeneity of variances among cell groups for all dependent variables. In case of homogeneity, univariate ANOVA's were used to examine the effect of collaboration on that variable. Variables with unequal variances were analyzed by means of t tests with separate variance estimates. Pearson correlations were computed to examine the relationship between search outcomes and search performance.

Results

Preliminary analyses were performed to determine whether Pairs and Singles were comparable in terms of computer experience, Web searching experience, and self-regulation. The majority of the Collaborative Web searching participants (54%) spent more than 10 hours per week working with computers and searching the Web. The least experienced students used computers and the Internet for 2 to 5 hours per week. Chi square tests indicated that participants with varying levels of computer experience (χ 2 (2)=0.11, p=0.95) and Web searching experience (χ 2 (2)=0.37, p=0.83) were proportionally distributed over experimental conditions. Likewise, no differences were observed between Pairs and Singles concerning their level of self-regulation (see Table 2) --------------------Insert Table 2 about here --------------------Table 3 summarizes the statistics for search outcomes. Pairs produced significantly higher performance success scores, indicating that they successfully completed more tasks than Singles did. Pairs also needed significantly less time to complete a task.

--------------------Insert Table 3 about here --------------------Search performance was studied to examine the presence of self-regulatory activities in Pairs and Singles' search behavior. As shown in Table 4, Pairs demonstrated a richer repertoire of search strategies, starting a new approach more than once per task. For singles, this measure was significantly lower. No differences were found with regard to the number of modifications, indicating that Pairs and Singles just as often adapted an existing strategy.

In regards to monitoring, Pairs and Singles were equally proficient in judging the relevance of Websites from the search engines hit list. There is however reason to believe that Pairs were better at judging information within sites. When on the target site, Pairs tended to locate the target information more often than Singles did. Furthermore, Singles showed significantly higher proportions of incorrect initial answers: on 22.6% of the occasions they derived an incorrect initial answer from the target site.

Scores for evaluation also differed in favor of the Pairs. They checked their initial answer more often than Singles did, and almost twice as often tried to correct initially erroneous answers. Yet neither difference was supported by standard measures of statistical significance, which is probably due to the high variability of scores.

- -------------------Insert Table 4 about here --------------------Table 5 shows the relationships between search performance and search outcomes. The mean time to complete search tasks was relatively independent of measures indicating regulation of search performance. There was however a significant negative correlation between time and the proportion of initially correct answers, indicating that participants with higher proportions of initially correct answers also needed less time to complete the search tasks. The proportion initially (in)correct answers was also associated with performance success, and so were the proportions of checks and corrections. Successful Pairs and Singles apparently had more initially correct answers, checked their answers more often, and more often revised their answers.

--------------------Insert Table 5 about here --------------------

Discussion

This study examined the influence of collaboration on Web search behavior and search outcomes.

Pairs of students were expected to achieve superior search outcomes and their search behavior was assumed to show higher proportions of self-regulatory activities.

The first hypothesis is clearly supported by the results. Pairs scored higher than Singles on measures that relate to search outcomes. They were faster and produced a greater number of correct responses to the tasks. These findings imply that Pairs search more efficiently: they need less time to successfully complete a task.

The results largely confirm the second hypothesis that collaboration would affect the regulation of search behavior. Pairs exhibited a richer repertoire of search strategies and were more proficient in monitoring and evaluating their search behavior. Superior monitoring arose because Pairs Collaborative Web searching tended to be more proficient in locating the target information when it appeared on screen.

Conversely, Singles overlooked this information approximately 25% of the time, retrieving irrelevant or partially irrelevant information from the target site instead. The results for evaluation also differed in favor of the Pairs. They crosschecked initial answers more frequently and modified initially incorrect responses more than twice as often. Although the latter findings are not supported by standard measures of statistical significance, the direction of effect is supportive. The observed differences in search outcomes can in part be explained by use of self-regulatory skills. Correlational analyses revealed that higher success rates are associated with higher degrees of monitoring and evaluating, and the occurrence rate of these activities differed in favor of the Pairs.

These correlations further imply that checks, and particularly corrections, have converted initially incorrect answers into correct ones. Post hoc analyses support this contention: Pairs successfully corrected 7 out of 19 initially erroneous answers, versus 2 out of 19 for Singles. Performance time and regulation of search performance were unrelated, except for a significant negative correlation with initially correct answers. Fast-performing students apparently locate the target information more quickly, which is another indication that Pairs search more efficiently than Singles. It also proves that performing self-regulatory skills does not necessarily increase the time on task.

These correlations may nevertheless give a somewhat skewed reflection of the relationship between self-regulation and search outcomes. Checks and corrections can easily overshadow the effects of other regulatory activities because they have an immediate effect on search outcomes. The effects of conscious planning and careful monitoring can be wiped out in one fell swoop if students change an initially correct answer to an incorrect one. Post hoc analyses substantiate this. Stepwise multiple regression analysis revealed that the number of corrections accounts for 34% of the variance in search outcomes. However, in forced entry multiple regression none of the self-regulatory activities contribute to the variation in search outcomes.

These findings point out various directions for further research. First and foremost, this study should be replicated. As its sample was relatively small, research using a larger, and therefore more representative sample, is called for. These attempts might reveal additional benefits of collaboration: small sample size research is sensitive to type II errors, which means that there is a chance that true effects do not show in statistical analyses.

Future research might also seek to deepen our understanding of the relation between collaboration and self-regulation. One suggestion would be to examine qualitative differences in selfregulation. The scores presented here show that Pairs outperform Singles. They also indicate which regulatory activities account for these differences, but provide little insight into how exactly they were put into practice. In-depth analyses might uncover how collaboration adds to the quality of selfregulation and whether qualitative differences affect task outcomes.

It might be interesting from a didactical perspective to examine whether collaboration promotes the development of self-regulatory skills. Although self-regulatory skills can be taught through direct instruction, this approach seems inappropriate for technology-enhanced learning environments [START_REF] Lazonder | Principles for designing Web searching instruction Education and Information Technologies[END_REF]. Implementing self-regulatory skills instruction through collaboration could be a fruitful alternative since the presence of a fellow student provides a natural incentive to self-regulate. Future studies should also examine whether gains derived by collaboration carry over to individual performance. At some point students will presumably have to perform a task as individuals.

The instructional efficacy of collaboration thus hinges on the transferability of collaboratively acquired self-regulatory skills to individual task performance.

While the exploratory nature of this study (and its sample size) does not allow for definitive conclusions on the benefits of collaborative Web searching, the results suggest that undergraduate students should work together when searching for information on the Web. As the effects of collaboration generalize across age groups, it is probably fair to conclude that high school students too can benefit from collaborative search efforts. This conclusion fits with the way the Web, or computers in general are being used in many classrooms. Practical constraints such as the availability of computers often leave teachers little choice but to embrace collaborative learning methods. This study suggests that these teachers have made a virtue of necessity. 2. What extinct species is found in Alice in Wonderland? 3. There have been quite some attempts to circle the earth by hot air balloon. How many tries were needed to become the first man to achieve solo circumnavigation of the earth in a hot air balloon? 4. For what reasons was the European Union established? 5. Most people know that indians live in tents. Inside and outside these tents certain traditions and rules should be complied with. To illustrate, indians don't want 'direct' help in taking down their tent. Why is that so? 6. Find the current opinions of three major European countries on a possible war with Iraq. No. of corrections 1.4 (1.5) 3.9 (4.9) 1.67 14.94 1 These scores do not add up to 100% because participants could also derive an incorrect answer from an incorrect site * p<.10 ** p<.01

Collaborative Web searching

Table 1 :

 1 Search tasks used in the experiment 1. When was the first version of MS Windows released?

Table 2 :

 2 Mean scores (and standard deviations) for level of self-regulationNote. Analyses compared scores of individual participants from both conditions (n=25)

		Condition
		Single	Pair	F(1,22)	MSE
	Self-efficacy	5.5 (0.6)	4.9 (0.9)	2.24	0.76
	Task value	6.1 (0.9)	5.7 (1.4)	0.68	1.58
	Test anxiety	2.2 (1.0)	2.8 (1.1)	1.31	1.21
	Cognitive strategies	4.2 (1.2)	4.3 (0.5)	0.40 a	0.48
	Metacognitive strategies	5.5 (0.8)	5.0 (0.7)	1.90	0.54

a t-test with separate variance estimate

Table 3 :

 3 Mean scores (and standard deviations) for search outcomes

		Condition
		Single	Pair	F(1,14)	MSE
	Success	2.0 (1.0)	3.6 (1.0)	9.38**	1.02
	Time (min.)	6.4 (1.7)	4.3 (2.1)	4.78*	3.66
	** p<.01 * p<.05

Table 4 :

 4 Mean scores (and standard deviations) for regulation of search performance

	Condition
	Single	Pair	F(1,14)	MSE

Acknowledgements

This study used the dataset of an unpublished study by Amber Walraven and Bastiaan van Rootselaar, whose permission to re-analyze their data files and screen recordings is gratefully acknowledged. Special thanks to Hans van der Meij and Sarah Manlove for helpful comments provided during the preparation of this paper.


Ard W Lazonder
email: lazonder@edte.utwente.nl

Exploring Novice Users' Training Needs in Searching Information on the WWW

Searching for information on the WWW involves locating a Website and locating information on that site. A recent study implied that novice users' training needs exclusively relate to locating Websites.

The present case study tried to reveal the knowledge and skills that constitute these training needs.

Fourteen pre-university students, classified as novice (n=7) or experienced WWW-user (n=7) performed three Web search tasks. Their actions and verbalisations were recorded. Between-group comparisons showed minimal performance differences. However, qualitative analyses of novice users' search performance suggest that their ability to locate Websites could be enhanced by instructing monitoring skills and advanced system knowledge.

Exploring Novice Users' Training Needs in Searching Information on the WWW Introduction Internet's World Wide Web (WWW) offers a vast volume of information. As more and more people go online, this extensive database is increasingly being consulted by interested lay people with little or no formal training in information seeking. Yet these users have to cope with an information space even information specialists consider difficult to search. Few of them (if anyone) know precisely what information the WWW opens up or how specific information can best be retrieved [START_REF] Bruce | Searching for digital libraries in education: Why computers cannot tell the story[END_REF].

Information scientists and software designers try to improve the accessibility of information on the WWW by designing sophisticated retrieval tools. Although their efforts have been useful to some extent, present-day WWW browsers and search engines still perform merely the routine actions of a search, leaving the brainwork to the user. That is, the users still have to perform the cognitive and metacognitive skills required to retrieve information from the WWW. In case of first-time users, these skills should be trained since research has shown that unguided exploration is a highly inefficient approach to learning to use software (e.g., [START_REF] Kamouri | Training by exploration: Facilitating the transfer of procedural knowledge through analogical reasoning[END_REF][START_REF] Kluwe | Learning by doing in the control of a complex system[END_REF].

But, what knowledge and skills should be taught to efficiently search the WWW? A classical yet fruitful way to answer this question is to identify differences between naive and experienced searchers. Some researchers studied the search performance of students with varying levels of WWW-experience (e.g., [START_REF] Fidel | A visit to the information mall: Web searching behavior of high school students[END_REF][START_REF] Hill | A conceptual framework for understanding information seeking in open-ended information systems[END_REF][START_REF] Hill | Cognitive strategies and learning from the World Wide Web[END_REF][START_REF] Watson | If you don't have it, you can't find it": A close look at students' perceptions using technology[END_REF]. Their work revealed many idiosyncrasies of Web searching and has unmistakably increased our understanding of the search process. However, as these studies did not treat the students' level of expertise as an experimental variable, they are unsuited for identifying novice users' training needs. This observation gave rise to a systematic comparison of novice and expert search performance (Lazonder, Biemans & Wopereis, in press). In this study, the search process was subdivided into two phases: locating a site, and locating information on that site. The experts clearly outperformed their novice counterparts on the first phase of the search. They needed less time to locate a site and successfully located more sites than novice users did. No performance differences were found on tasks requiring participants to locate information on a site. It was therefore concluded that novice WWWusers should be trained at locating sites; their ability to locate information on a site already equals that of experts.

The present study was designed to specify novice users' training needs. The study employed crosscase comparisons to contrast novice and expert search performance, and within-case comparisons to identify inefficient patterns of novice users' search performance. A process model, outlining the stages users tend to process in Web searching, guided the analyses. This model was adapted from theoretically based frameworks for how users search electronic environments such as the WWW [START_REF] Hill | A conceptual framework for understanding information seeking in open-ended information systems[END_REF][START_REF] Marchionini | Information seeking in electronic environments[END_REF][START_REF] Sutcliffe | Towards a cognitive theory of information retrieval[END_REF].

Process model of Web searching

Web searching includes locating a site and, subsequently, locating information on that site. Both phases of the search comprise four activities (goal formation, strategy selection, strategy execution, monitoring) which are described below. As this study concentrates on locating Websites, the process model merely details this part of the search, treating the 'locate information' phase as a black box (see Figure 1).

------------------------------- insert Figure 1 about here -------------------------------

Goal formation

Before initiating a search, a user must first sense a need for information. Information needs thus constitute the users' search goal, identifying the information they aim to retrieve. The user may decompose the overall goal into subgoals addressing distinct parts of the search (locate site, locate information). Users may also search the WWW when no specific goal is present apart from the intention to explore the information space [START_REF] Chen | Internet browsing and searching: Use evaluations of category map and concept space techniques[END_REF][START_REF] Marchionini | Information seeking in electronic environments[END_REF][START_REF] Sutcliffe | Towards a cognitive theory of information retrieval[END_REF]. However, such explorations are rare in an experimental setting where the experimental task serves as the goal, designating what to look for, and sometimes even where to look for it. As users can identify the search goal by reading the task description, goal formation was not taken into account in the present study (cf. [START_REF] Guthrie | Locating information in documents: Examination of a cognitive model[END_REF] Strategy selection Search strategies comprise the activities a user consciously selects, applies, and monitors to perform a search [START_REF] Marchionini | Information seeking in electronic environments[END_REF]. By selecting a search strategy, the user decides on the approach to the search problem. Generally speaking, the user may choose between (1) entering a site's URL, (2) browsing subject categories, and (3) content-based searching. Since search strategies represent information seeking skill [START_REF] Sutcliffe | Towards a cognitive theory of information retrieval[END_REF], experienced searchers were expected be faster and better at selecting appropriate strategies. Less knowledgeable users are more likely to adopt suboptimal or even naive strategies (e.g., [START_REF] Hill | Cognitive strategies and learning from the World Wide Web[END_REF][START_REF] Marchionini | Finding facts vs. browsing knowledge in hypertext systems[END_REF][START_REF] Vassileva | A task-centered approach for user modeling in a hypermedia office documentation system[END_REF]. Whether a strategy is effective depends on the features of the search task at hand. For example, entering a URL is the best way to locate a site which URL is known, but its effectiveness is minimal in case the user is unaware of potentially relevant sites.

Strategy execution

Having selected a strategy, the user executes it. Strategy execution typically involves skills like clicking hyperlinks, entering keywords, and scrolling a page. Performing these skills requires at least some basic understanding of the WWW browser and search engine. For example, to locate a site by content-based searching a user must know where to enter a query, how to use multiple search terms, and which button to click to start the search. As experts have a more extensive, well-structured knowledge base than novices [START_REF] Chi | Categorization and representation of physics problems by experts and novices[END_REF]Ericsson & Smith, 1991), they are predicted to execute a strategy swiftly and correctly. Novice searchers with insufficient system knowledge will perform less efficiently because they need more time to perform an action and because they perform incorrect actions (e.g., [START_REF] Fidel | A visit to the information mall: Web searching behavior of high school students[END_REF][START_REF] Hill | Cognitive strategies and learning from the World Wide Web[END_REF][START_REF] Watson | If you don't have it, you can't find it": A close look at students' perceptions using technology[END_REF]. It is important to note that the skills in itself are not particularly difficult to perform because they also apply to (and, hence, transfer from) operating other Windows programs (cf. [START_REF] Kamouri | Training by exploration: Facilitating the transfer of procedural knowledge through analogical reasoning[END_REF][START_REF] Kluwe | Learning by doing in the control of a complex system[END_REF].

Monitoring

Users monitor their search by evaluating intermediate search outcomes. That is, they assess whether the provisional search outcomes might fulfil their information need, and decide what ought to be done next to locate a relevant site [START_REF] Hill | A conceptual framework for understanding information seeking in open-ended information systems[END_REF]. Users automatically skip this step if they locate a site by entering its URL. This strategy produces no intermediate outcomes because it directly opens up the requested site. In case of searching or browsing, intermediate search outcomes come in the form of a hit list. Users evaluate the list of potentially relevant sites from summary and evaluative metainformation [START_REF] Harris | Evaluating Internet resources[END_REF]. Summary metainformation includes a site's title, abstract, content summary, and so on. Evaluative metainformation includes all types of information that provide judgement of a site's content. Examples are relevance ratings, number of hits, and keywords found on a site. Given their thorough system knowledge, experts were expected to take advantage of both types of metainformation. Novices will mainly refer to summary metainformation to monitor their search.

Depending on the presumed relevance of the search outcomes, the user may decide to alter the search or to start a new search. In both cases, the user returns to a preceding stage in the model. The user may also choose to view the content of a site, thus shifting the focus of the search from 'locating sites' to 'locating information'.

In sum, experienced searchers were assumed to perform each step of the model swiftly and correctly.

Novices were expected to perform these steps less efficient. More specifically, novices would need more time and make more errors when selecting a strategy, executing it, and monitoring the outcomes.

The reported case study examined whether these predictions prove correct. The study also analysed novice users' performance in order to identify inefficient behaviour that could indicate training needs.

Method

Participants

Fourteen fourth graders from pre-university education participated in the study. There were nine males and five females with a mean age of 15.3 (SD=. 6). Participants were denominated as novice (n=7) in case they had worked with the WWW for less than ten hours and considered themselves proficient in at most four out of twelve Internet facilities. Experts (n=7) had over fifty hours of WWW-experience.

Their self-reported proficiency ranged from eight to eleven. Both groups were equivalent with regard to sex, ethnic background, and expertise in the task domain (i.e., Dutch literature). They differed with respect to age, however, with the experts being about one year younger that the novices.

Materials

The experiment was performed on Pentium II computers with Microsoft Internet Explorer 4.0. A Dutch search engine called Ilse was used to access information on the WWW (http://www.ilse.nl).

Each computer was equipped with a registration program that captured the participants' actions and utterances in an AVI (Audio-Video Interleaved) file.

A background questionnaire recorded some personal data and assessed the participants' level of WWW-experience. A self-report questionnaire measured the participants' perceived proficiency in using browsers and search engines. Items dealt with basic issues such as following hyperlinks and printing Web pages, but also addressed advanced topics like downloading files and creating a personal home page. Participants scored each item on a 'yes/no' scale, indicating whether they thought themselves able to perform that task individually. Scores could range from zero to 12.

Three search tasks assessed the participants' capacities in locating Web sites. These tasks were identical to those used by Lazonder et al. (in press). Task complexity was determined by the level of inferencing required to deduce the site's URL from the task description (cf. [START_REF] Khan | Searching through cyberspace: The effects of link display and link density on information retrieval from hypertext on the World Wide Web[END_REF][START_REF] Mosenthal | Defining prose task characteristics for use in computer-adaptive testing and instruction[END_REF]. The simple task referred explicitly to the site to be located, while the more complex tasks involved some inferencing (see Table 1).

- -----------------------------insert Table 1 about here ------------------------------Procedure All sessions took place in the schools' computer class and lasted up to one hour. Individual participants attended one session (the questionnaires were administered prior to the experiment). A time limit of 13 minutes was attached to each task, but participants were free to relinquish a task.

Participants were instructed to work individually and to think aloud during task performance. After the instruction, participants started working on their first task. Consecutive tasks were administered when participants completed or abandoned a task, or when they exceeded the time limit. If necessary, the experimenter reminded participants to think aloud with undirected prompts such as "What are you doing?", and "What are you looking at".

Results

Strategy selection

Table 2 reports the mean time to select a search strategy. Experts needed less time to select a strategy for task 1 and 2. No significant difference was found on task 3.

------------------------------------insert Table 2 and 3 about here -------------------------------------

The experts also tended to select better search strategies. On task 1 and 2, all experts immediately identified the site's name or address from the task description and transformed it into a successful strategy (see Table 3). Some novices overlooked these references; others were unable to use them properly. On task 1, for example, one novice identified the URL from the task description, but used it as a query instead of entering it on the address bar. Another novice completely overlooked the URL and searched for the collection of poems. On the second task, two novices initially disregarded the site's name and searched for keywords related the subject matter (i.e., Piet Paaltjens, Snikken en Grimlachjes).

Task 3 showed no difference in strategy selection between novices and experts. As this task contained no reference to a particular site, selecting a successful strategy hinged on information seeking skills rather than system knowledge. Searching for multiple keywords would be the best strategy, but none of the participants chose this approach on their first attempt. Three experts and three novices selected the second best approach: searching for the author of the biography. The other participants choose an unsuccessful strategy.

Table 3 also shows the strategies that were selected on subsequent attempts. On task 1 and 2, participants who initially selected a successful strategy held on to their choice. Some novices whose initial strategy was unsuccessful choose a successful approach at a later stage; others did not. On task 3, most participants tried more than one strategy, regardless of the quality of their initial attempt. In general, the participants systematically searched for every keyword presented in the task description, and there were no striking differences between novice and experienced participants.

Strategy execution

On task 1, experts tended to be faster at executing a search strategy (see Table 2). The transcripts indicated that the experts were faster because they executed a strategy readily and correctly, whereas novices made errors and performed unnecessary actions. For example, novices pressed the 'start' or 'search' button to begin a search, pressed the 'stop' button to clear the screen, explored the bookmarks, and consulted the search engine's help function. These actions seem to result from insufficient system knowledge. In fact, several novices wondered about the operation of the Internet browser and the search engine as was indicated by utterances like "...you automatically get there by clicking. But I'm not sure if that goes for the black words too..." and "I'm looking for this Internet site, but I have no idea how to get there".

On task 2 and 3, novices committed fewer errors and explorations and, consequently, executed their strategies three to four seconds faster compared to task 1. The novices even performed slightly faster than the experts, but this difference failed to reach statistical significance due to a high variability of scores.

Monitoring

Table 2 also shows the mean time to monitor search outcomes. On task 2, experts and novices took an equal amount of time to evaluate the sites from the hit list. The experts were faster at monitoring on task 3. The standard deviation for the novices further indicates considerable within-group differences on this task. Some novices spent much time on monitoring search outcomes; other established their merit at a single glance.

- -----------------------------insert Table 4 about here - -----------------------------Monitoring occurs by judging the search engines' hit list. As Table 4 shows, participants predominantly used summary metainformation for this purpose. Evaluative metainformation was rarely referred to; even the experts consulted it on an irregular basis. The following examples show some of the adverse consequences of disregarding evaluative metainformation.

On task 1, the novice who searched for the title of the collection of poems chose to inspect a site which URL differed from the URL presented in the task description. The novices who employed an unsuccessful strategy on the second task acted accordingly. They choose to examine a site which title and summary were relevant, ignoring the fact that its URL did not refer to SMC in any way. Another novice correctly searched for SMC, but preferred a site called 'SMC-information point' to the SMChomepage. Her choice was incorrect, and she might have realised this if she had considered the relevance estimates (63% vs. 100%).

The consequence of disregarding evaluative metainformation was most apparent on the third task.

Almost every hit list contained a site that was relevant, but had an indecisive title and abstract. Most participants overlooked this site, although the relevance estimate and 'keywords found' showed its significance. Four participants (1 expert, 3 novices) eventually located this site, but their decision to view its content was motivated by inferences about the site's content ("I will look at this poetry site because he has written poetry as well") rather than an examination of evaluative metainformation.

Monitoring also entails the decision about what action to take next. Surely, a full description of every decision and its resulting actions would go beyond the scope of this paper. Some decisions must be pointed out, however. On task 3, five experts and two novices decided to refine a search. They made the right decision, but generally failed to put it into effect. Instead of selecting the 'refine' command or using Boolean operators, most participants added keywords to the original query.

Ironically, this broadened their search because the search engine, by default, searched for 'any word' instead of 'all words'. Although this imperfection could have been negated by referring to the 'keywords found', the participants predominantly judged the relevance of a site by its title and abstract, thus undoing the advantage of their advanced search strategy.

Discussion

Web searching involves locating a site and, subsequently, locating information on that site. A recent study showed that experienced WWW-users outperform inexperienced users on the first part of the search (Lazonder et al., in press). Starting from this conclusion, the present study examined why novices are less efficient at locating sites and what kind of instructional support is needed to enhance their search performance.

The cross-case comparisons show minimal differences between experts and novices. On the simple task, experts tended to be more proficient in selecting and executing a search strategy. However, these differences decreased as the search task became more complex. On the one hand, experts needed more time to select a strategy and choose successful strategies just as often as the novices did. On the other hand, the novices became more skilled in operating the search engine, which reduced the initial differences in strategy execution.

There may be several reasons why the predicted findings failed to appear. The small sample size is probably the most obvious explanation. Furthermore, the experts were experienced WWW-users, yet their level of expertise was not up to that of true experts such as librarians or information specialists; the novices were more knowledgeable than absolute beginners for they had up to ten hours of WWWexperience. Taken together, the anticipated differences might have shown if more participants with more divergent levels of WWW-expertise were compared. This prediction should be verified in future research because extreme group comparisons provide a sound basis establishing the instructional content.

Another interpretation is that the two groups differed with regard to system knowledge and skills, but not on information seeking skills. This study postulated that these types of expertise would go hand in hand-a reasonable assumption because the WWW is exclusively being used to find information. Unfortunately, the within-case comparisons disproved this presumption by showing that the experienced participants lacked information-seeking skills (especially monitoring skills). As a consequence, no instructional implications can be drawn from the cross-case comparisons until research establishes the relation between WWW-experience and information seeking skills.

The within-case comparisons yield interesting directions for enhancing novice users' search performance. For example, several findings suggest that the basic operation of the search engine should not be elaborated on in the instruction (cf. [START_REF] Marchionini | Information-seeking strategies of novices using a full-text electronic encyclopedia[END_REF]. Initially the novices executed their search strategies somewhat inefficiently. They made errors, explored the function of task buttons and menu commands and consulted the on-line help. However, their hands-on skills improved rapidly during the course of the experiment (cf. [START_REF] Khan | Searching through cyberspace: The effects of link display and link density on information retrieval from hypertext on the World Wide Web[END_REF] even though they received no instructional support at all. In spite of these performance gains, novices never took full advantage of the search engine's potentials. For example, they were unaware that they could change the search engine's default from 'any word' to 'all words' in order to search for multiple keywords. They also overlooked the 'refine' and 'broaden' commands, and failed to use Boolean operators. Teaching novices the meaning and use of these advanced features might yield a further increase in performance efficiency.

Novice searchers should also be taught to identify and interpret the information that appears on screen. Even experienced participants hardly used system cues such as relevance ratings and 'keywords found' to monitor search outcomes. The consequences of this neglect were most apparent on the complex task. Participants frequently overlooked relevant sites, examined the content of irrelevant ones instead, and failed to notice that their attempt to refine a search had an adverse effect.

Because the complex task closely resembles the information-seeking problems students encounter in real practice (i.e., open-ended search tasks without reference to a particular site), their monitoring skills need improving in order to search the WWW efficiently.

Finally, the results are indecisive as to whether novices should be taught to select search strategies.

The participants' initial approach to the search tasks showed great similarity, although the novices tended to be less proficient at strategy selection on tasks designating which sites to visit. Their choice of strategies on subsequent attempts depends, at least to some extent, on their ability to monitor search outcomes. For example, participants decide to refine a search only if they are convinced the intermediate outcomes are relevant to their information needs. Given the participants' weak monitoring skills, their true capability to select search strategies cannot be assessed.

Acknowlegdement

Netherlands Organization for Scientific Research (NWO) is gratefully acknowledged for funding this project. This research was conducted while the author was supported by a grant of the Foundation for Behavioral and Educational Sciences of this organization (575-33-006).

Note. Numbers indicate the number of participants that selected a strategy. Successful strategies are marked with an *.


Ard W Lazonder
email: lazonder@edte.utwente.nl

Minimalist Instruction for Learning to Search the World Wide Web

Keywords: World Wide Web, learning, minimalism, instruction, self-regulation

This study examined the efficacy of minimalist instruction to develop self-regulatory skills involved in Web searching. Two versions of minimalist self-regulatory skill instruction were compared to a control group that was merely taught procedural skills to operate the search engine. Acquired skills were tested on Web search tasks and search tasks in an online library catalogue. Self-regulatory skills instruction was found to increase practice time by 25%. However, it did not enhance search performance on the test tasks. Explanations are advanced for these findings and topics for further research are identified.

Introduction

Information seeking is becoming the buzzword for this decade. Its popularity was probably incited by the proliferation of Internet and the World Wide Web. A rapidly growing number of people search the Web to obtain information for professional and private purposes. The use of the Web for educational purposes is increasing accordingly. This is aptly illustrated by the increasing number of assignments that require students to collect source materials from the Web. The pervasive use of the Web in schools and at home raise the impression that students are able to search the Web. Yet, this is a false impression since nearly half of the second graders from Dutch secondary education consider themselves incapable of operating Web browsers and search engines (Ten Brummelhuis and Slotman, 2000). A comparative study further showed that these findings are consistent with the situation in many European countries [START_REF] Pelgrum | ICT and the emerging paradigm for lifelong learning[END_REF].

These studies convey the need to teach students the skills to access and navigate the Web.

Although literally hundreds of books, articles and Web sites designed to train users to become skilled Web searchers are available, few of these publications appear to have been tested in the classroom or have been developed from the research on the topic [START_REF] Carroll | Expert Internet information access[END_REF]. The present research therefore set out to examine how students can best be supported in gaining Web searching expertise.

Minimalist instruction [START_REF] Carroll | The Nurnberg Funnel: Designing minimalist instruction for practical computer skill[END_REF]) may foster the development of procedural skills to operate Web search engines. Minimalism originated from the field of software training in the early 1980s. Its development was prompted by qualitative research studies of people learning to use computer applications. These studies revealed idiosyncratic difficulties people encounter when first trying to use software. The minimalist approach proposed a set of design principles and heuristics to anticipate these problems. In short, these principles aim to minimize the extent to which instructional materials obstruct learning and focus the design on activities that support learner-directed activities. Empirical studies have substantiated the instructional efficacy of these principles in teaching novice users the basics of various computer programs (e.g., [START_REF] Carroll | The Nurnberg Funnel: Designing minimalist instruction for practical computer skill[END_REF][START_REF] Lazonder | The minimal manual: Is less really more?[END_REF][START_REF] Ramsay | Designing minimal computer manuals from scratch[END_REF][START_REF] Lazonder | The minimal manual: Is less really more?[END_REF]. In all of these studies minimalist instruction lead to faster training and better learning as compared to other instructional approaches.

However, Web searching involves more than operating search engines. The Web opens up so much information that designated skills to manage the information-flow are called for. That is, Web users need self-regulatory skills to plan, monitor and evaluate their actions. Since the minimalist approach has predominantly been applied to teach procedural skills, it is unclear how it might be extended to support the development of self-regulatory skills. Nor has it been established whether minimalist instruction in self-regulatory skills enhances search performance. Yet it is important to consider these issues because self-regulatory skills are of greater importance to Web searching than are procedural skills [START_REF] Jacobson | Teaching reflection: Information seeking and evaluation in a digital library environment[END_REF][START_REF] Marchionini | Information seeking in electronic environments[END_REF]. This paper addresses these issues from a theoretical and practical perspective. Section 2 explores the applicability of minimalism to teaching self-regulatory skills. To this end, the characteristics of the minimalist approach are compared to the conditions for successful self-regulatory skill instruction.

Section 3 addresses the issue of how the minimalist approach should be extended to foster the development of self-regulatory skills. Two instructional strategies are advanced, and their application in minimalist instruction is described. The sections that follow report a study that attempted to offer initial evidence regarding the efficacy of these strategies to enhance search performance.

Instructional conditions for self-regulatory skill learning

Instructional conditions denote the terms under which the instruction leads to the desired learning outcomes. The present learning outcomes primarily relate to acquiring the self-regulatory skills Web searching entails. The ability to transfer these skills to different information retrieval systems such as online public access catalogues (OPAC) and CD-ROMs is another, probably equally desirable learning outcome.

Three instructional conditions apply to the acquisition of self-regulatory skills. The first is that selfregulatory skills should be taught in context [START_REF] Boekaerts | Self-regulated learning: A new concept embraced by researchers, policy makers, educators, teachers, and students[END_REF][START_REF] Hattie | Effects of learning skills interventions on student learning[END_REF][START_REF] Puntambekar | Design and development of MIST -A system to help students to develop metacognition[END_REF]. Self-regulatory skills are always used within a given task domain. Their spontaneous application in that content area seems to depend on the students' conditional knowledge about when and where to use particular skills. Without this knowledge, the self-regulatory skills remain inert, and students may fail to invoke them during task performance. The instruction should therefore associate the self-regulatory skills with the circumstances in which they are applicable. This in turn may increase the students' perceived utility of a self-regulatory skill [START_REF] Shunk | Self-regulation and academic learning: Self-efficacy enhancing interventions[END_REF].

The second instructional condition states that there must be ample opportunity to practice the selfregulatory skills [START_REF] Garner | Metacognition: Answered and unanswered questions[END_REF]Vermunt, 1998;Weinstein et al., 2000). Students not only need to understand that a self-regulatory skill exists or when it should be used, they also need to know how to put it in practice. Self-regulatory skills require significant effort to learn; hence, practice should be extensive. Following from the first instructional condition, the opportunities for practice should be designed around authentic tasks.

The third condition refers to the mechanism of cognitive scaffolding [START_REF] Boekaerts | Self-regulated learning: A new concept embraced by researchers, policy makers, educators, teachers, and students[END_REF][START_REF] Jong | Self-regulation: Regulation of the learning process and learning to regulate-a process view[END_REF]Shunk and Zimmerman, 1997;Vermunt, 1998). As self-regulatory skills require significant effort to learn, students can easily become overwhelmed by the number of regulatory activities they have to perform. Scaffolded instruction aims to avoid this problem by offering students an adaptable and temporary support system during the initial phase of the learning process [START_REF] Boekaerts | Self-regulated learning: A new concept embraced by researchers, policy makers, educators, teachers, and students[END_REF]. The amount of external support is inversely proportional to the students' level of self-regulation. Initially, a fair amount of external support is given. As students become more proficient, the external support is gradually faded.

The literature on self-regulation presents no instructional conditions for transfer. There is, however, reason to believe that instruction that meets the abovementioned conditions will generalize beyond the context in which it is provided. In case of Web searching, transfer concerns the spontaneous use of self-regulatory skills in searching different information retrieval systems such as OPACs and CD-ROMs. This requires near transfer: the self-regulatory skills are performed in almost identical situations that differ from Web searching only with regard to the operation of the search system.

Near transfer is based on rule automation. It occurs spontaneously when skills automatized in one context are triggered in another, highly similar context. There seem to be three guidelines for the design of instruction that aims at rule automation and, consequently, near transfer [START_REF] Perkins | Teaching cognitive and metacognitive strategies[END_REF][START_REF] Salomon | Rocky roads to transfer: rethinking mechanisms of a neglected phenomenon[END_REF]Van Merriënboer and Paas, 1990). The first one implies that extensive, step-by-step practice is required to automatize a skill. By repeated practice, behavior becomes fast, effortless, and unlimited by processing capacities. The second guideline asserts that practice should be divergent, linking the skills to the contexts in which they will be applied. Although automation facilitates evocation of the elements that are applicable in comparable situations, the instruction should force the skills to adapt to similar contexts, yielding the ability to apply these skills in situations different from the instructional setting.

However, extensive practice on varying, authentic tasks is relatively ineffective. The lack of guidance and modeling during practice seems to impose a high cognitive load. This may cause students to direct their attention to other, nonessential parts of the task; it may even cause them to completely loose track of what they are doing. The instruction should therefore reduce cognitive load and redirect the student's attention to the relevant aspects of the task. This may be achieved by using completion problems to practice a skill (Van Merriënboer, 1997). Completion problems contain a given state, a goal state, and a part of the solution. During practice, students have to complete the partial solution.

These guidelines match with the instructional conditions for self-regulatory skill learning. Both sets of directives advocate ample opportunities for practice in realistic settings. The use of completion problems can be considered a form of cognitive scaffolding (Van Merriënboer, 1997). Both support mechanisms seek to reduce cognitive load by offering external support. Both mechanisms also entail a gradual decrease of external support when learners acquire more experience. As for completion problems, external support comes in the form of a part of the solution to training tasks. During the initial stages of learning, students have to produce a small part of the incomplete solution. As experience increases, the students' share in solving the problem increases accordingly.

These similarities lead to the tentative conclusion that instruction designed in compliance with these conditions will foster the acquisition and transfer of self-regulatory skills. The question then becomes whether instruction designed in compliance with the guidelines of the minimalist approach satisfies these conditions. The essence of the minimalist approach is embodied in the principles and heuristics presented in Table 1. The remaining part of this section discusses if and how these minimalist principles comply with the conditions for successful self-regulatory skill instruction. This in turn provides a brief description of the minimalist approach; a comprehensive overview appears in Van der Meij and Carroll (1995).

- --------------------------------Insert Table 1 about here ---------------------------------Minimalist instruction is anchored in the task domain, thus complying with the condition of contextual instruction. This minimalist principle reveals itself especially in the design of practice.

Training tasks include genuine activities that represent the core tasks of a domain. Minimalist instruction also advocates ample opportunities for practice. It provides users with an immediate opportunity to engage in meaningful learning activities. Explanations are reduced to the bare minimum and definitions are operational instead of conceptual. That is, they are presented directly before or after the relevant action steps and explain what something 'does' rather than what it 'is'. The gradual decrease of external support (i.e., cognitive scaffolding) pertains to the minimalist principle 'support reading to do, study and locate'. To support the flexible ways in which people process instructional materials, minimalist instruction capitalizes on the users' accumulating understanding of the program.

Minimalist instruction initially provides detailed guidance and support. Throughout the instruction the amount of external support is gradually reduced as a function of the users' need for detailed instructional support. Among other things, this requires a gradual fading of action steps. An example of the use of fading techniques can be found in [START_REF] Carroll | Ten misconceptions about minimalism[END_REF].

In addition, the minimalist approach already holds some elements that facilitate self-regulation.

Error-information, for example, models problem solving behavior and reduces cognitive load by helping users to detect, diagnose, and correct errors [START_REF] Osman | Metacognition research and theory: Analysis and implications for instructional design[END_REF]. Its presence also invites users to monitor task performance. Action statements like 'Look on the screen to...' and 'Check if the dialog window...opens' also support monitoring. These prompts replace lengthy explanations in case information can be found on the screen or can easily be inferred.

The length of minimalist instruction may call for some concern, however. Minimalist instruction is typically brief, whereas developing self-regulatory skills requires a considerable amount of practice.

Yet, the time on task does not necessarily impede the development of self-regulatory skills. Van der Meij (1999) stressed that minimalist instruction leads to 'high quality training time'. It reduces training time by 25 to 52%, and at the same time, yields superior task performance. The time learners spent processing minimalist instruction is obviously well spent.

Web-searching instruction

In sum, this section attested that the minimalist approach satisfies the conditions for successful selfregulatory skill instruction. It was therefore concluded that minimalist instruction can support the acquisition and transfer of self-regulatory skills. The question of how minimalism can be larded with self-regulatory skill instruction is addressed in the next section.

Instructional strategies for self-regulatory skill learning Both theoretical and empirical evidence suggests that embedded instruction is the most effective way to teach self-regulatory skills [START_REF] Bielaczyc | Training in self-explanation and self-regulation strategies: Investigating the effect of knowledge acquisition activities on problem solving[END_REF][START_REF] Hattie | Effects of learning skills interventions on student learning[END_REF][START_REF] Osman | Metacognition research and theory: Analysis and implications for instructional design[END_REF]. However, the joint practice of skills to operate the search engine and self-regulatory skills to manage search behavior may provoke additional cognitive load. [START_REF] Chandler | Cognitive load while learning to use a computer program[END_REF] showed that the coordinate handling of manual, keyboard and screen requires considerable mental effort.

Adding self-regulatory skills could make it even harder for learners to allocate their attention to the relevant parts of the learning task. The success of embedded self-regulatory skill instruction thus seems to hinge on its potential to minimize the learners' cognitive load [START_REF] Osman | Metacognition research and theory: Analysis and implications for instructional design[END_REF].

According to [START_REF] Perkins | Teaching cognitive and metacognitive strategies[END_REF], three instructional strategies may serve this purpose. The memory aid strategy starts from the notion that much of the cognitive load in self-regulatory skill learning has a mnemonic character. Learners have to remember a skill, remember when to use it, remember how to use it, and so on. Shifting this load on to an external object (a memory aid) may free processing capacity for other learning activities. The timesharing strategy is guided by the idea of parallel processing. It acknowledges the difficulty involved in simultaneously performing procedural and self-regulatory skills by explicitly prompting the learner to shift attention from executing a search, to regulating search performance. Automation of self-regulatory skills is yet another strategy to reduce cognitive load. Because automatized skills require fewer cognitive resources, the learner can direct more attention to new skills. However, this strategy is inappropriate to accommodate cognitive load during the early stages of skill acquisition. Automation requires extensive practice. Paradoxically, practice only induces automation if cognitive load is properly managed. Worded differently, automation should be the outcome of initial skill learning rather than a prerequisite to it.

Consequently, only the memory aid and timesharing strategy were considered in designing the selfregulatory skills instruction. In the timesharing instruction, self-regulatory skills were taught concurrent with the procedural skills to operate the search engine. Brief instruction prompts incited learners to perform a self-regulatory skill every time it was needed. The memory aid instruction introduced the self-regulatory skills before the operation of the search engine on the basis of a diagram. This diagram served as a job aid learners could consult in performing practice tasks to learn to operate the search engine. A comprehensive description of the instructional materials appears in the next section.

An experiment was performed to assess the effectiveness of both instructional strategies. This study compared the memory aid and timesharing instruction to a control group that was merely taught to operate the search engine. Both experimental conditions were expected to yield better search performance than the control condition. That is, learners in the memory aid and timesharing condition were expected to outperform their control counterparts on Web search tasks and OPAC search tasks.

The memory aid and timesharing condition were compared to reveal which instructional strategy is best suited for teaching self-regulatory skills. No predictions were formulated concerning this comparison.

Several learner characteristics might affect experimental findings. For example, students with high levels of prior Web-experience might show superior search performance during and after practice compared to students with low levels of Web-experience (e.g. Hill and Hannafin, 1997;[START_REF] Khan | Searching through cyberspace: The effects of link display and link density on information retrieval from hypertext on the World Wide Web[END_REF][START_REF] Lazonder | Differences between novice and experienced users in searching information on the World Wide Web[END_REF]. The students' level of prior self-regulation might have a similar effect on learning activities and learning outcomes. To anticipate these extraneous effects, the study controlled for the students' level of Web-experience and self-regulation, thus allowing for a valid comparison of instructional strategies. The study did not consider the students' knowledge of the topics being searched. Hsieh-Yee (1998) asserted that domain expertise becomes a factor only after a certain amount of search experience had been acquired. Research further suggests that domain expertise can be left out of account when teaching a homogeneous group of students to search the Web [START_REF] Lazonder | Learning to search the World Wide Web[END_REF].

Method

Participants

Participants were second and third graders from a school for secondary education. There were 58 males and 44 females with a mean age of 14.0 (SD=0.8). Their level of Web-experience ranged from less than 1 hour to over 200 hours. Participants were randomly assigned to the memory aid condition (n=32), timesharing condition (n=39), or control condition (n=31).

Instructional materials

An introduction manual was designed to familiarize students with the browser and search engine.

The manual had a minimalist design and addressed basic procedural skills that were prerequisite to the self-regulatory skill instruction (e.g., entering a URL, following hyperlinks, using task bar buttons). It did not treat the self-regulatory aspects of the search process.

Three versions of minimalist instruction (timesharing, memory aid, control) were designed to teach self-regulatory skills. All versions addressed the procedural skills to operate the search engine, differing exclusively with regard to the instructional strategy to teach self-regulatory skills. The timesharing instruction introduced the self-regulatory skills 'on the spot'. Throughout the entire manual, self-regulatory skill instruction was integrated with the instructions to learn to use the search engine. The instruction explained the 'what, how, and why' of a self-regulatory skill (e.g., [START_REF] Osman | Metacognition research and theory: Analysis and implications for instructional design[END_REF][START_REF] Shunk | Self-regulation and academic learning: Self-efficacy enhancing interventions[END_REF]. Instructional support was gradually faded from a full description of a skill, through a brief description, to a question that prompts participants to perform that skill [START_REF] Jong | Self-regulation: Regulation of the learning process and learning to regulate-a process view[END_REF]). An illustrative page of the timesharing instruction is shown in Appendix B.

The memory aid instruction addressed the same self-regulatory skills, explained the 'what, how, and why' of each skill, but differed with regard to presentation. All self-regulatory skills were introduced before the operation of the search engine. The self-regulatory skills instruction centered on a diagram of the search process (see Appendix A) that was explained and practiced on the basis of a Web search task. The memory aid instruction also used a different fading technique. It merely contained the full descriptions and introduced them in a single sweep in the first chapter of the instruction. Subsequent chapters exclusively dealt with the operation of the search engine (see question ("It took me great effort to understand this lesson") that was administered after each session.

Participants answered this question on a scale ranging from 1 (totally disagree) to 5 (totally agree).

Dependent variables for the test phase were learning outcomes (i.e., achievement on Web search tasks) and transfer (i.e., achievement on OPAC search tasks). For both measures, the mean number of completed tasks and successfully completed tasks was scored. Given the time constraints in the test session, higher scores on these measures automatically imply faster task performance. Performance efficiency was indicated by the number of successfully completed tasks to the time to complete these tasks.

The effect of instructional condition on these measures was assessed with MANCOVA's.

Univariate analyses followed when a significant multivariate effect was observed. The test statistics for the covariates are reported in case of significance only. Missing data were excluded on an analysisby-analysis basis.

Results

Table 2 displays the mean learning activity scores. There was a multivariate main effect of instructional condition on these measures (F(4,194)=2.79, p<.05). A univariate effect was found for practice time (F(2,97)=4.96, p<.01). Post hoc comparisons of adjusted means revealed that this effect arose because control participants completed practice significantly faster than participants in the other two groups did. Memory aid and timesharing participants needed an equal amount of practice time.

Instructional condition also affected mental effort scores, but this difference did not reach traditional levels of statistical significance (F(2,97)=2.56, p<.10). The participants' level of Web experience affected learning activity scores as a covariate (F(2,96)=4.86, p<.05). Significant univariate effects were found for practice time (F(1,97)=8.29, p<.01) and mental effort (F(1,97)=4.64, p<.05).

Unstandardized regression coefficients indicated that higher levels of Web-experience were associated with lower amounts of practice time and mental effort. The participants' prior level self-regulation had no effect on learning activities.

- --------------------------------------------insert Table 2 and Table 3 about here - --------------------------------------------Table 3 summarizes the scores on the Web search tasks. Instructional condition had no effect on these measures (F(6,168)=.73). Participants in all three conditions completed as many Web search tasks, produced an equal number of correct answers and performed equally efficient. Similar findings were obtained on the OPAC search tasks (F(6,148)=.82), indicating that participants in all groups performed these tasks equally proficient. The covariates also yielded no effects on achievement outcomes. Search performance on both types of tasks was not affected by the participants' level of Web-experience and self-regulation.

Discussion

This paper examined whether and how minimalist instruction may support the development of selfregulatory Web searching skills. Theoretical evidence suggests that minimalist instruction may serve this purpose. To validate this presumption, two instructional strategies were proposed and their effect on learning activities and learning outcomes was assessed.

Results obtained during practice indicate that self-regulatory skill instruction extends training time by approximately 10 minutes (i.e., 25%). This increase is probably due to the volume of the instructional materials. Control participants merely learned to operate the search engine, which is obviously less time-consuming than learning to plan, execute, monitor, and evaluate a search. Control participants also seemed to require less mental effort to understand the instructional content, but this difference was not supported by standard measures of statistical significance. Yet the mental effort scores do imply that both instructional strategies are equally fit for managing students' cognitive load.

Contrary to expectations, self-regulatory skill instruction did not enhance performance on Web search tasks. Students in all groups performed these tasks equally successful and efficient. These findings seem to suggest that students in the memory aid and timesharing condition did not develop the self-regulatory skills -or at least not enough to yield superior search performance. However, there may be other reasons why the anticipated effects failed to appear. One is the time constraints in the test session. Students in the memory aid and timesharing condition may have decided not to employ the acquired self-regulatory skills because it would penalize them in the sense that they might complete fewer tasks. A second reason is sensitivity of the test. The Web search tasks measured in a holistic manner what the self-regulatory skill instruction intended to improve: search performance.

Although this outcome measure is ecologically valid and has substantial practical value, it may not reflect the actual differences in self-regulatory skills. Attempts to reanalyze the data are currently being performed. It should be noted, however, that this is merely a theoretical issue: practically speaking, the self-regulatory skills instruction did not enhance search performance.

Memory aid and timesharing instruction were further expected to enhance transfer. This hypothesis too was not supported by the results. Students in all groups performed equally successful and efficient on OPAC search tasks. One reason for nontransfer is the amount of practice. Transfer may have failed because the instruction was too brief to automatize the self-regulatory skills. This inadequacy can be accommodated by further extending memory aid and timesharing instruction. More practice improves the facilities to automatize self-regulatory skills, which in turn, is expected to enhance near transfer.

An alternative interpretation is the distance of transfer. Near transfer occurs when students perceive high similarity between the learning tasks and the transfer task. The OPAC search tasks were therefore designed to resemble the Web searching practice tasks as much as possible. Nonetheless the students might have considered searching an OPAC and searching the Web as distinct activities. In that case, the OPAC search tasks would require far transfer. Clearly, the instruction was not designed to support this type of transfer.

To conclude, this study failed to confirm the assumed functionality of minimalist instruction to develop self-regulatory Web searching skills. The concise nature of minimalist instruction is probably the main reason why this is so. Future research should therefore examine whether extending minimalist instruction with repeated step-by-step practice would overcome this problem. Future attempts should also consider using different transfer tasks. Near transfer might, for example, be indicated by the students' capacity to search the Web with a different search engine. Their ability to search an OPAC would then be considered an instance of far transfer. Comparing students' search performance on these tasks might shed a decisive light on the functionality of minimalism to develop self-regulatory skills.

Despite the lack of supportive evidence, this study holds important practical implications.

Educators aiming for students to become proficient Web searchers should carefully consider course

Web-searching instruction duration. The results of this study suggest that extensive repeated practice is required to develop selfregulatory skills. Paradoxically, students typically resent rehearsing the same skills over and over again, causing them to skip most review exercises [START_REF] Carroll | The Nurnberg Funnel: Designing minimalist instruction for practical computer skill[END_REF]. The amount of repeated practice should therefore be well balanced, allowing students to develop a skill without loosing interest.

Integrating repeated practice in self-regulatory skills throughout the curriculum seems a potentially fruitful alternative, especially since subject teachers increasingly ask students to obtain information from the Web.

Table 1 Characteristics of the minimalist approach and their relationship with the conditions for selfregulatory skills instruction

Minimalist principles and heuristics

Instructional conditions 1. Choose a task-oriented approach

• Design instructional activities that are real tasks

• Let components of the instruction reflect the task structure

Choose an action-oriented approach

• Provide an immediate opportunity to act

• Encourage and support exploration and innovation

• Respect the integrity of the user's activity

Support reading to do, study and locate

• Gradually fade out action information

• Omit information that can easily be inferred

• Make chapters brief and self-contained Note. Adapted from Van der Meij and Carroll (1995).

•

  Prevent mistakes whenever possible • Provide error information when actions are error-prone or when correction is difficult • Provide error information that supports detection, diagnosis, and correction • Provide on-the-spot error information

Table 2

 2 Mean learning activity scores(and standard deviations)

		Training time (min.)	Mental effort
	Memory aid	39.9 (11.5)	2.0 (.8)
	Timesharing	38.0 (12.2)	2.1 (.9)
	Control	30.1 (11.6)	1.6 (.7)

Table 3

 3 Mean test scores(and standard deviations)

			Web search tasks			OPAC search tasks
		Completed	Successfully	Efficiency	Completed	Successfully	Efficiency
		tasks	completed		tasks	completed
			tasks			tasks
	Memory aid	1.9 (.7)	1.5 (.6)	35.0 (23.1)	2.5 (1.0)	1.6 (.6)	31.1 (27.9)
	Timesharing	1.9 (.9)	1.7 (.9)	31.0 (22.5)	2.4 (.9)	1.5 (.5)	27.0 (19.9)
	Control	1.8 (.5)	1.4 (.6)	27.0 (16.1)	2.3 (.9)	1.4 (.6)	24.9 (18.6)

Acknowledgement

Netherlands Organization for Scientific Research (NWO) is gratefully acknowledged for funding this project. This research was conducted while the author was supported by a grant of the Foundation for Behavioral and Educational Sciences of this organization (A575-33-006). Special thanks to Hans van der Meij and Mark Gellevij for comments on earlier versions of this paper.

Appendix C), but the students were encouraged to consult the diagram in learning to operate the search engine. The diagram thus served as a flexible support tool students could adapt to their own information needs. It was given to memory aid students only.

The control condition contained no self-regulatory skill instruction; it merely addressed the procedural skills to operate the search engine. In this respect, the control condition matched the memory aid and timesharing condition. It addressed the same procedural skills, contained the same instructions, and used the same practice tasks. Hence the design of the control instruction was identical to the instructional materials exemplified in Appendix C.

Questionnaires and tests

Two questionnaires were administered to assess the participants' levels of Web-experience and self-regulation. A background questionnaire determined the participants' experience in working with the Web. It also gathered some personal data such as age, sex, and ethnic background. The Motivated Strategies for Learning Questionnaire (MSLQ) assessed different facets of self-regulation, including self-efficacy, task value, test anxiety, cognitive strategies, and metacognitive strategies [START_REF] Pintrich | A manual for the use of the Motivated Strategies for Learning Questionnaire (MSLQ)[END_REF]. The MSLQ contained 53 items; each item was judged on a Likert scale that ranged from 1 (not at all true of me) to 7 (very true of me). The scale 'test anxiety' was not taken into account due to insufficient reliability (α=.68). Coefficient alphas for the other scales ranged from 0.80 to 0.88.

Two tests were used to assess achievement outcomes. A performance test recorded acquired Websearching skills. It contained four fact-based Web search tasks that were comparable (but not identical) to the practice tasks. These tasks dealt with general topics such as finding the entrance fee to a museum or the data of a sailing camp. A transfer test assessed search performance with a new search system. It consisted of four fact-driven OPAC search tasks that asked participants to find various books on the painter Paul Gauguin.

Procedure

All sessions took place in a computer class equipped with 30 Pentium II computers. Internet Explorer and a Dutch search engine called Ilse (www.ilse.nl) were used to access information on the Web. A registration program was installed on each computer to record the participants' test performance. It captured the action from screen and saved it as an AVI movie file. The experiment was conducted in nine groups of 16 to 26 students. Each group attended four sessions of 50 minutes each. The time between sessions was one week.

At the beginning of the first session, the participants were informed on the experiment's goal and received instructions. Next, they filled in the background questionnaire and the MSLQ. During the remaining part of this session, participants worked through the introduction manual. The same manual was used for all conditions. The second and third session involved the self-regulatory skills training.

Participants received a manual (memory aid, timesharing, or control). The three types of manuals were proportionally divided among the participants in each group. The manuals contained sheets that marked the beginning and end of a session. These sheets also prompted participants to write down the time (as displayed on screen) and to rate the mental effort to grasp the subject matter of that particular session. The tests were administered during the fourth session. After a brief introduction, participants were given 20 minutes to complete each test. A counterbalanced administration was used to anticipate order effects. Participants were encouraged to perform the search tasks in given order, but they were free to relinquish a task. They were not allowed to use their manual or to consult the experimenter during both tests.

Design and analyses

The study used a between-subjects design with instructional condition (memory aid, timesharing, control) as the independent variable and the participants' prior levels of Web-experience and selfregulation as covariates. Web-experience was defined as the time participants' had worked with the Web. Level of self-regulation was indicated by the mean MSLQ scores.

Performing a keyword search

Selecting a keyword 1 Click

The search engine Ilse appears on the screen.

SEARCH TASK

For your Dutch language class, you have to interpret a text entitled Whizkids on the electronic highway. This text was published in Time Magazine. The author, prof.dr. ir. P. Akkermans, frequently uses computer terms like 'black hole', 'URL', 'gateway' en 'digital literacy'. Find the meaning of these terms in a Web dictionary.

1 Select the best keyword (Note: you may select only one keyword)

2 Search for that keyword 3 Evaluate the search outcomes 4 Find the meaning of the computer terms listed in the search task.