Katharina Scheiter
email: k.scheiter@iwm-kmrc.de.

Peter Gerjets

Richard Catrambone

Making the abstract concrete: Visualizing mathematical solution procedures

Keywords: problem solving, skill acquisition, worked-out examples, visualizations, pictures, animation, imagery, cognitive load

This article reports on an experiment investigating the effects of different kinds of computerbased visualizations on the acquisition of problem-solving skills in the domain of probability theory. Learners received either purely text-based worked examples, text plus an instruction to mentally imagine the examples' contents, or the possibility of retrieving either static pictures or concrete animations that depicted the problem statement and the problem states achieved by applying a specific solution-step. It could be shown that frequent use of both static pictures and imagining the examples' contents improved performance on isomorphic problems. Frequent use of animations, in contrast, led to substantial increases in learning time and a decrease in performance. Thus, the use of concrete animations to visualize solution procedures was more harmful than helpful for conveying problem-solving skills. Reasons for these outcomes are discussed in the light of cognitive load theory.

Making the Abstract Concrete: Visualizing Mathematical Solution Procedures

Understanding mathematical solution procedures is a necessary prerequisite for solving both familiar problems, as well as novel problems requiring adaptation of acquired solution procedures. However, it has been noted that students often face severe difficulties in understanding solution procedures even when they have received elaborate instructional explanations of the individual solution-steps. This may result from the fact that the solutionsteps are often conveyed in a rather abstract way so that learners experience difficulties in imagining which changes in the problem state are achieved by applying a specific solutionstep to a problem.

The use of multimedia learning environments may offer ways to overcome these difficulties [START_REF] Mayer | Multimedia learning[END_REF]. In multimedia learning environments, information presentation can be accomplished by using different representational formats (textual and pictorial) which may be processed in different sensory channels (auditory and visual). Additionally, information presentation is not restricted to static displays (e.g., diagrams, pictures, written text), but the representations used can involve changes over time (e.g., dynamic visualizations, spoken text).

This research addresses the effects of augmenting a purely text-based hypertext environment called HYPERCOMB by different kinds of visualizations. HYPERCOMB teaches students how to calculate the probability of complex events. Calculating complex-event probabilities are related to situations where the probability of randomly selecting a particular configuration of elements out of a set of elements has to be determined (cf. Figure 1).

The main instructional principle underlying HYPERCOMB is the use of worked-out examples for conveying knowledge on different problem categories. Worked-out examples consist of a problem statement and a step-by-step solution procedure. Research over the last especially in well-structured domains like mathematics, physics, or computer programming [START_REF] Atkinson | Learning from examples: Instructional principles from the worked examples research[END_REF].

In a series of prior experiments, different example formats in HYPERCOMB [START_REF] Gerjets | Designing instructional examples to reduce intrinsic cognitive load: Molar versus modular presentation of solution procedures[END_REF] were compared. Conventionally designed or molar examples focus on explaining how to categorize problems according to multiple structural task features and how to apply category-specific solution formulas. Molar examples demonstrate a fast and convenient approach to problem solving, but may also impose high cognitive demands on learners who have to simultaneously consider multiple problem features in order to solve a problem. Thus, an alternative modular example format was developed which completely avoids references to molar concepts like problem categories or formulas and where complex solutions are broken down into smaller, meaningful solution elements that can be conveyed separately. Though it was demonstrated that a modular example format reduces cognitive load and boosts performance compared to conventionally designed molar examples (Gerjets et al.), there is still space left for improvement. We assumed that this space might be claimed by the benefits achieved through the use of visualizations.

According to Mayer's multimedia principle (2001) embellishing textual learning materials by static pictures or dynamic visualizations (i.e., animations) helps promote the learners' understanding of instructions. With regard to the acquisition of problem-solving knowledge, visualizations of worked-out examples may first help learners understand the situation described in the problem statement (i.e., the initial problem state) and thus to correctly represent its meaning in a situation model [START_REF] Nathan | A theory of algebra-word-problem comprehension and its implications for the design of learning environments[END_REF].

Second, visualizations of the solution-steps may promote an understanding of changes with regard to the initial problem state that are achieved by applying a solution-step to a problem.

Visualizing worked-out examples can be done by presenting either static pictures, animations, or an instruction to mentally imagine the examples' contents.

Static Pictures

Making the abstract concrete 5 Static pictorial representations are known to foster the immediate and delayed retention of facts contained in the accompanying text (cf. for a review [START_REF] Carney | Pictorial illustrations still improve students' learning from text[END_REF][START_REF] Levin | Validating functions of pictures in prose[END_REF]. Moreover, with regard to the acquisition of problem-solving knowledge in domains like mathematics and physics, the added value of abstract diagrammatic and graphical representations has been acknowledged (e.g., [START_REF] Shah | Review of graph comprehension research: Implications for instruction[END_REF][START_REF] Vekiri | What is the value of graphical displays in learning?[END_REF]. These types of visualizations are said to be computationally effective in that they facilitate specific inferential processes needed for some learning tasks [START_REF] Larkin | Why a diagram is (sometimes) worth ten thousand words[END_REF]. However, the researchers have found no research on whether concrete pictures depicting a problem situation and its modification caused by the application of a solution procedure promote acquisition of problem-solving knowledge in abstract domains. There is research on the use of video demonstrating the application of observable procedural skills (e.g., tying nautical knots, [START_REF] Schwan | The cognitive benefits of interactive videos: Learning to tie nautical knots[END_REF]; however, this does not allow for predicting the effectiveness of concrete visualizations for teaching abstract mathematical skills that do not have a natural representation in the real world.

It can however be assumed that representing the problem statement in a picture will help to understand which features and interrelations of objects are relevant to the solution of the problem and thus to build a situation model of it [START_REF] Nathan | A theory of algebra-word-problem comprehension and its implications for the design of learning environments[END_REF]. For instance, in the sprinter example in Figure 1 seven sprinters on the racetrack are depicted out of whom three can win a medal (i.e., gold, silver, and bronze), represented by the pedestal. Additionally visualizing the solution-steps may support learners in inferring which change is achieved by applying a solution procedure when they compare the new problem state to the previous one.

For instance, comparing the picture illustrating the second solution-step with the one depicting the first step helps clarify the fact that only six sprinters are eligible for the silver medal in the second solution-step, because one sprinter has already been assigned the gold medal in the first solution-step and thus already stands on the pedestal.

Animations

Making the abstract concrete 6 An animation is "any application which generates a series of frames, so that each frame appears as an alteration of the previous one, and where the sequence of frames is determined either by the designer or the user" (Betrancourt & Tversky, 2000, p. 313). Thus, animations do not only depict the current state of objects; they also deliver information concerning changes of objects and their position over time (motion) as well as information concerning the direction of these changes (trajectory, [START_REF] Rieber | Animation in computer-based instruction[END_REF]. Several findings suggest that animations can be used successfully for delivering abstract content such as mathematical rules, Newton's laws, or computer algorithms [START_REF] Baek | Color, graphics, and animation in a computer-assisted learning tutorial lesson[END_REF][START_REF] Byrne | Evaluating animations as student aids in learning computer algorithms[END_REF][START_REF] Catrambone | Using animations to help students learn computer algorithms[END_REF][START_REF] Rieber | Animation in computer-based instruction[END_REF]. With respect to conveying problem-solving knowledge, the visual-spatial properties of the visualization may be used to deliver information on the current problem state and its relevant structural features. Moreover, the changes over time that can be depicted in an animation may be used to reflect the changes in problem states that result from applying a solution-step to a specific problem state of the example -without the need to compare multiple representations as is necessary when using static pictures.

However, learning from animations is known to impose certain requirements on the learners that they may have difficulty meeting [START_REF] Betrancourt | Effect of computer animation on users' performance: A review[END_REF]. [START_REF] Pane | Assessing dynamics in computer-based instruction[END_REF] have demonstrated that learners often fail to use animations to a sufficient extent and thus miss important information. [START_REF] Lowe | Extracting information from an animation during complex visual learning[END_REF] suggests that learners have trouble focusing on the most relevant parts of an animation and are often distracted by salient, but irrelevant details. Furthermore, due to the dynamic changes of the display, the information that has to be remembered can only be viewed for a limited amount of time and may therefore have vanished before learners have identified it. Finally, dynamic visualizations may lead to overly passive information processing and prevent learners from performing effortful cognitive processes required for a deeper understanding [START_REF] Palmiter | Animated demonstrations for learning procedural computer-based tasks[END_REF][START_REF] Schnotz | Individual and co-operative learning with interactive animated pictures[END_REF]. In line with Salomon's principle of least effort (1984) Making the abstract concrete 7 learners may refrain from deeply processing the contents of an animation and passively watch it like an ongoing movie.

Imagery

To circumvent the problem of passive information processing, learners can be instructed to imagine the contents of a text-based worked-out example by mentally envisioning objects and their relations included in the examples. In this case, learners do not receive any pictorial representations, but are told to construct their own visualizations. Hodes (1992) compared the effectiveness of imagery instructions and instructional visuals for fact recall and understanding. Both instructional methods were helpful in inducing an imagery strategy and in improving posttest performance; however, for some performance measures, achievements due to presenting external visuals were larger than for the imagery instructions. [START_REF] Ginns | When imagining information is effective[END_REF] showed that imagining procedures (vs. studying textbased materials) was only helpful for learners possessing sufficiently high prior knowledge in the domain. This replicates findings of [START_REF] Cooper | Learning by imagining[END_REF] who found interactions between domain-specific abilities and the type of instruction given.

Ginns et al. explained these findings by assuming that "to successfully imagine a procedure or a concept, all of the relevant elements must be processed simultaneously in working memory.

That may be possible only after a schema had been constructed" (p. 231). Contrary to the experiment outlined in the remainder of this paper, learners in these studies (Cooper et al.; Ginns et al.) were not instructed to imagine a worked-out example's concrete objects. Rather, they were told to envision the operations they needed to perform in order to solve a problem (e.g., adding an addition sign to an equation) and the examples did not contain any concrete elements, but only the mathematical information.

Comparing the three different visualization methods (static pictures, animations, and imagery) to each other yields insights on specific advantages as well as drawbacks associated with these methods. The trade-off between these advantages and drawbacks may determine Making the abstract concrete 8 learning outcomes (Table 1). Learning from static pictures may engage learners in a more active way, because they have to compare multiple visualizations in order to understand the to-be-learned solution procedures. As this information is permanently visible, these comparisons may be conducted without overloading the cognitive system. However, learners may make erroneous inferences and may miss important information so that their internal representation of the solution procedures may be incorrect as well as incomplete. Moreover, because the external representations cannot be modified by a learner, they are not adapted to his or her preferences or prior knowledge level. Dynamic pictures may reduce learner activity and induce a rather passive style of processing. The cognitive demands imposed by the need to extract the relevant information from a changing display and to memorize this information may be rather high. Unless animations are highly interactive, they do not allow for any adaptation to a learner's preferences or prior knowledge. However, an advantage of dynamic visualizations of solution-steps is that all the information that is needed to understand problem states and their changes is in principle contained in the representation and thus it is correct as well as complete. Finally, while imagery instructions should foster learner activity in an optimal way, the need to envision all information may require rather demanding processes that may be overwhelming for learners with low cognitive prerequisite knowledge in the domain [START_REF] Cooper | Learning by imagining[END_REF][START_REF] Ginns | When imagining information is effective[END_REF]. There is also the danger that learners may miss important information or that they make incorrect inferences. A possible advantage of mental imagery is that, because it is based on self-generated images, these images are adapted to a learner's preferences and prior knowledge level. These trade-offs between possible promises and drawbacks were investigated in the experiment outlined in the remainder of the article.

Experiment

Method

Participants. Participants were 124 students (88 female, 36 male) at the University of Tuebingen, Germany, who participated for either course-credit or payment. One participant in the static-pictures condition had to be excluded from the analysis, because learning time was more than four standard deviations above the mean learning time in this condition. Twentynine participants were in the text-only condition, 30 in the static-pictures condition, 32 in the animation condition, and 32 in the imagery condition. The average age was 23.60 years (SD = 4.28). Most of the participants were familiar with the domain of probability theory and were experienced computer users.

Materials and procedure. HYPERCOMB consisted of a technical instruction, a short introduction to the domain, an example-based learning phase, and a subsequent test phase.

Before starting with the experiment, participants filled in a multiple-choice questionnaire with eleven questions on important concepts and definitions from the field of probability theory. This questionnaire was used to measure prior knowledge. In the first part of HYPERCOMB, learners were given a short technical introduction to the system and to the experiment. After that, the basic notion of random experiments and the general rationale behind calculating the probability of outcomes were explained in a short introduction to the domain. In the subsequent example-based learning phase, learners had to acquire knowledge on four different problem categories, where each category was explained by means of two worked-out examples. Depending on the experimental condition, participants also received an imagery instruction or could retrieve static pictures or dynamic visualizations that augmented the worked-out examples. Participants were not forced to process these visualizations; rather, they could choose to select them by clicking on a button in order to view them. The visualizations depicted the contents of the worked-out examples in a concrete way (cf. Figure 1). For every worked-out example there was always one visualization of the problem statement and one of every worked-out solution-step. In accordance with the coherence principle [START_REF] Mayer | Multimedia learning[END_REF] the visualizations were kept as simple as possible and were not Making the abstract concrete 10 cluttered with irrelevant details. Participants could decide by themselves when to start working on the test problems. The instructional materials were no longer available during problem solving.

Before solving the test problems, learners had to give an estimate of the cognitive load they had been experiencing during learning (see 'dependent measures' for details). For the 11 test problems transfer distance with respect to the worked-out examples was varied by presenting five isomorphic and six novel problems. Isomorphic test problems differed from the instructional examples only with regard to their surface features. Novel test problems were constructed in a way that two complex-event probabilities had to be considered whose outcomes had to be multiplied in order to calculate the required probability. An example of a novel test problem would be: At a soccer stadium, there are two dressing rooms for the two opposing teams.

The 11 players from Oxford wear T-shirts with odd numbers from 1 to 21 and the 11 players from Manchester have even numbers from 2 to 22. Because the aisle from the dressing rooms is very narrow only one player at a time can enter the field. The players of the two teams leave their rooms alternately with a player from Oxford going first. What is the probability of the first five players entering the field having the numbers 5, 2, 13, 8, and 1 (i.e., the first has the number 5, the second has the number 2, and so on)?

Design and dependent measures. Participants studied in one of four instructional conditions. In the text-only condition only the written worked-out examples were available. In the imagery condition learners were additionally told to mentally imagine the contents as vividly and with as many details as possible. Mental imagery was trained at the beginning of the experiment by instructing learners to imagine the contents of a short text passage describing a traffic situation involving multiple cars approaching an intersection from different directions. Moreover, the participants were constantly reminded to use imagery in Making the abstract concrete 11 the example-based learning phase by a sticker "imagine the situation" attached to the frame of the computer screen. In the pictures condition, static visualizations could be retrieved for each component of the worked-out examples, whereas in the animation condition clicking the playbutton for any of the example components resulted in the presentation of a dynamic visualization. In the sprinter animation, for example, the sprinters were first shown entering the racetrack (problem statement) and were then running around the track. The visualization of the solution-steps always depicted one sprinter passing the finishing line and ascending the pedestal. The last frame of the animation was always used as the static picture (cf. Figure 1) depicting the problem state resulting from the application of the respective solution-step.

The dependent measures were problem-solving performance for isomorphic and novel test problems, time spent learning (learning time), and cognitive load. For each of the 11 test problems, 1 point was assigned for a correct answer; no partial credits were given. The sums across the five isomorphic problems and across the six transfer problems were each transformed into a percentage for ease of interpretation. Cognitive load was assessed by means of a modified version of the NASA-TLX [START_REF] Hart | Development of NASA-TLX (Task Load Index): Results of experimental and theoretical research[END_REF], which had been successfully used in order to distinguish between different aspects of cognitive load in prior studies [START_REF] Gerjets | Designing instructional examples to reduce intrinsic cognitive load: Molar versus modular presentation of solution procedures[END_REF]. This cognitive load measure consisted of three items, each rated on a scale from 0 (very low cognitive load) to 100 (very high cognitive load). The following subscales were used: 'task demands' (How much mental and physical activity was required, e.g., thinking, deciding, calculating, remembering, looking, searching etc.), 'effort' (How hard the participant had to work to understand the contents of the learning environment), and 'navigational demands' (How much effort the participant had to invest to navigate the learning environment). According to Cognitive Load Theory [START_REF] Sweller | Cognitive architecture and instructional design[END_REF] task demands are caused by inherent properties of the learning task (intrinsic cognitive load), higher-level processes for achieving a deeper understanding -as reflected in the availability of sophisticated and automated schemata (germane load), and activities not Making the abstract concrete 12 directed to learning such as decision processes required for navigation and information selection (extraneous load). Thus, a mapping is assumed between the theoretical assumptions of Cognitive Load Theory and the items of the modified version of the NASA-TLX.

The control measures were learners' intrinsic motivation prior to learning determined by a shortened version of the Intrinsic Motivation Inventory [START_REF] Ryan | Control and information in the intrapersonal sphere: An extension of cognitive evaluation theory[END_REF], domain-specific prior knowledge determined by a questionnaire included in the environment, and spatial abilities determined by the mental rotation test [START_REF] Vandenberg | Mental rotations, a group test of three-dimensional spatial visualization[END_REF].

Results

The statistical analysis is divided into three parts. First, all four instructional conditions were compared in an overall ANOVA in order to detect possible differences between them with regard to problem-solving performance, learning time, and cognitive load.

In a second step, a closer look was taken at the utilization of external visualizations (i.e., static pictures and animations) and the way this affects the aforementioned variables. Finally, the external-visualization conditions were compared to the imagery condition to answer the question of whether learners can generate helpful visualizations on their own by envisioning the contents of the worked-out examples. Within the last two steps of analysis the respective five conditions (i.e., frequent/sparse use of static pictures/animations, imagery) were also contrasted with the text-only condition to test whether using a specific type of external visualization either frequently or sparsely and whether instructing learners to imagine the worked-out examples was superior to presenting only text.

Overall analyses. Prior knowledge, intrinsic motivation, and spatial abilities were comparable across all four instructional conditions (prior knowledge: F < 1, intrinsic motivation: F < 1; spatial abilities: F(3, 119) = 1.65; MSE = 51.22; p > .10).

In a first step, problem-solving performance and learning time across all four conditions were analyzed by means of one-factor ANOVAs (Table 2). Performance on isomorphic problems varied slightly, though not significantly, as a function of instructional Making the abstract concrete 13 condition (F(3, 119) = 2.32; MSE = 769.06; p < .10), while performance on novel problems was left unaffected by the experimental manipulation (F < 1). Learning time increased rather naturally with the more instructional materials being available for processing (F(3, 119) = 6.21; MSE = 48052.96; p = .001). Furthermore, there were no differences between the four instructional conditions with regard to task demands, effort, and navigational demands experienced by learners (all Fs < 1). Thus, at first sight it seems that although presenting external visualizations increased the time learners devoted to learning, these increases were not accompanied by gains in performance. Contrarily, performance for isomorphic problems was worst in the animation condition. However, it is not clear whether the finding that problem-solving performance was only slightly affected by variations of the instructional materials is due to the ineffectiveness of these variations or whether it is due to the fact that learners did not sufficiently use the external visualizations. To address this question, the frequency by which learners retrieved these visualizations was analyzed in a second step.

Effects of utilizing external visualizations. For every participant it was possible to determine how often (s)he had retrieved visualizations. Though there were a total of 31 visualizations available in each of the conditions, these visualizations were seldom retrieved (see Figure 2). The figure displays the number of participants using a specific number of visualizations in each of the two conditions. For instance, nine participants in the animation condition and 11 participants in the static pictures condition used only one or two visualizations. On average, static pictures were viewed 7.0 times, animations 10.3 times.

Thus, less than one quarter of the available static pictures and only one third of the animations were used for learning.

For the further course of the statistical analyses a median split within each of the two conditions was conducted to distinguish between learners who sparsely used external visualizations and those who made frequent use of the representations. In the static pictures Making the abstract concrete 14 condition the median for visualization retrieval was Mdn = 3, with 18 participants being classified as sparse users and 12 as frequent users. In the animation condition the median was Mdn = 9 with 17 participants classified as sparse users and 15 as frequent users of visualizations. The resulting variable was used as a second factor in an ANOVA (instructional condition x visualization utilization).

Analyzing performance for isomorphic problems in the two conditions with external visualizations (Table 3) by means of this ANOVA yielded a significant main effect of instructional condition in favor of learning from static pictures (F(1, 58) = 5.56; MSE = 743.58; p < .05) as well as a marginally significant main effect for the utilization of visualizations (F(1, 58) = 3.08; MSE = 743.58; p < .10). Moreover, there was a significant interaction (F(1, 58) = 7.20; MSE = 743.58; p < .01) indicating that -when frequently usedstatic pictures were superior to animations (t(25) = 3.44; p < .01), while there were no differences when the visualizations were sparsely used (t(33) = -0.24; p > .80).

There were no significant main effects for novel problems (instructional condition: F < 1; information utilization: F(1, 58) = 1.30; MSE =474.28; p > .20) nor interactions (F(1, 58) = 1.12; MSE =474.28; p > .20). Learning times were longer for students in the animation condition than in the static pictures condition (F(1, 58) = 7.97; MSE = 44333.61; p < .01) and increased dramatically with frequent use of visualizations (F(1, 58) = 18.02; MSE = 44333.61; p < .001). Moreover, a significant interaction (F(1, 58) = 8.78; MSE = 44333.61; p < .01) revealed that frequently using static pictures compared to using them sparsely had no impact on learning time (t(33) = 0.11; p > .90), while frequent use of animations almost doubled the time spent with the instructional materials (t(25) = -3.95; p = .001).

Interestingly, learners did not experience any differences on cognitive load during learning. That is, the task demands, amount of effort, and navigational demands were judged as being equal across instructional conditions and irrespective of information utilization Making the abstract concrete 15 behavior (all Fs < 1 with one exception: instructional condition x visualization utilization for task demands: F(1, 58) = 1.05; MSE = 545.14; p > .30).

In a final step the problem-solving performance, learning times, and cognitive load of learners in the four visualization groups (Table 3) were compared to those of learners who had only text at their disposal (i.e., the baseline, Table 2) by means of multiple t-tests.

Frequently using static pictures was clearly associated with a better performance on isomorphic problems compared to learning from text only (t(39) = 2.27; p < .05), while there were no differences for novel problems (t(39) = -0.60; p > .50). At the same time, this frequent use did not require more learning time (t(39) = 1.23; p > .20) nor did it coincide with increases in cognitive load experienced by learners (task demands: t(39) = -0.58; p > .50; effort: t(39) = -0.22; p > .80; and navigational demands: t(39) = 1.23; p > .20).

Whenever static pictures were only sparsely used there were no effects on problemsolving performance (isomorphic problems: t(45) = -1.42; p > .15; novel problems: t(45) = -0.57; p > .50), learning time t(45) = 0.15; p > .80, or cognitive load (task demands: t(45) = 0.18; p > .80; effort: t(45) = 0.60; p > .50; and navigational demands: t(44) = -0.12; p > .90).

Frequently using animations was associated with a significant decrease in performance for isomorphic problems compared to learning from text only (t(42) = -2.11; p < .05). There were no differences for novel problems (t(42) = -1.09; p > .20). Not astonishingly, a frequent use of animations increased the time for learning (t(42) = 5.61; p < .001). No significant differences could be detected for the three cognitive load measures (task demands: t(42) = 0.47; p > .60; effort: t(42) = 0.90; p > .30; and navigational demands: t(42) = 1.06; p > .20). Sparsely using animations was not related to any changes in performance (isomorphic problems: t(44) = -1.13; p > .20; novel problems: (t(44) = 0.62; p > .50) or in learning time (t(44) = 0.03; p > .90) when compared to the text baseline. Similarly, there were no Making the abstract concrete 16 differences with regard to cognitive load (task demands: t(44) = -0.58; p > .50; effort: t(44) = -0.22; p > .80; and navigational demands: t(44) = 1.23; p > .20).

To sum up, using static pictures frequently was superior to learning from animations in that it was associated with a better performance for isomorphic problems while at the same time requiring less learning time. Based on the comparisons to the text baseline, static pictures proved to be a helpful augmentation of verbal materials. On the contrary, frequently using animations led to substantial increases in learning time while at the same time an inferior performance as compared to learning from text. Sparse users of external visualizations produced the same outcomes as learners who did not have visualizations at their disposal.

Therefore, whereas concrete visualizations of the contents of the worked-out examples supported learning, the dynamics contained in the animations were unnecessary or even harmful. If complex dynamic visualizations cannot be recommended for learning, the question arises whether there is a need for external visualizations at all. That is, if simple visualizations like static pictures of the content help to achieve an understanding of the principles, maybe learners are able to generate these images by themselves. The question whether external visualizations are more helpful than the instruction to imagine the contents of the worked-out examples is addressed in the remainder of this article.

The benefits of imagery compared to external visualizations. In this final analysis, the imagery condition was compared to the two external-visualization conditions by means of multiple t-tests whereby in the latter two sparse and frequent users of visualizations were distinguished.

The imagery condition had no impact on any of the performance measures when compared to the group of learners who had frequently used static pictures (isomorphic problems: t(42) = 1.42; p > .10; novel problems: t(42) = -0.78; p > .40). However, learning from static pictures took more time than imagining the contents of the examples (t(42) = 2.30; p < .05). There were no differences with regard to any of the cognitive load measures (task Making the abstract concrete 17 demands: t(42) = -1.30; p > .20; effort: t(42) = 0.41; p > .60; and navigational demands: t(42) = 0.34; p > .70).

Instructing learners to imagine the contents of the worked-out examples improved performance on isomorphic problems slightly, though not significantly, compared to a sparse use of static pictures (t(48) = -1.96; p < .10), while there were no differences for novel test problems (t(48) = -0.76; p > .40). There were no differences with regard to learning time between the two groups (t(48) = 0.88; p > .30) nor with regard to any of the cognitive load measures (task demands: t(48) = -0.87; p > .30; effort: t(48) = 0.18; p > .80; and navigational demands: t(48) = -0.48; p > .60).

Mental animation was thus more effective and efficient compared to a sparse use of static pictures in that it slightly improved performance without requiring more time.

Moreover, it was more efficient than frequent use of static pictures as it led to the same problem-solving performance with lower time demands. It can be expected that this pattern of results will be even more pronounced when comparing the mental animation condition to the groups of learners using external animations.

Learners in the mental animation condition clearly outperformed learners who had frequently used external animations with regard to isomorphic problems (t(45) = -2.51; p < .05), while the differences for novel problems were not pronounced (t(45) = -1.34; p > .10). Moreover, frequently using external animations led to prolonged learning times (t(45) = 7.32; p < .001). Again, none of these differences was reflected in learners' judgments of cognitive load (task demands: t(45) = -0.62; p > .50; effort: t(45) = 0.59; p > .50; and navigational demands: t(45) = 0.61; p > .50).

Finally, imagery was slightly superior to sparse use of dynamic visualizations (isomorphic problems: t(47) = -1.70; p < .10; novel problems: t(47) = 0.57; p > .50), while requiring the same amount of learning time (t(47) = 0.71; p > .40). There were no differences Making the abstract concrete 18 for the cognitive load measures (task demands: t(47) = -1.64; p > .10; effort: t(47) = -0.76; p > .40; and navigational demands: t(43) = 0.77; p > .40).

Thus, imagining the contents of the examples was more effective than learning from dynamic visualizations and more efficient in that it improved performance while requiring equal or even less amounts of learning time.

Finally comparing mental animation to the text-only condition yielded no significant differences with regard to performance (isomorphic problems: t(59) = -0.88; p > .30; novel problems: t(59) = -0.14; p > .80), learning time (t(59) = 0.79; p > .40), or cognitive load (task demands: t(59) = -1.21; p > .20; effort: t(59) = -0.57; p > .50; and navigational demands: t(59) = -0.44; p > .60).

Discussion

Evidence for the differential effectiveness of external and internal visualizations for cognitive skill acquisition has been presented here. The results support the assumption that learners might benefit from a concrete visualization of problem states tied to worked-out examples. Interestingly, generating these visualizations by oneself in the imagery condition produced learning outcomes similar to those in the "best" external visualization condition (i.e., frequent use of static pictures) and was more efficient than any of the external visualization conditions by requiring less learning time. Unfortunately, the effects of imagery were not strong enough to yield significant differences with the text-only condition. Before addressing possible reasons for the latter result, the differential effectiveness of external visualizations observed in this experiment are first discussed.

While it was demonstrated that a frequent use of static pictures was associated with performance improvements on isomorphic problems, frequently retrieving animations coincided with the opposite effect in that it was related to a significant decrease in problemsolving performance. The sparse use of external visualizations was no different to not having visualizations at all. Thus, similar to prior findings, it could be demonstrated that considering Making the abstract concrete 19 strategic variables (i.e., frequency of using different types of external visualizations) is important, as these variables may moderate the relationship between design variables and learning outcomes [START_REF] Gerjets | Goal configurations and processing strategies as moderators between instructional design and cognitive load: Evidence from hypertext-based instruction[END_REF].

The initial idea behind using animations had been that the dynamics of an animation might be helpful for illustrating information on changes in problem states that occur due to applying a specific solution-step. However, it seems that explicitly representing these changes was more harmful than helpful in that it may have distracted learners. These results are in line with prior findings [START_REF] Lowe | Extracting information from an animation during complex visual learning[END_REF][START_REF] Pane | Assessing dynamics in computer-based instruction[END_REF].

Interestingly, in contrast to results of prior experiments [START_REF] Gerjets | Designing instructional examples to reduce intrinsic cognitive load: Molar versus modular presentation of solution procedures[END_REF], no differences in cognitive load as measured with the modified version of the NASA-TLX could be detected. Rather than doubting its sensitivity in the current context, two factors which have effects in opposite directions may have contributed to this result. On the one hand, based on the findings by [START_REF] Lowe | Extracting information from an animation during complex visual learning[END_REF] or [START_REF] Pane | Assessing dynamics in computer-based instruction[END_REF] one might assume that dynamic visualizations should be associated with higher navigational demands and effort ratings. Salomon's findings (1983Salomon's findings ( , 1984)), on the other hand, suggest that the amount of invested mental effort is influenced by the learners' perception of the medium and the demands it imposes. Viewing an animation -similar to watching TV as in the Salomon study (1984) -is often perceived as being "easy" and thus learners may decide to refrain from a deeper processing of the presented information. Accordingly, from a prescriptive point of view learners would need to invest more resources into learning from animations; however, from a descriptive point of view they might not experience this need and may refrain from performing effortful cognitive processes required for a deeper understanding [START_REF] Schnotz | Individual and co-operative learning with interactive animated pictures[END_REF]. Accordingly, it does not necessarily have to be the case that learners report higher cognitive load values for the animation condition. This research, thus, pursues the idea of further reducing the cognitive demands imposed by animations, while at the same time hindering that learners passively watch the visualizations. A reduction of cognitive load is aimed at by using abstract rather than concrete dynamic visualizations of the worked-out examples. These abstract animations are characterized by the fact that the visualizations of all examples share a common representation of objects and of the relevant relations among them. That is, irrespective of an example's cover story, objects such as the sprinters are represented by marbles selected from an urn. This common representation across examples should help learners to focus on the structural similarities and differences between the examples while being able to ignore their surface features. Thus, there is not only less information that needs to be processed in total, the ratio between mathematically relevant and irrelevant information is improved compared to the concrete animations investigated in the current paper. Thus, this simplified representation should be less demanding and should leave cognitive resources free in order to cope with the dynamics of the visualization.

At the same time, instructional methods which reduce the danger of a superficial processing of the presented animations and which force learners to engage in a deeper processing of the instructional materials and invest more mental effort in the learning task will be investigated. Until now, the animations have been non-interactive; that is, learners had no opportunity to stop or replay the visualizations, which may enhance the impression of an ongoing movie whose presentation is predetermined. Accordingly, it is planned to implement interactive animations in a subsequent experiment. Interactivity seems to be a value in its own right and produces positive results rather consistently [START_REF] Betrancourt | Effect of computer animation on users' performance: A review[END_REF][START_REF] Ferguson | Learning with real machines or diagrams: Application of knowledge to real-world problems[END_REF][START_REF] Gonzales | Does animation in user interfaces improve decision making[END_REF][START_REF] Mayer | When learning is just a click away: Does simple user interaction foster deeper understanding of multimedia messages[END_REF][START_REF] Tversky | Animation: Can it facilitate?[END_REF]. In particular, it affords learners in thinking about whether they have grasped the information presented or whether they should replay the animation.

Additionally, the work with regard to cognitive skill acquisition from static pictures and imagery will be continued. First, research will be carried out on ways of improving the learners' use of static pictures. In particular, the effectiveness of retrieval prompts which Making the abstract concrete 21 guide learners to use static pictures more often will be tested. Prior studies demonstrated that prompting learners to retrieve profitable information units is an effective means to foster problem-solving performance, especially for students with low prior knowledge (Gerjets, Scheiter, & Schuh, in press). Second, once static pictures have been retrieved, their processing should be supported by additional instructional guidance. In particular, learners may receive instructions to compare multiple static pictures to enable them to more easily discover changes in problem states that have occurred due to applying a solution-step.

Finally, the use of imagery might be further improved. In the current experiment, learners were instructed to use imagery only at the beginning of the experiment and the reminder they received during the learning phase could be easily overlooked. This may also explain why giving an imagery instruction neither extended learning times nor increased effort ratings compared to the text-only condition, which one might expect if learners are asked to envision an example's content in addition to merely reading the text. The argument is similar to the one made with regard to the observed lack of differences for cognitive load ratings between the animation condition and the other instructional conditions. That is, if learners in the imagery condition had invested the amount of mental effort required to benefit from imagining worked-out examples, then this should have expressed itself in longer learning times and higher cognitive load ratings. The fact that these differences with the textonly condition were not observed indicates that learners did not engage in the imagination task to a sufficient degree. In order to circumvent this problem, computer-based prompts that frequently remind learners to envision the examples' contents are to be investigated in future experiments. At the Olympics 7 sprinters participate in the 100m-sprint. What is the probability of correctly guessing the winner of the gold, the silver, and the bronze medals?

In order to find the first event probability you have to consider the number of acceptable choices and the pool of possible choices. The number of acceptable choices is 1 because only 1 sprinter can win the gold medal. The pool of possible choices is 7 because 7 sprinters participate in the 100m-sprint. Thus, the probability of correctly guessing the winner of the gold medal is 1/7.

In order to find the second event probability you again have to consider the number of acceptable choices.

The number of acceptable choices is still 1 because only 1 sprinter can win the silver medal. The pool of possible choices is reduced to 6 because only the remaining 6 sprinters participating in the sprint are eligible to receive the medal. Thus, the probability of correctly guessing the winner of the silver medal is 1/6.

1/6

Find 2nd event probability:

Find 1st event probability:

Figure 1 .Figure 2 :

 12 Figure 1. Screenshot of a worked-out example with external visualization

Figure 2 Note:

 2 Figure 2

Table 2

 2 Means and Standard Deviations for Learning Time (in Sec), Problem-Solving Performance (in % correct) and Cognitive Load as a Function of Instructional Condition

			Instructional condition
		Text	Static pictures	Animation	Imagery
	Performance:	49.7	51.3	38.1	55.6
	Isomorphic problems	(23.1)	(33.5)	(24.0)	(29.1)
	Performance:	19.5	15.6	18.8	20.3
	Novel problems	(23.2)	(17.5)	(25.3)	(20.6)
	Learning time	493.6	530.8	678.3	457.4
		(204.9)	(184.5)	(302.3)	(152.2)
	Task demands	34.8	34.2	34.1	42.0
		(23.6)	(25.5)	(20.8)	(22.8)
	Effort	52.1	57.0	54.4	55.3
		(23.6)	(19.7)	(24.6)	(19.0)
	Navigational demands	16.6	18.3	24.2	19.1
		(19.6)	(24.0)	(24.0)	(24.2)

Table 3

 3 Means and Standard Deviations for Learning Time (in Sec), Problem-Solving Performance (in % correct) and Cognitive Load as a Function of Visualization Utilization in the Two

	External Visualization Conditions
			Instructional condition
		Static pictures	Animation
	Visualization	Sparse use	Frequent use	Sparse use	Frequent use
	utilization
	Performance:	38.9	70.0	41.2	34.7
	Isomorphic problems	(28.7)	(32.5)	(26.9)	(20.7)
	Performance:	15.7	15.3	24.5	12.2
	Novel problems	(20.2)	(13.2)	(30.7)	(16.0)
	Learning time	503.1	572.4	495.4	885.6
		(211.0)	(133.3)	(219.2)	(244.5)
	Task demands	36.1	31.3	30.6	38.0
		(23.4)	(29.2)	(24.1)	(16.2)
	Effort	56.4	57.9	50.3	59.0
		(21.1)	(18.3)	(26.8)	(21.8)
	Navigational demands	15.8	22.1	24.7	23.7
		(19.6)	(30.1)	(24.9)	(23.8)

Author Note

This research was supported by the Alexander von Humboldt-Foundation (TransCoop-Program). We thank the participants of the practical course in experimental psychology (summer semester 2003) for conducting the experiment.

Advantages (+) and Drawbacks (-) of Visualization Methods

Static pictures


Katharina Scheiter

Peter Gerjets

Richard Catrambone

Using visualizations to

published or not. The documents may come

INTRODUCTION

Understanding mathematical solution procedures is a necessary prerequisite in order to be able not to solve only familiar problems, but also to work on novel problems requiring an adaptation of known solution procedures. However, it has been noted that students often face severe difficulties in understanding solution procedures even when they have received elaborated instructional explanations of the individual solution steps. This may result from the fact that the solution steps are often conveyed in a rather abstract way so that learners experience difficulties in imagining which changes in the problem state are achieved by applying a specific solution step to a problem.

The use of multimedia learning environments may offer ways to overcome these difficulties that can hardly be achieved by traditional instructional means [START_REF] Mayer | Multimedia learning[END_REF]. In multimedia learning environments information presentation can be accomplished by using different representational formats (textual and pictorial) which may be processed in different sensory channels (auditory and visual). Additionally, information presentation is not restricted to static displays (e.g., diagrams, pictures, written text), but the representations used can involve changes over time (e.g., dynamic visualizations, spoken text).

In the current paper we are interested in the effects of augmenting a purely textbased hypertext environment called H Y P E R C O M B by different kinds of visualizations. HYPERCOMB teaches how to calculate the probability of complex Scheiter, K., [START_REF] Gerjets | Designing instructional examples to reduce intrinsic cognitive load: Molar versus modular presentation of solution procedures[END_REF]. Using visualizations to teach problem-solving skills in mathematics: Which kind of visualization works? In P. Gerjets, P. A. Kirschner, J. Elen & R. Joiner (Eds.), Instructional design for effective and enjoyable computer-supported learning. Proceedings of the first joint meeting of the EARLI SIGs "Instructional Design" and "Learning and Instruction with Computers" [CD-ROM] (pp. [256][257][258][259][260][261][262][263][264][265][266][267][268]. Tübingen: Knowledge Media Research Center.

At the Olympics 7 sprinters participate in the 100m-sprint. What is the probability of correctly guessing the winner of the gold, the silver, and the bronze medals?

In order to find the first event probability you have to consider the number of acceptable choices and the pool of possible choices. The number of acceptable choices is 1 because only 1 sprinter can win the gold medal. The pool of possible choices is 7 because 7 sprinters participate in the 100m-sprint. Thus, the probability of correctly guessing the winner of the gold medal is 1/7.

In order to find the second event probability you again have to consider the number of acceptable choices. The number of acceptable choices is still 1 because only 1 sprinter can win the silver medal. The pool of possible choices is reduced to 6 because only the remaining 6 sprinters participating in the sprint are eligible to receive the medal. Thus, the probability of correctly guessing the winner of the silver medal is 1/6.

1/6

Find 2nd event probability: Find 2nd event probability: events. Problems of calculating complex event probabilities are related to situations where the probability of selecting a particular configuration of elements randomly out of a set of elements has to be determined (cf. Figure 1). [START_REF] Gerjets | Designing instructional examples to reduce intrinsic cognitive load: Molar versus modular presentation of solution procedures[END_REF] there is still space left for improvements. We assumed that this space might be claimed by the benefits achieved through the use of visualizations.

According to the multimedia principle [START_REF] Mayer | Multimedia learning[END_REF] embellishing textual learning materials by static pictures or dynamic visualizations (i.e., animations) helps to promote learners' understanding of instructions. With regard to the acquisition of problem-solving knowledge visualizations of worked-out examples may first help learners to understand the situation described in the problem statement (i.e., the initial problem state) and thus to correctly represent its meaning in a situation model [START_REF] Nathan | A theory of algebra-word-problem comprehension and its implications for the design of learning environments[END_REF]. Second, visualizations of the solution steps may promote an understanding of changes with regard to the initial problem state that are achieved by applying a solution step to a problem. Visualizing worked-out examples can be done by presenting either static pictures, animations, or an instruction to mentally imagine the examples' contents.

Static Pictures. Static pictorial representations are known to foster the immediate and delayed retention of facts contained in the accompanying text (cf. for a review [START_REF] Levin | Validating functions of pictures in prose[END_REF]. Moreover, with regard to the acquisition of problem-solving knowledge in domains like mathematics and physics the added value of abstract diagrammatic and graphical representations has been acknowledged (e.g., [START_REF] Shah | Review of graph comprehension research: Implications for instruction[END_REF]. These types of visualizations are said to be computationally effective in that they facilitate specific inferential processes needed for some learning tasks [START_REF] Larkin | Why a diagram is (sometimes) worth ten thousand words[END_REF]. However, there is no research to our knowing on whether concrete pictures that depict a problem statement and its associated solution procedure promote the acquisition of problemsolving knowledge. Representing the problem statement in a picture might help to understand which object features and interrelations are relevant to the solution of the problem. For instance, in the sprinter example seven sprinters on the racetrack are depicted out of which three can win the gold, silver, and the bronze medalrepresented by the pedestal. Additionally visualizing the solution steps may support learners in inferring which change is achieved by applying a solution procedure when they compare the new problem state to the previous one. For instance, comparing the picture illustrating the second solution step to the one of the first step helps to clarify the fact that only six sprinters are eligible for guessing the winner of the silver medal, because one sprinter has already been assigned to the gold medal in the first solution step and thus already stands on the pedestal.

Animations. An animation is "any application which generates a series of frames, so that each frame appears as an alteration of the previous one, and where the sequence of frames is determined either by the designer or the user" (Betrancourt & Tversky, 2000, p. 313). Thus, animations do not only depict the current status of objects; rather they additionally deliver information concerning changes of objects and of their position over time (motion) as well as information concerning the direction of these changes (trajectory, [START_REF] Rieber | Animation in computer-based instruction[END_REF]. Several findings suggest that animations can be used successfully for delivering abstract contents like mathematical rules, Newton's laws, or computer algorithms [START_REF] Baek | Color, graphics, and animation in a computer-assisted learning tutorial lesson[END_REF][START_REF] Byrne | Evaluating animations as student aids in learning computer algorithms[END_REF][START_REF] Catrambone | Using animations to help students learn computer algorithms[END_REF][START_REF] Rieber | Animation in computer-based instruction[END_REF]. With respect to conveying problem-solving knowledge the visual-spatial properties of the visualization may be used to deliver information on the current problem state and its relevant structural features. Moreover, the changes over time that can be depicted in an animation may be used to reflect the changes in problem states that result from applying a solution step to a specific problem state of the example -without the need to compare multiple representations as it is necessary when learning from static pictures.

However, learning from animations is known to impose certain requirements onto the learners that they may have difficulty to meet [START_REF] Betrancourt | Effect of computer animation on users' performance: A review[END_REF]. [START_REF] Pane | Assessing dynamics in computer-based instruction[END_REF] have demonstrated that learners often fail to use animations to a sufficient extent and thus miss important information. The results of [START_REF] Lowe | Extracting information from an animation during complex visual learning[END_REF] additionally suggest that learners have trouble focusing on the most relevant parts of an animation and are often distracted by salient, but irrelevant details. Furthermore, due to the dynamic changes of the display the information that has to be remembered can only be viewed for a limited amount of time and may therefore have vanished before learners have identified it. Finally, dynamic visualizations may lead to an overly passive information processing and prevent learners from performing effortful cognitive processes required for a deeper understanding [START_REF] Palmiter | Animated demonstrations for learning procedural computer-based tasks[END_REF]. In line with Salomon's principle of least effort [START_REF] Salomon | Television is "easy" and print is "tough": The differential investment of mental effort in learning as a function of perceptions and attributions[END_REF] learners may refrain from deeply processing the contents of an animation and passively watch it as an ongoing movie.

Imagery. To circumvent the problem of passive information processing we implemented a third visualization condition in our experiment in which we instructed learners to imagine the contents of a text-based worked-out example. That is, learners did not receive any pictorial representations at all; rather, they were told to construct their own visualizations. [START_REF] Hodes | The effectiveness of mental imagery and visual illustrations: A comparison of two instructional variables[END_REF] compared the effectiveness of imagery instructions and instructional visuals for fact recall and understanding. Both instructional methods were helpful in inducing an imagery strategy and in improving posttest performance; however, for some performance measures achievements due to presenting external visuals were larger than for the imagery instructions. [START_REF] Ginns | When imagining information is effective[END_REF] showed that imagining (vs. studying text-based materials) was helpful only for learners who possessed sufficiently high prior knowledge in the domain. The authors explained this finding by assuming that "to successfully imagine a procedure or a concept, all of the relevant elements must be processed simultaneously in working memory. That may be possible only after a schema had been constructed" (p. 231).

Comparing the three different visualization methods (static pictures, animations, and imagery) to each other yields insights on specific promises as well as on drawbacks associated with these methods. The trade-off between these promises and drawbacks may determine learning outcomes (Table 1). Learning from static pictures may engage learners in a more active way, because they have to compare multiple visualizations in order to understand the to-be-learned solution procedures. As this information is permanently visible these comparisons may be conducted without overloading the cognitive system. However, learners may make wrong inferences and they may miss important information so that their internal representation of the solution procedures may be incorrect as well as incomplete. Moreover, because the external representations cannot be modified by a learner they are not adapted to his or her preferences or prior knowledge level. Dynamic pictures may on the other hand reduce learner activity and induce a rather passive style of processing. The cognitive demands imposed by the need to extract the relevant information from a changing display and to memorize this information may be rather high. Unless animations are highly interactive they do not allow for any adaptation to a learner's preferences or prior knowledge. However, an advantage of dynamic visualizations of solution steps is that all the information that is needed to understand problem states and their changes is in principle contained in the representation and thus it is correct as well as complete. Finally, imagery instructions should foster learner activity in an optimal way; however, the need to envision all information may at the same time require rather demanding processes [START_REF] Ginns | When imagining information is effective[END_REF]. There is also the danger that learners may miss important information or that they make incorrect inferences. A possible advantage of mental imagery is that because it is based on self-generated images, these images are adapted to a learner's preferences and prior knowledge level. These trade-offs between possible promises and drawbacks were investigated in the experiment outlined in the remainder of the paper.

EXPERIMENT

Method

Participants. Participants were 124 students (88 female, 36 male) of the University of Tuebingen, Germany, who participated for either course credit or payment. Average age was 23.7 years. Most of the subjects were familiar with the domain of probability theory and were experienced computer users.

Materials and Procedure. HYPERCOMB consisted of a technical instruction, a short introduction to the domain, an example-based learning phase and a subsequent test phase. Before starting with the experiment participants filled in a multiple-choice questionnaire with eleven questions on important concepts and definitions from the field of probability theory. This questionnaire was used to measure participants' prior knowledge. In the first part of HYPERCOMB learners were given a short technical introduction to the system and to the experiment. Consequently, the basic notion of random experiments and the general rationale behind calculating the probability of outcomes were explained in a short introduction to the domain. In the subsequent example-based learning phase learners had to acquire knowledge on four different problem categories, whereby each category was explained by means of two worked-out examples. Depending on the experimental condition subjects additionally received an imagery instruction or they could retrieve static pictures or dynamic visualizations that augmented the worked-out examples. Subjects were not forced to process these visualizations; rather, they had to select them by clicking on a button in order to view them. The visualizations depicted the contents of the worked-out examples in a concrete way (cf. Figure 1). For every worked-out example there was always one visualization of the problem statement and one of every worked-out solution step. Following the coherence principle [START_REF] Mayer | Multimedia learning[END_REF] the visualizations were kept as simple as possible and were not cluttered with any irrelevant details. Participants could decide by themselves when to start working on the test problems. The instructional materials were no longer available during problem solving. For the eleven test problems we varied the transfer distance with respect to the worked-out examples by presenting isomorphic as well as novel problems. Isomorphic test problems differed from the instructional examples only with regard to their surface features. Novel test problems were constructed in a way that two complex-event probabilities had to be considered whose outcomes had to be multiplied in order to calculate the required probability.

Design and Dependent Measures. Subjects learned in one of four instructional conditions. In the text-only condition only the written worked-out examples were available. In the imagery condition learners were additionally told to mentally imagine the contents as vividly and with as many details as possible. Mental imagery was trained at the beginning of the experiment by instructing learners to imagine the contents of a short text passage describing a traffic situation involving multiple cars which approach a crossing from different directions. Moreover, they were constantly reminded to use imagery in the example-based learning phase by a sticker "imagine the situation" attached to the computer screen. In the pictures condition static visualizations could be retrieved for each component of the workedout examples, whereas in the animation condition clicking the play-button for any of the example components resulted in the presentation of a dynamic visualization. For instance, in the sprinter animation the sprinters were first entering the racetrack (problem statement) and were then running across the racetrack. The visualization of the solution steps always depicted one sprinter passing the finishing line and ascending the pedestal. As static pictures we always used the last frame of the animations (cf. Figure 1) that depicted the problem state resulting from the application of the respective solution step.

As dependent measures we registered problem-solving performance for isomorphic and novel test problems and the time spent on learning (learning time).

Results

Overall Analyses. Prior knowledge was comparable across all four instructional conditions. In a first step we analyzed problem-solving performance and learning time across all four conditions by means of one-factorial ANOVAs. Performance on isomorphic problems (Figure 2, left) varied slightly as a function of instructional condition (F(3,120) = 2.19; Mse = 783.90; p < .10), while performance on novel problems (Figure 2, right) was left unaffected by the experimental manipulation (F < 1). Learning time (Figure 3) increased rather naturally with the more instructional materials being available for processing (F(3,120) = 5.36; Mse = 55915.01; p < .01). Thus, at first sight it seems that although presenting external visualizations increased the time learners devoted for learning, these increases in learning time were not accompanied by respective gains in performance. On the contrary, performance for isomorphic problems was worst in the animation condition. However, it is not clear whether the finding that problem-solving performance was only slightly affected by variations of the instructional materials is due to the ineffectiveness of these variations or whether it is due to the fact that learners did not sufficiently use the external visualizations. To address this question we analyzed the frequency by which learners retrieved these visualizations in a next step. Utilization of External Visualizations. For every subject we determined how often he or she had retrieved visualizations by clicking on the respective button in the two visualization conditions. There were a total of 39 visualizations available in each of the conditions; however, these were only seldom retrieved (Figure 4). Static pictures were retrieved 6.6 times on average, whereas animations were played 10.9 times. This difference between the two conditions was marginally significant in a twotailed t-test (t(57) = 1.71; p < .10). For the further course of the statistical analyses we conducted a median split within each of the two conditions to distinguish between learners who used external visualizations only sparsely and those who made frequent use of the representations. The resulting variable was used as a second factor in an ANOVA (instructional condition x visualization utilization). Analyzing performance in the conditions with external visualizations by means of this ANOVA revealed no significant effects for performance on novel problems (all Fs < 1). However, for performance on isomorphic test problems (Figure 5) there were marginally significant main effects of external visualization and visualization utilization. Learners showed a slightly superior performance when learning from static pictures (F(1,55) = 3.66; Mse = 753.00; p < .10). Furthermore, learners who made use of the visualizations more frequently outperformed those who only used them sparsely (F(1,55) = 3.59; Mse = 753.00; p < .10). Finally, there was a significant interaction (F(1,55) = 5.12; Mse = 753.00; p < .05) indicating thatwhen frequently used -static pictures were superior to animations (t(25) = 2.64; p < .05), while there were no differences when the visualizations were retrieved only sparsely (t(30) = -0.28; p > .70). Figure 5 additionally displays the performance in the text-only condition as a baseline. Contrasting the various conditions with this baseline indicated that only the frequent use of static pictures resulted in performance improvements that were significant at the .05 level. All the other conditions achieved a performance that lay below the baseline performance that learners were able to achieve when studying the text-based worked-out examples only. Furthermore, as an analysis of the learning time (Figure 6) revealed this deterioration in performance due to frequently using animations was accompanied by an increase in time learners needed to study the instructional materials. The type of external visualization had no impact on learning time (F < 1), whereas learning time certainly increased with a more intensive use of external visualizations (F (1,55) = 15.77; Mse = 62160.57; p < .001). As indicated by a marginally significant interaction these increases in learning time due to a frequent retrieval of external visualizations were especially apparent in the animation condition (F (1,55) = 2.89; Mse = 62160.57; p < .10). That is, in the pictures condition learners who frequently used visualizations did not differ from those using them only sparsely with regard to learning time (t(27) = 1.58; p > .10), whereas using animations frequently lead to highly significant increases in learning time (t(28) = 4.26; p < .001). To sum up, using static pictures frequently did not increase learning time demands and nevertheless improved performance. On the contrary, frequently using animations was rather damaging in that it required more learning time and did not yield any performance improvements. Therefore, while concrete visualizations of the contents of the worked-out examples proved beneficial for learning, the dynamics contained in the animations were unnecessary or even harmful. If complex dynamic visualizations cannot be recommended for learning, the question arises whether there is a need for external visualizations at all. That is, if already simple visualizations like static pictures of the content help to achieve an understanding of the principles, then maybe learners are able to generate these images by themselves. The question whether external visualizations are more helpful than the instruction to imagine the contents of the worked-out examples was addressed in the last analysis.

The Benefits of Imagery Compared to External Visualizations. In this final section we compared the imagery condition to the two external-visualization conditions whereby in the latter two we further distinguished between sparse and frequent use of visualizations. Because none of the analyses revealed any effects for performance on novel problems we will report the results for isomorphic problems only. There were no differences with regard to performance on isomorphic problems between the imagery condition and the two subgroups who learned from static pictures (imagery vs. sparse picture use: t(45) = 1.42; p > .15; imagery vs. frequent use of pictures: t(41) = -1.27; p > .20). Using animations frequently was worse than receiving an imagery instruction only (t(46) = 1.97; p < .10), while there were smaller differences between the sparsely use of animations and the latter condition (t(44) = 1.64; p > .10). With regard to learning time, the frequent use of external visualizations increased the time spent studying the instructional materials compared to the imagery instruction (imagery vs. frequent picture use: (t(41) = -2.69; p = .01; imagery vs. frequent animation use: t(46) = 6.82; p < .001), whereas the sparse use did not yield any longer learning times (imagery vs. sparse picture use: t(45) = 0.15; p < .80; imagery vs. sparse animation use: t(44) = 0.52; p > .60). Thus, imagining the contents of the worked-out examples was as effective as viewing static pictures that depicted these contents and more efficient because it required less time for learning. Moreover, mental imagery was slightly more effective than receiving dynamic visualizations and was again accompanied by less learning time.

CONCLUSIONS

In this paper we presented evidence for the differential effectiveness of external and internal visualizations for cognitive skill acquisition. Our results support the assumption that learners might benefit from a concrete visualization of problem states that is tied to the cover stories of worked-out examples. However, while we were able to show that a frequent use of static pictures fostered performance at least on isomorphic problems, including dynamics in the external visualizations worsened performance. The initial idea of using animations had been that the dynamics of an animation might be used to depict information on changes in problem states that occur due to applying a specific solution step. However, it seems that representing these changes explicitly was more harming than helpful in that it may even have distracted learners. These results are in line with prior findings [START_REF] Lowe | Extracting information from an animation during complex visual learning[END_REF][START_REF] Pane | Assessing dynamics in computer-based instruction[END_REF] showing that learners are often overwhelmed by the number of details they need to identify, select, and to remember in a limited period of time (i.e., while the information is present on the changing display) when learning from animations.

In our current research we thus pursue the idea of further reducing the cognitive demands imposed by animations by using abstract rather than concrete dynamic visualizations of the worked-out examples. These abstract animations are characterized by the fact that the visualizations of all examples share a common representation of objects and of the relevant relations among them. That is, irrespective of an example's cover story objects like the sprinters are represented by marbles which are selected from an urn. This common representation across examples should help learners to focus on the structural similarities and differences between the examples while being able to ignore their surface features. Thus, there is not only less information that needs to be processed in total; moreover, the ratio between relevant and irrelevant information is improved compared to the concrete animations investigated in the current paper. Thus, this simplified representation should be less demanding and should leave free cognitive resources in order to cope with the dynamics of the animation.

Additionally, we want to continue our work based on the promising results with regard to cognitive skill acquisition from static pictures and imagery. First, we would like to investigate means of improving learners' use of static pictures. In particular, we aim at testing the effectiveness of retrieval prompts which guide learners to use static pictures more often. In prior studies we were able to demonstrate that prompting learners to retrieve profitable information units is an effective means to foster problem-solving performance in particular for students with low prior knowledge [START_REF] Gerjets | Instruktionale Unterstützung beim Fertigkeitserwerb aus Beispielen in hypertextbasierten Lernumgebungen[END_REF]. Second, once static pictures have been retrieved to a sufficient extent, their processing should be supported by additional instructional guidance. In particular, learners may receive instructions to compare multiple static pictures to enable them to extract changes in problem states that have occurred due to applying a solution step more easily. Finally, we are also convinced that the use of imagery might be further improved. In the current experiment learners were instructed to use imagery only at the beginning of the experiment and the reminder during the learning phase could be easily overlooked. Thus, we would like to investigate the use of computer-based prompts that frequently remind learners to envision the examples' contents. All three instructional devices (retrieval prompts, comparison instruction, and imagery prompts) aim at increasing the time learners devote for studying the instructional materials as well as at improving the quality of processing the worked-out examples.

AFFILIATIONS

Katharina

Figure 1 :

 1 Figure 1: Screenshot of a worked-out example with external visualization The main instructional principle underlying HYPERCOMB is the use of worked-out examples for conveying knowledge on different problem categories. Research over the last 20 years has shown that worked-out examples are of great help for knowledge acquisition in particular in well-structured domains like mathematics, physics, or programming (Atkinson, Derry, Renkl, & Wortham, 2000). However, although we have identified a way of designing worked-out examples that boosts performance compared to conventionally designed examples[START_REF] Gerjets | Designing instructional examples to reduce intrinsic cognitive load: Molar versus modular presentation of solution procedures[END_REF] there is still space left for improvements. We assumed that this space might be claimed by the benefits achieved through the use of visualizations.According to the multimedia principle[START_REF] Mayer | Multimedia learning[END_REF] embellishing textual learning materials by static pictures or dynamic visualizations (i.e., animations) helps to promote learners' understanding of instructions. With regard to the

Figure 2 :Figure 3 :

 23 Figure2: Problem-solving performance (in % correct) for isomorphic(left) and novel problems (right) as a function of instructional condition

  USING VISUALIZATIONS TO TEACH PROBLEM-SOLVING SKILLS IN MATHEMATICS 8

Figure 4 :

 4 Figure 4: Frequency of using external visualizations in the static and in the dynamic visualization condition

Figure 5 :

 5 Figure 5: Performance on isomorphic problems (in % correct) as a function of external visualization and visualization utilization

Figure 6 :

 6 Figure 6: Learning time (in sec) as a function of external visualization and visualization utilization

  TEACH PROBLEM-SOLVING SKILLS IN MATHEMATICS 10

Table 1 :

 1 Promises (+) and drawbacks (-) of visualization methods

		Static pictures Dynamic pictures	Imagery
	Learner activity	+	-	+
	Cognitive load	+	-	-
	Correctness / completeness	-	+	-
	Adaptation to learner preferences / knowledge	-	-	+