Domingo Martín

Germán Arroyo

M Victoria Luzón

Tobias Isenberg

Scale-Dependent and Example-Based Grayscale Stippling

Keywords: Stippling, high-quality rendering, scale-dependent NPR, example-based techniques, illustrative visualization

We present an example-based approach to synthesizing stipple illustrations for static 2D images that produces scaledependent results appropriate for an intended spatial output size and resolution. We show how treating stippling as a grayscale process allows us to both produce on-screen output and to achieve stipple merging at medium tonal ranges. At the same time we can also produce images with high spatial and low color resolution for print reproduction. In addition, we discuss how to incorporate high-level illustration considerations into the stippling process based on discussions with and observations of a stipple artist. Also, certain features such as edges can be extracted and used to control the placement of dots to improve the result. The implementation of the technique is based on a fast method for distributing dots using halftoning and can be used to create stipple images interactively. We describe both a GPU implementation of the basic algorithm that creates stipple images in real-time for large images and an extended CPU method that allows a finer control of the output at interactive rates.

Introduction

Stippling is a traditional pen-and-ink technique that is popular for creating illustrations in many domains. It relies on placing stipples (small dots that are created with a pen) on a medium such as paper such that the dots represent shading, material, and structure of the depicted objects. Stippling is frequently used by professional illustrators for creating illustrations, for example, in archeology (e. g., see Figure 2), entomology, ornithology, and botany.

One of the essential advantages of stippling that it shares with other pen-and-ink techniques is that it can be used in the common bilevel printing process. By treating the stipple dots as completely black marks on a white background they can easily be reproduced without losing spatial precision due to halftoning artifacts. This property and the simplicity of the dot as the main rendering element has lead to numerous approaches to synthesize stippling within Non-Photorealistic Rendering (NPR), describing techniques for distributing stipple dots such that they represent a given tone. Unfortunately, computergenerated stippling sometimes creates distributions with artifacts such as unwanted lines, needs a lot of computation power due to the involved computational complexity of the approaches, cannot re-create the merging of stipple dots in middle tonal ranges that characterizes many handdrawn examples, or produces output with dense stipple points unlike those of hand-drawn illustrations.

To address these issues, our goal is to realize a stippling process for 2D images (see example result in Figure 1) that is easy to implement, that considers the whole process of hand-made stippling, and that takes scale and output devices into account. For this purpose we do not consider stippling to always be a black-and-white technique, in contrast to previous NPR approaches (e. g., [START_REF] Kim | Stippling By Example[END_REF]). In fact, we use the grayscale properties of hand-made stipple illustrations to inform the design of a grayscale stipple process. This different approach lets us solve not only the stipple merging problem but also lets us create output adapted to the intended output device. To summarize, our paper makes the following contributions:

• an analysis of high-level processes involved in stippling and a discussion on how to support these using image-processing,

• a method for example-based stippling in which the stipple dot placement is based on halftoning,

• the scale-dependent treatment of scanned stipple dot examples, desired output, and stipple placement,

• a grayscale stippling process that can faithfully reproduce the merging of stipples at middle tonal ranges,

• the enabling of both print output in black-and-white and on-screen display in gray or tonal scales at the appropriate spatial and color resolutions,

• an optional special treatment of image edges that improves the control of the placement of stipples,

• a technique that is easy to implement and permits the interactive creation of stipple images,

• a description of a GPU implementation of the basic technique, and

• a comparative analysis of the stipple dot placement statistics for both the hand-drawn example and generated synthetic stipple images.

This article is an extended version of a paper [START_REF] Martín | Example-Based Stippling using a Scale-Dependent Grayscale Process[END_REF] published at NPAR 2010. The remainder is structured as follows. First we review related work in the context of computer-generated stippling in Section 2. Next, we analyze hand-made stippling in Section 3, both with respect to high-level processes performed by the illustrator and low-level properties of the stipple dots. Based on this analysis we describe our scale-dependent grayscale stippling process in Section 4. We analyze the performance and describe a GPU implementation in Section 5, before discussing the results in Section 6. We conclude the paper and suggest some ideas for future work in Section 7.

Related Work

Pen-and-ink rendering and, specifically, computer-generated stippling are well-examined areas within NPR. Many approaches exist to both replicating the appearance of hand-drawn stipple illustrations and using stippling within new contexts such as animation. In the following discussion we distinguish between stipple rendering based on 3D models such as boundary representations or volumetric data on the one side and stippling that uses 2D pixel images as input on the other.

The variety of types of 3D models used in computer graphics is also reflected in the diversity of stipple rendering approaches designed for them. There exist techniques for stippling of volume data [START_REF] Lu | Non-Photorealistic Volume Rendering using Stippling Techniques[END_REF][START_REF] Lu | Illustrative Interactive Stipple Rendering[END_REF] typically aimed at visualization applications, stipple methods for implicit surfaces [START_REF] Foster | Pen-and-Ink for BlobTree Implicit Models[END_REF][START_REF] Schmidt | Sketching, Scaffolding, and Inking: A Visual History for Interactive 3D Modeling[END_REF], point-sampled surfaces [START_REF] Xu | Stylized Rendering of 3D Scanned Real World Environments[END_REF][START_REF] Zakaria | Interactive Stylized Silhouette for Point-Sampled Geometry[END_REF], and stippling of polygonal surfaces [START_REF] Lu | Hardware-Accelerated Interactive Illustrative Stipple Drawing of Polygonal Objects[END_REF][START_REF] Sousa | Precise Ink Drawing of 3D Models[END_REF]. The placement of stipple dots in 3D space creates a unique challenge for these techniques because the viewer ultimately perceives the point distribution on the 2D plane. Related to this issue, animation of stippled surfaces [START_REF] Pastor | Real-Time Animated Stippling[END_REF][START_REF] Vanderhaeghe | Dynamic Point Distribution for Stroke-based Rendering[END_REF] presents an additional challenge as the stipples have to properly move with the changing object surface to avoid the shower-door effect. A special case of stippling of 3D models is the computation in a geometry-image domain [START_REF] Yuan | Stippling and Silhouettes Rendering in Geometry-Image Space[END_REF] where the stippling is computed on a 2D geometry image onto which the 3D surface is mapped.

While Yuan et al. [START_REF] Yuan | Stippling and Silhouettes Rendering in Geometry-Image Space[END_REF] map the computed stipples back onto the 3D surface, many approaches compute stippling only on 2D pixel images. The challenge here is to achieve an evenly spaced distribution that also reflects the gray value of the image to be represented, an optimization problem within stroke-based rendering [START_REF] Hertzmann | A Survey of Stroke-Based Rendering[END_REF]. One way to achieve a desired distribution is Lloyd's method [START_REF] Lloyd | Least Squares Quantization in PCM[END_REF][START_REF] Mccool | Hierarchical Poisson Disk Sampling Distributions[END_REF] that is based on iteratively computing the centroidal Voronoi diagram (CVD) of a point distribution. Deussen et al. [START_REF] Deussen | Floating Points: A Method for Computing Stipple Drawings[END_REF] apply this technique to locally adjust the point spacing through interactive brushes, starting from initial point distribution-generated, e. g., by random sampling or halftoning-in which the point density reflects the intended gray values. The interactive but local application addresses a number of problems: the computational complexity of the technique as well as the issue that automatically processing the entire image would simply lead to a completely evenly distributed set of points. Thus, to allow automatic processing while maintaining the desired density, Secord [START_REF] Secord | Weighted Voronoi Stippling[END_REF] uses weighted Voronoi diagrams to reflect the intended local point density. A related way of achieving stipple placement was explored by Schlechtweg et al. [START_REF] Schlechtweg | RenderBots-Multi Agent Systems for Direct Image Generation[END_REF] using a multi-agent system whose RenderBots evaluate their local neighborhood and try to move such that they optimize spacing to nearby agents with respect to the desired point density.

Besides evenly spaced distributions it is sometimes desirable to achieve different dot patterns. For instance, Mould [START_REF] Mould | Stipple Placement using Distance in a Weighted Graph[END_REF] employs a distance-based path search in a weighted regular graph that ensures that stipple chains are placed along meaningful edges in the image. In another example, Kim et al. [START_REF] Kim | Feature-Guided Image Stippling[END_REF] use a constrained version of Lloyd's method to the arrange stipples along offset lines to illustrate images of faces. However, in most cases the distribution should not contain patterns such as chains of stipple points-professional illustrators specifically aim to avoid these artifacts. Thus, Kopf et al. [START_REF] Kopf | Recursive Wang Tiles for Real-Time Blue Noise[END_REF] use Wangtiling to arrange stipple tiles in large, non-repetitive ways and show how to provide a continuous level of detail while maintaining blue noise properties. This means that one can zoom into a stipple image with new points continuously being added to maintain the desired point density.

In addition to stipple placement, another issue that has previously been addressed is the shape of stipple points. While most early methods use circles or rounded shapes to represent stipples [START_REF] Deussen | Floating Points: A Method for Computing Stipple Drawings[END_REF][START_REF] Secord | Weighted Voronoi Stippling[END_REF], several techniques have since been developed for other shapes, adapting Lloyd's method accordingly [START_REF] Hiller | Beyond Stippling -Methods for Distributing Objects on the Plane[END_REF], using a probability density function [START_REF] Secord | Fast Primitive Distribution for Illustration[END_REF], or employing spectral packing [START_REF] Dalal | A Spectral Approach to NPR Packing[END_REF].

It is also interesting to examine the differences between computer-generated stipple images and hand-drawn examples. For example, Isenberg et al. [START_REF] Isenberg | Non-Photorealistic Rendering in Context: An Observational Study[END_REF] used an ethnographic pile-sorting approach to learn what people thought about both and what differences they perceive. They found that both the perfectly round shapes of stipple dots and the artifacts in placing them can give computergenerated images away as such, but also that people still valued them due to their precision and detail. Looking specifically at the statistics of stipple distributions, Maciejewski et al. [START_REF] Maciejewski | Measuring Stipple Aesthetics in Hand-Drawn and Computer-Generated Images[END_REF] quantified these differences with statistical texture measures and found that, for example, computer-generated stippling exhibits an undesired spatial correlation away from the stipple points and a lack of correlation close to them. This lead to the exploration of example-based stippling, for instance, by Kim et al. [START_REF] Kim | Stippling By Example[END_REF]. They employ the same statistical evaluation as Maciejewski et al. [START_REF] Maciejewski | Measuring Stipple Aesthetics in Hand-Drawn and Computer-Generated Images[END_REF] and use it to generate new stipple distributions that have the same statistical properties as hand-drawn examples. By then placing scanned stipple dots onto the synthesized positions Kim et al. [START_REF] Kim | Stippling By Example[END_REF] are able to generate convincing stipple illustrations. However, because the technique relies on being able to identify the centers of stipple points in the hand-drawn examples it has problems with middle tonal ranges because there stipple dots merge into larger conglomerates.

This issue of stipple merging in the middle tonal ranges is one problem that remains to be solved. In addition, while computer-generated stippling thus far has addressed stipple dot placement, stipple dot shapes, and animation, other aspects such as how to change an input image to create more powerful results (i. e., how to interpret the input image) have not yet been addressed.

Analysis of Hand-Drawn Stippling

To inform our technical approach for generating highquality computer-generated stipple illustrations, we start by analyzing the process professional stipple illustrators perform when creating a drawing. For this purpose we involved a professional stipple artist and asked her to explain her approach/process using the example illustration shown in Figure 2. From this analysis we extract a number of specific high-level processes that are often employed by professional stipple artists that go beyond simple dot placement and use of specific dot shapes. We discuss these in Section 3.1 before analyzing the low-level properties of stipple dots in Section 3.2 that guide our synthesis process.

High-Level Processes

The manual stipple process has previously been analyzed to inform computer-generated stippling. As part of this analysis and guided by literature on scientific illustration (e. g., [START_REF]The Guild Handbook of Scientific Illustration[END_REF]), researchers identified an even distribution of stipple points as one of the major goals (e. g., [START_REF] Deussen | Floating Points: A Method for Computing Stipple Drawings[END_REF][START_REF] Secord | Weighted Voronoi Stippling[END_REF]) as well as the removal of artifacts (e. g., [START_REF] Kopf | Recursive Wang Tiles for Real-Time Blue Noise[END_REF][START_REF] Maciejewski | Measuring Stipple Aesthetics in Hand-Drawn and Computer-Generated Images[END_REF]). Also, Kim et al. [START_REF] Kim | Stippling By Example[END_REF] noted the use of tone maps by illustrators to guide the correct reproduction of tone. While these aspects of stippling concentrate on rather low-level properties, there are also higher-level processes that stipple artists often employ in their work. Artists apply prior knowledge about good practices, knowledge about shapes and properties of the depicted objects, knowledge about the interaction of light with surfaces, and knowledge about the goal of the illustration. This leads to an interpretation of the original image or scene, meaning that stippling goes beyond an automatic and algorithmic tonal reproduction.

To explore these processes further we asked Elena Piñar, a professional illustrator, to create a stipple illustration (Figure 2) from a digital photo (Figure 3(a)). We observed and video-recorded her work on this illustration and also met with her afterwards to discuss her work and process. In this interview we asked her to explain the approach she took and the techniques she employed. From this conversation with her we could identify the following higher-level processes (see Figure 3 for a visual explanation with respect to the hand-made illustration in Figure 2 and photograph in Figure 3(a)). While this list is not comprehensive, according to Elena Piñar it comprises the most commonly used and most important techniques (some of these are mentioned, e. g., by Guptill [START_REF] Guptill | Rendering in Pen and Ink[END_REF]). Also, each artist has his or her own set of techniques as part of their own personal style.

Abstraction: One of the most commonly used techniques is removing certain parts or details in the image to focus the observer's attention on more important areas. In our example, the sky and some parts of the landscape have been fully removed (shown in violet in Figure 3(b)). In addition, removing areas contributes to a better image contrast.

Increase in contrast: Some parts of the original color image exhibit a low level of contrast, reducing their readability when stippled. To avoid this problem illustrators increase the contrast in such regions (green area) through global and local evaluation of lightness, enhancing the detail where necessary.

Irregular but smoothly shaped outlines:

If objects in an image are depicted with a regular or rectilinear shape they are often perceived as being man-made. To avoid this impression for natural shapes, stipple artists eliminate parts of these objects to produce an irregular form and add a tonal gradient (yellow).

Reduction of complexity:

It is not always possible to remove all unimportant areas. In these cases the complexity or amount of detail is reduced. This effect is shown in orange in Figure 3(b): the artist has removed some small parts that do not contribute to the illustration's intended message.

Additional detail:

As visible in the red areas in Figure 3(b), some parts that are not (or not clearly) visible in the original are still shown in the illustration. Here, the illustrator has enhanced details of the rocks based on her prior knowledge.

Inversion: Sometimes artists convert very dark zones or edges into very clear ones to improve the contrast. This technique is applied subjectively to specific parts of the drawing rather than to the image as a whole.

In the hand-made stippled drawing the cracks between rocks are shown in white while they are black in the original photograph (blue in Figure 3(b)).

Despite the fact that these high-level processes are an integral part of hand-drawn stipple illustration, computergenerated stippling techniques have largely concentrated on dot placement and dot shapes. This is understandable as these low-level processes can be automated while the higher-level processes to a large degree rely on human intelligence and sense of aesthetics. To be able to incorporate higher-level interpretations of images, therefore, we manually apply global and local image processing operations to the input image (Figure 4). Instead of directly using a gray-level input image (Figure 4(a)) we first apply pre-processing to accommodate the identified high-level processes. Following the list of processes given above (see Figure 4(b)), we remove non-relevant parts from the image such as the sky and parts of the background. Also, we increase the contrast globally but also increase the brightness of some regions locally. Next, we locally delete parts of natural objects and smooth the border of these regions. To reduce the complexity of certain parts such as the metal grid we select these regions and apply a large degree of blur. Adding elements based on previous knowledge typically requires the artist painting into the image. Some additional information, however, can be added with algorithmic support, in particular edges that border regions that are similar in brightness such as the top borders of the ruin. We support this by either extracting an edge image from the original color image, taking only regions into account that have not been deleted in Figure 4(b) and adding them to the helper image. Alternatively, artists can manually draw the necessary edges as shown in Figure 4(c). Finally, inversion can be achieved by also manually drawing the intended inverted edges as white lines into the touched-up grayscale image (see Figure 4(c)).

While this interactive pre-processing could be included into a comprehensive stipple illustration tool, we apply the manipulations using a regular image processing suite (e. g., Adobe Photoshop or GIMP). This allows us to make use of a great variety of image manipulation tools and effects to give us freedom to achieve the desired effects. The remainder of the process, on the other hand, is implemented in a dedicated tool. Before discussing our algorithm in detail, however, we first discuss some lowlevel aspects of hand-drawn stipple dots that are relevant for the approach.

Low-Level Properties of Stipple Dots

Stipple dots and their shapes have been analyzed and used in many previous computer stippling techniques, inspired by the traditional hand-drawn stippling. Hodges [START_REF]The Guild Handbook of Scientific Illustration[END_REF] notes that each dot should have a purpose and that dots should not become dashes. In computer-generated stippling, therefore, dots have typically been represented as circular or rounded shapes1 or pixels that are placed as black elements on a white ground. However, each use of a pen to place a dot creates a unique shape (e. g., Figure 5) which is partially responsible for the individual characteristics of a hand-made stipple illustration. Thus, recent computer-generated stippling employed scans of dots from hand-made illustrations to better capture the characteristics of hand-drawn stippling [START_REF] Kim | Stippling By Example[END_REF].

We follow a similar approach and collected a database of stipple points from a high-resolution scan of a handdrawn original illustration, a sample of which is shown in Figure 5. These stipple dots are not all equal but have varying shapes and sizes. One can also notice that the stipples are not completely black but do exhibit a grayscale texture. This texture is likely due to the specific interaction of the pen's ink with the paper and typically disappears when the stipple illustration is reproduced in a printing process. This lead to the assumption that stipple dots are always completely black marks on a white background as used in much of the previous literature. However, the grayscale properties of real stipple dots are a characteristic of the real stippling process which one may want to reproduce provided that one employs an appropriate reproduction technology. In addition, we also make use of these grayscale properties for realizing a technique that can address one of the remaining challenges in stippling: the merging of dots in the middle tonal ranges.

A Grayscale Stippling Process

Based on the previous analysis we now present our process for high-quality example-based stippling. In contrast to previous approaches, our process captures and maintains the stipple image throughout the entire process as highresolution grayscale image, which allows us to achieve both stipple merging for the middle tonal ranges and high-quality print output. Also, to allow for interactive control of the technique, we use Ostromoukhov's [START_REF] Ostromoukhov | A Simple and Efficient Error-Diffusion Algorithm[END_REF] fast error-diffusion halftoning technique to place the stipples. Below we step through the whole process by explaining the stipple placement (Section 4.1), the stipple dot selection and accumulation (Section 4.2), and the generation of both print and on-screen output (Section 4.3). In addition, we discuss adaptations for interactive processing (Section 4.4).

Stipple Dot Placement using Halftoning

Our stippling process starts by obtaining a grayscale version of the target image using the techniques described in Section 3.1. In principle, we use this image to run Ostromoukhov's [START_REF] Ostromoukhov | A Simple and Efficient Error-Diffusion Algorithm[END_REF] error-diffusion technique to derive locations for placing stipple dots, similar to the use of halftoning to determine the starting distribution in the work by Deussen et al. [START_REF] Deussen | Floating Points: A Method for Computing Stipple Drawings[END_REF]. However, in contrast to their approach that adds point relaxation based on Lloyd's method, we use the locations derived from the halftoning directly which allows us to let stipple dots merge, unlike the results from relaxation (see Figure 6). The reason for choosing a halftoning technique over, for example, distributions based on hand-drawn stipple statistics [START_REF] Kim | Stippling By Example[END_REF] is twofold. The main reason is that halftoning has the evaluation of tone built-in so that it does not require a tone map being extracted from the hand-drawn example. The second reason lies in that halftoning provides a continuum of pixel density representing tonal changes as opposed to the approach of using both black pixels on a white background for brighter regions and white pixels on a black background for darker areas as used by Kim et al. [START_REF] Kim | Stippling By Example[END_REF].

Specifically, we are employing Ostromoukhov's [START_REF] Ostromoukhov | A Simple and Efficient Error-Diffusion Algorithm[END_REF] error-diffusion, partially because it is easy to implement and produces results at interactive frame-rates. More importantly, however, the resulting point distributions have blue noise properties, a quality also desired by related approaches [START_REF] Kopf | Recursive Wang Tiles for Real-Time Blue Noise[END_REF]. This means that the result is nearly free of dot pattern artifacts that are present in results produced by many other error-diffusion techniques, a quality that is important for stippling [START_REF]The Guild Handbook of Scientific Illustration[END_REF].

Using a halftoning approach, however, means that we produce a point distribution based on pixels that are arranged on a regular grid, in contrast to stipples that can be placed at arbitrary positions. In addition, we cannot use the grayscale input image in the same resolution as the intended output resolution for the stipple image. Let us use an example to better explain this problem and describe its solution. Suppose we have a hand-stippled A4 image (in landscape) that we scan for analysis and extraction of stipple dot examples at 1200 ppi. 2 This means that the resulting image has roughly 14,000 × 10,000 pixels, with stipple dot sizes ranging from approximately 10 × 10 to 20 × 20 pixels. 3 If we now were to produce an equivalent A4 output image at 1200 ppi, would hence use an 14,000 × 10,000 pixel grayscale image, and compute its halftoned version at this size, each pixel in this image would represent one dot. This means we would need to place scanned stipple dots (whose average size is 16 × 16 pixels, this is equivalent to a spatial size of 0.338 mm, slightly larger than the nominal size of 0.2 mm of the tip of the used pen) at the pixel locations of the halftoned image. Consequently, this would produce a result that is 16× larger than the intended output image and reproducing it at A4 size would result in the characteristics of the stipple dot shapes being lost because each stipple would again be shrunken down to the size of approximately one pixel.

Therefore, for a given output resolution res o we compute the dot distribution using error-diffusion halftoning at a smaller halftoning resolution res ht , which implies that the halftoning image must be obtained by scaling the input image. The value of the scaling is the halftoning factor f ht . Intuitively for our 1200 ppi example, one could suggest a halftoning factor f ht whose value is 1 16 of res o to compute res ht , using the average stipple's diameter of 16 pixels:

res ht = f ht • res o ; f ht = 1 16 ; res o = 1200 ppi . (1)
In a completely black region and for ideally circular stipples, however, this would result in a pattern of white spots because the black dots on the grid only touch. To avoid this issue one has to use a factor f ht of √ 2/16 to allow for a denser packing of stipple points such that they overlap (see Figure 7). For realistic stipple points with noncircular shapes one may even have to use a f ht of 2 16 or more. On the other hand, even in the darkest regions in our example the stipple density is not such that they completely cover the canvas. Thus, we leave this choice up to the user to decide how dense to pack the stipples, and introduce a packing factor f p to be multiplied with f ht to control the density of the stipples:

f ht = f p 16 . (2)
For f p = 1 and, thus, f ht = 1 16 we would perform the halftoning in our example on an image with size 875 × 625 pixels to eventually yield a 1200 ppi landscape A4 output. For other output resolutions, however, the situation is different. Because the stipples have to use proportionally smaller pixel sizes for smaller resolutions to be reproduced at the same spatial sizes, the factor between output image and halftoning image has to change proportionally as well. For example, for a 600 ppi output resolution the average stipple's diameter would only be 8 pixels, and consequently f ht would only be 1 8 . Thus, we can derive the halftoning resolution res ht for a given output resolution res o in ppi based on the observations we made from scanning a sample at 1200 ppi as follows:

res ht = f ht • res o ; f ht = f p • 1200 ppi 16 • res o res ht = 75 ppi • f p . (3)
This means that the halftoning resolution is, in fact, independent of the output resolution. Consequently, the pixel size of the halftoning image only depends on the spatial size of the intended output and the chosen packing factor (and ultimately the chosen scanned example stippling whose stipple dot size depends on the used pen). This leaves the other mentioned problem that arises from computing the stipple distribution through halftoning: the stipple dots would be arranged on a regular grid, their centers always being at the centers of the pixels from the halftoning image. To avoid this issue, we perturb the locations of the stipple dots by a random proportion of between 0 and ±100% of their average diameter, in both x-and y-direction. Together with the random selection of stipple sizes from the database of scanned stipples and the blue noise quality of the dot distribution due to the chosen halftoning technique this successfully eliminates most observable patterns in dot placement (see Figure 8).

Stipple Dot Selection and Accumulation

We begin the collection of computed stipple points by creating a grayscale output buffer of the desired resolution, with all pixels having been assigned the full intensity (i. e., white). Then we derive the stipple placement by re-scaling the modified grayscale image to the halftoning resolution and running the error-diffusion as described in the previous section. Based on the resulting halftoned image and the mentioned random perturbations we can now derive stipple location with respect to the full resolution of the output buffer.

For each computed location we randomly select a dot from the previously collected database of stipple dot scans. This database is organized by approximate stipple dot sizes, so that for each location we first determine the size class using a random function that follows a uniform distribution (all the sizes have the same probability) or a normal distribution centered on the average size, and then randomly select a specific dot from this class. The selection of which random function to use is of aesthetic nature and left up to the user. The default random function uses the uniform distribution, producing a less regular result. The selected dot is then added to the output buffer, combining intensities of the new stipple dot i s and the pixels previously placed into the buffer i bg using (i s • i bg )/255 (for 8 bits). This not only ensures that stipples placed on a white background are represented faithfully but also that the result gets darker if two dark pixels are combined (accumulation of ink; for example, given i s = 45 and i bg = 64, the result is 11 which is darker than the original values).

Both the range of stipple sizes and the partially random stipple placement ensure that stipples can overlap. This overlapping is essential for our approach, it ensures the gradual merging of stipples into larger conglomerates as darker tones are reproduced (see example in Figure 9). Therefore, we can for the first time simulate this aspect of the aesthetics of hand-drawn stippling.

Generation of Print and Screen Output

One challenge that remains is to generate the appropriate output for the intended device. Here we typically have two options: on-screen display and traditional print reproduction. These two options for output differ primarily in their spatial resolution and their color resolution. While normal bilevel printing offers a high spatial resolution (e. g., 1200 dpi), it has an extremely low color resolution (1 bit or 2 bit) while typical displays have a lower spatial resolution (approximately 100 ppi) but a higher color resolution (e. g., 8 bit or 256 bit per primary). These differences also affect the goals for generating stipple illustrations. For example, it does not make much sense to print a grayscale stipple image because the properties of the individual stipple points (shape, grayscale texture) cannot be reproduced by most printing technology, they would disappear behind the pattern generated by the printer's halftoning [START_REF] Isenberg | Breaking the Pixel Barrier[END_REF]. In contrast, for on-screen display it does not make sense to generate a very high-resolution image because this cannot be seen on the screen. Therefore, we adjust our stippling process according to the desired output resolutions, both color and spatial. For output designed for print reproduction we run the process at 1200 ppi, using a stipple library from a 1200 ppi scan, and compute the scaling factor for the halftoning process to place the stipples accordingly. The resulting 1200 ppi grayscale output image is then thresholded using a user-controlled cut-off value, and stored as a 1 bit pixel image, ready for high-quality print at up to 1200 dpi (e. g., Figure 16). These images, of course, do no longer contain stipples with a grayscale texture but instead are more closely related to printed illustrations in books.

As a print reproduction alternative we also provide the option to apply a dedicated halftoning step (using Ostromoukhov's [START_REF] Ostromoukhov | A Simple and Efficient Error-Diffusion Algorithm[END_REF] error-diffusion algorithm) to a grayscale stippling result. This allows us to maintain the grayscale properties of the stipple dots even in a black-and-white output medium such as print (Figure 10). For instance, for 1200 dpi output we can reproduce approximately 32 grayscale tones, while bypassing the problem that many printers apply a Postscript dithering which is limited compared to Ostromoukhov's stochastic dithering. On sufficiently high-resolution (1200 dpi and higher) printers, the dedicated dithering allows us to maintain the grayscale properties of the stipple dots while the dither pattern is close to being unnoticeable.

For on-screen display, in contrast, we run the process at a lower resolution, e. g., 300 ppi (while 300 ppi is larger than the typical screen resolution, it also allows viewers to zoom into the stipple image to some degree before seeing pixel artifacts). For this purpose the stipples in the database are scaled down accordingly, and the appropriate scaling value for the halftoning process is computed based on average stipple size at this lower resolution. The resulting image (e. g., Figure 18) is smaller spatially but we preserve the texture information of the stipples. These can then be appreciated on the screen and potentially be colored using special palettes (e. g., sepia). In addition, grayscale stipple images can also be used for special continuous tone printing processes such as dye-sublimation.

User Interaction

Several parameters of the process can be adjusted interactively according to aesthetic considerations of the user, in addition to applying the high-level processes (parallel or as pre-processing). The most important settings are the intended (spatial) size and output resolution because these affect the resolutions at which the different parts of the process are performed. For example, a user would select A5 as the output size and 300 ppi as the intended resolution. Based on this the (pixel) resolution of both output buffer and halftoning buffer are derived as outlined in Section 4.1. To control the stipple density, we let the users interactively adjust the packing factor (the default value is 2). In addition we let users control the amount of placement randomness as a percentage of the average size of the stipple dots (the default value is 25%). This means that we specify the packing factor and placement randomness based on the average stipple size at the chosen resolution, which results in visually equivalent results regardless of which specific resolution is chosen.

Stipple Dot Placement Improvements

Of course, our general approach is not without flaws. One problem arises from the use of halftoning on a resolution lower than the output one to derive stipple distributions because this initially leads to the stipples being placed on a grid. While we address this grid arrangement by introducing randomness to the final stipple placement, this also leads to noise being added to otherwise clear straight lines in the input image. This effect can be observed by comparing the upper edge of the ruin in Figure 16 with the same location in the hand-made example in Figure 2 where the line of stipple dots is nicely straight. Given the importance of lines and borders in the illustration process, we address this issue by analyzing the local character of the source image. We detect the edges in the source image using an edge detector. These border filters produce grayscale images, with darker values showing that borders are nearer. This value is used to control the random displacement of the stipple dots: the degree of the detected edge (d e , value between 0 and 1) controls the maximum random displacement, i. e., it is used as a factor for the stipple points average diameter as used in the displacement computation. Figure 11 illustrates this displacement process, showing that reducing the displacement for stipples dots on edges reduces the noise introduced for these effects. While the improvement is subtle in these detail images, it is more visible in complete images (see the examples in . This seems to be a global perception since the dots are better placed overall in the image, producing a more contrasted and defined result.

Specifically, we experimented with the Sobel operator [START_REF] Sobel | A 3 × 3 Isotropic Gradient Operator for Image Processing[END_REF] because it is fast to compute and with the Difference of Gaussians (DoG) [START_REF] Marr | Theory of Edge Detection[END_REF] because it relates to the processing in the human perceptual system. While there is no ideal method to compute the borders to control the random displacement, the ones we implemented allow users to experiment with the effects by controlling their parameters.

The control of randomness of stipple dot placement is combined with other measures to reduce the noise on lines and borders. These measures include more control over the sizes of the stipple dots using a random selection that follows a normal distribution centered around the mean dot size to ensure that most dots on a line or border have the same mean size which maintains the aesthetic of more regularity of stipple placement on these lines.

In addition, the dot placement needs to be controlled based on the visual center of the scanned dot so that linedup dots also visually form a line. The concept of a visual center is based on the observation that, while the scanned dots are embedded in a regular and square grid, they do not have to have a regular shape or ink distribution and the geometric center of the grid is not necessarily the visual center. For example, the dot may have one main dark zone that is close to a corner, or there may be two dark zones, etc. For computing the visual center we use a straightforward approach-we 'posterize' each dot image: First, the number of gray levels is reduced to a low number of levels, usually between 4 and 7 (see Figure 12). Then, the mean position of the pixels in the darkest level is computed and recorded as the stipple dot's visual center.

All techniques used together allow us to improve the treatment of borders and lines in the stipple images but still fail to solve all issues. For example, our resolutiondependent stippling requires us to scale the input image to the size of the halftoning image, which is then subjected to the halftoning process to inform the stipple placement. The linear interpolation employed in this scaling process leads to a distribution of gray values that, after halftoning, may lead to the black pixels used for stipple dot placement not being aligned even though the original was a perfectly straight line. This produces patterns as shown in Figure 13. We leave the treatment of this problem for future work.

Performance and GPU Implementation

An important aspect to discuss about the stippling process is its computation and rendering performance. While we can easily allow interactive work with the CPU implementation at resolutions of up to 300 ppi for A4 output, the process is less responsive for larger images. For example, stippling the image shown in Figure 4(c) takes approximately 0.51, 0.25, and 0.13 seconds for A4, A5, and A6 output, respectively, while a completely black input image requires 1.41, 0.70, and 0.36 seconds, respectively (Intel Core2 Duo E6600 at 3GHz with 2GB RAM, running Linux). However, our approach can easily allow users to adjust the parameters interactively at a lower resolution and then produce the final result at the intended high resolution such as for print output. For example, stippling Figure 4(c) on A4 at 1200 dpi in black-and-white takes approximately 10.1 seconds while a completely black image requires approximately 15.2 seconds. We ensure that both lowand high-resolution results are equivalent by inherently computing the same halftoning resolution for both resolutions using the resolution-dependent scaling factor and appropriately seeding the random computations.

We can improve the performance of the computation by implementing the stipple dot placement on the GPU. This is particulary useful for systems where the CPU is slower than the GPU such in some notebooks or netbooks or when the technique is used on high-resolution or large displays. For this purpose we store the library of scanned stipple points in a texture, load the halftoned image into another texture, generate a texture with random numbers due to the CPU producing better pseudo-random numbers than the GPU, and prepare a frame buffer object (FBO) to record the final output as shown in Figure 14.

We then create a fragment shader that takes over the task of placing the stipples which is the time-consuming part in the CPU implementation but which can be highly parallelized on the GPU. To create this shader we have to consider that a shader only sees the fragment it is processing without being able to look into its neighborhood and that the shader is computed in parallel for many fragments at a time. The latter, in particular, means that the shader could potentially compute contributions for the same output fragment but that arise from different input fragments due to the overlapping neighboring stipple dots (from packing and random offset). However, since the fragment shaders cannot operate recursively, we need to ensure that, while placing one stipple dot, no other neighboring stipple dots are being placed. We avoid the processing of overlapping stipple dots by only computing dots on a regular grid in one pass, spaced apart in both the horizontal and the vertical direction (see Figure 15). The spacing between concurrently processed dots depends on the maximum dot size, the maximum random displacement, and the packing factor, and the scaling factor. Specifically, the minimal separation for the dots is computed as the size of the larger dot divided (integer division) by the average size of the dot and then multiplied by the average size of the dot plus twice the average size of the dot. Then, the process/pass is repeated by repeatedly moving the grid to the next untreated set of stipple dots until all potential dots have been placed. For example, the largest dot in our dot library for 1200 dpi b/w images has a size of 56 2 pixels, while the average size is 24 2 pixels. Thus, the separation is computed as sep = (ceil(size largest dot /size average dot ) + 2) • size average dot = (ceil(56/24) + 2) • 24 = 5 • 24 to be 120 pixels.

The number of passes per coordinate axis depends on how dense the stipple points lie to each other, which is controlled by the packing factor f p . The total number of passes, therefore, can be computed as passes = ( f loor(sep• f p /size average dot )) 2 . Hence, in our example we have to compute 100 passes of the shader code using a packing factor of 2. An example of the two first steps of this algorithm is shown in Figure 15. 4To ease the actual stipple dot placing, we actually scale the halftoning input image during pre-processing without interpolation (NEAREST filter) to avoid repeated position calculations. During processing, the shader determines if it is treating an active grid location and checks for a black pixel in the scaled halftoning texture. If this is the case, the shader determines the center location of the stipple point currently being treated, the pixel offset of the fragment being considered to the center of the dot, with this derives the stipple dot texel, and determines the overall offset for the current stipple dot. Based on this computation, the gray level of the output fragment is being determined and blended with the previous fragment at this location. For this purpose we use two FBOs in an alternating setup to avoid feedback loops. This means that the newly computed fragment is blended with the previous fragment at this location (from FBO 2 ) and is placed into FBO 1 . If no new fragment is computed (not on active grid or halftoning pixel not black) the fragment from FBO 2 is simply copied into FBO 1 . After each pass, the pointers to both FBOs are swapped. The result of the GPU algorithm is exactly the same as the basic CPU algorithm only if the random values are stored in the same order and if the blending operation were commutative (which is the case of our CPU implementation). Thus, the difference between both algorithms is the order of rendering because the GPU algorithm chooses the dots that do not overlap in the same pass, and then blending them with a previous set of dots. The main limitation with respect to the CPU algorithm lies in the memory limits of the GPU because the result is captured in the two FBOs, thus the GPU technique is limited with respect to the image sizes it can process.

The performance of the algorithm depends on the parallelization of the dots rendering. If the packing factor is small (i. e., 1-3), the algorithm will work very fast (i. e., approx. 100 fps for an A5 image at 1200 ppi on a GeForce 9400 GT) because grid spacing is small and thus the number of passes will be small (i. e., approx. 121). The reason that 121 passes are still rendered very fast is that the shader is relatively simple and that each pass only needs to render a single, screen-filling quad. In addition, the fragment shader used does not contain any loops so the operations are executed very fast for every fragment. However, if the packing factor is high (e. g., approx. 6), the number of passes will increase (approx. 529 in this case), thus losing performance (approx. 5 fps for the same image). Nevertheless, the algorithm on the GPU is faster than on the CPU because the number of black stipple dots in the halftoning image has almost no affect on the performance of the GPU algorithm.

Results and Discussion

Figure 16 shows a synthesized stipple image based on the photo (Figure 3(a)) in its touched-up form (Figure 4(c)) that was also used to create the hand-drawn example in Figure 2. Figure 16 was produced for print-reproduction at A5 and 1200 dpi. As can be seen from Figure 17, our process can nicely reproduce the merging of stipples in a way that is comparable between the hand-drawn example and the computer-generated result. In addition, this process preserves the characteristic stipple outlines found in hand-drawn illustrations. Figure 18 shows an example produced for on-screen viewing. In contrast to the black-and-white image in Figure 16, this time the grayscale texture of the stipple dots is preserved. In fact, in this example we replaced the grayscale color palette with a sepia palette to give the illustration a warmer tone, a technique that is often associated with aging materials. However, in typical print processes, images like this will be reproduced with halftoning to depict the gray values. These halftoning patterns typically 'fight' with the stipple shapes and placement patterns. To avoid these, one has to produce b/w output as discussed before by employing thresholding (or through stochastic dithering if the printer has a sufficiently high resolution) or use dye-sublimation printing which can reproduce gray values.

Figures 202122show examples with improved stipple placement using edge detectors. In particular Figure 22 exhibits an improved stipple placement which is visible, for example, at the edges of windows and other dark zones. For comparison, Figure 23 shows the higher-resolution black-and-white version of Figure 22. One final aspect that we would like to discuss is the use of a halftoning image as the basis for the stipple placement. While we have presented the reasons for our decision in Section 4.1, one may also argue that it could be better to use other types of halftoning (e. g., [START_REF] Chang | Structure-Aware Error-Diffusion[END_REF]) or try to adapt Kim et al.'s [START_REF] Kim | Stippling By Example[END_REF] technique to fit our needs. To investigate this issue further, we took samples from both Figures 2 and16 and analyzed these using the statistical approach described by Maciejewski et al. [START_REF] Maciejewski | Measuring Stipple Aesthetics in Hand-Drawn and Computer-Generated Images[END_REF]. The result of this analysis is shown in we compared the samples with respect to the correlation, energy, and contrast measures. We can see in Figure 19 that the the curves for related regions closely match each other, regardless if the sample was taken from the handdrawn image or the computer-generated one. In addition, the curves representing the two distinct regionsdensely stippled and sparsely stippled-also match each other quite closely. Finally, the graphs show that our computer-generated examples do not exhibit the correlation artifacts described by Maciejewski et al. [START_REF] Maciejewski | Measuring Stipple Aesthetics in Hand-Drawn and Computer-Generated Images[END_REF] for other computer-generated stippling techniques. Thus, we can conclude that our choice to base the stipple distribution on halftoning seems to be justified.

Conclusion and Future Work

In summary, we presented a scale-dependent, examplebased stippling technique that supports both low-level stipple placement and high-level interaction with the stipple illustration. In our approach we employ halftoning for stipple placement and focus on the stipples' shape and texture to produce both gray-level output for on-screen viewing and high-resolution binary output for printing. By capturing and maintaining the stipple dots as grayscale textures throughout the process we solve the problem of the merging of stipple dots at intermediate resolutions as previously reported by Kim et al. [START_REF] Kim | Stippling By Example[END_REF]. The combined technique allows us to capture the entire process from artistic and presentation decisions of the illustrator to the scale-dependence of the produced output. We discuss both CPU and GPU implementations of the technique, the CPU version allowing us to add further control while the GPU version can be run at real-time frame-rates for better interactive experimentation with the parameters.

One of the interesting observations from our stippling process is that the resolution at which the stipple distribution occurs (using halftoning in our case) depends on the spatial size of the target image but needs to be independent from its resolution, just like other pen-andink rendering [START_REF] Salisbury | Scale-Dependent Reproduction of Pen-and-Ink Illustration[END_REF][START_REF] Jeong | Detail Control in Line Drawings of 3D Meshes[END_REF][START_REF] Ni | Multi-Scale Line Drawings from 3D Meshes[END_REF]. For example, there should be the same number of stipples for a 1200 ppi printer as there should be for a 100 ppi on-screen display. However, there need to be fewer stipples for an A6 image compared to an A4 image. This complements the observation by Isenberg et al. [START_REF] Isenberg | Non-Photorealistic Rendering in Context: An Observational Study[END_REF] that stippling with many dense stipple points is often perceived by viewers to be computer-generated.

While our approach allows us to support the interactive creation of stipple illustrations, this process still has a number of limitations. One of the limitations we mentioned previously consists of issues concerning stipple placement at edges in the input image, where edges in the image would become fuzzy due to the introduced randomness. We partially solved this issue by controlling the amount of randomness when placing the stipples, depending on the results of an edge filter. However, the scaling involved in our scale-dependent stippling process combined with halftoning as the underlying placement principle still results in some lines not being as crisp as would be desired. One of the most important remaining limitations concerns the presentation of the interaction: the use of high-level processes as described in Section 3.1 is currently a separate process that does require knowledge of the underlying artistic principles-a better integration of this procedure into the user interface would be desirable. Also, we would like to investigate additional algorithmic support for these high-level interaction. This includes, for example, an advanced color-to-gray conversion techniques [START_REF] Gooch | Color2Gray: Salience-Preserving Color Removal[END_REF][START_REF] Rasche | Detail Preserving Reproduction of Color Images for Monochromats and Dichromats[END_REF][START_REF] Rasche | Re-Coloring Images for Gamuts of Lower Dimension[END_REF] to support illustrators in their work. In addition, an interactive or partially algorithmically supported creation of layering or image sections according to the discussed high-level criteria such as background, low or high detail, level of contrast, or inversion would be interesting to investigate as future work. For this automatic or saliencebased abstraction techniques [START_REF] Santella | Visual Interest and NPR: An Evaluation and Manifesto[END_REF] could be employed.

Figure 1 :

 1 Figure 1: An example stipple image created with our technique.

Figure 2 :

 2 Figure 2: Hand-drawn stipple image by illustrator Elena Piñar of the Roman theater of Acinipo in Ronda, Málaga (originally on A4 paper). This image is © 2009 Elena Piñar, used with permission.

  (a) Original photograph, Roman theater of Acinipo in Ronda, Málaga. (b) Interpreted regions.

Figure 3 :

 3 Figure 3: Original photograph and interpreted regions for Figure 2.

  (a) Grayscale version of Figure 3(a). (b) Adjusting global contrast and local detail of (a).

  (c) Manually added edges, inversion, local contrast.

Figure 4 :

 4 Figure 4: Deriving the helper image to capture the high-level stippling processes.

Figure 5 :

 5 Figure 5: Enlarged hand-drawn stipple dots (scanned at 1200 ppi).

Figure 6 :

 6 Figure 6: Basic process: one black pixel in the halftoning image is converted to one dot in the final image.

Figure 7 :

 7 Figure 7: Effect of the packing factor.

  (a) Grid placement of stipples. (b) After random perturbation.

Figure 8 :

 8 Figure 8: Magnified comparison of stipple placement before and after random perturbation of the stipple locations.

Figure 9 :

 9 Figure 9: Merging of synthesized stipples at two tonal ranges.

Figure 10 :

 10 Figure 10: Details of the 1200ppi outputs with the grayscale, black and white, and halftoning process.

  DoG filter. (c) Improved with DoG. (d) Sobel filter. (e) Improved with Sobel.

Figure 11 :

 11 Figure 11: Details of the improvement in the placement of borders dots (the images of the filters are scaled).

Figure 12 :

 12 Figure 12: High resolution dots (left) and their 'posterized' versions (right) for determining the dots' visual centers.

Figure 13 :

 13 Figure 13: Scaling problem with non horizontal straight lines.

Figure 14 :

 14 Figure 14: Three textures are passed to a fragment shader program: the halftoning, the stippling points, and a texture containing random values.The image is rendered into a frame buffer object (FBO). The partial result is blended with the previous one. Two FBOs are needed for this process because a FBO cannot feed itself.

Figure 15 :

 15 Figure15:The two first steps of the algorithm: In the 1 st step (left), the algorithm renders the dots with the necessary distance to prevent overlapping. In the 2 nd step (right), the next dots are rendered and blended to the previous step. The algorithm is repeated until the final illustration is complete.

Figure 16 :

 16 Figure 16: Example generated for A4 print reproduction at 1200 dpi resolution, using Figure 4(c) as input.

Figure 17 :

 17 Figure 17: Comparison of stipple merging between a hand-drawn and a synthesized sample, taken from the same region of the images.

  Figure 19 and reveals that the examined hand-drawn and our computer-generated stippling examples exhibit almost identical statistical behavior when compared with each other. Following the lead by Maciejewski et al. [27],

Figure 18 :

 18 Figure 18: Sepia tonal stipple example generated for A4 on-screen viewing at up to 300 ppi resolution, with slight gamma correction.

Figure 19 :

 19 Figure 19: Samples and results of the statistical evaluation.

Figure 20 :

 20 Figure 20: A5 300 ppi grayscale stippling of Annecy without enhancing the borders.

Figure 21 :

 21 Figure 21: The same example enhanced with a Sobel filter.

Figure 22 :

 22 Figure 22: The same example enhanced with a DoG filter.

Figure 23 :

 23 Figure 23: The 1200 ppi black-and-white version of Figure 22.

Aside from more complex artificial shapes that have also been used but that are not necessarily inspired by hand-drawn stippling, see Sec-

tion 2.

Please notice that ppi stands for pixels per inch and is used intentionally while dots per inch (dpi) is used when we discuss printing.

[START_REF] Lu | Non-Photorealistic Volume Rendering using Stippling Techniques[END_REF] These values are derived from the example in Figure2(done with a pen with a 0.2 mm tip) and can be assumed to be valid for many stipple images as similar pen sizes are used by professional illustrators[START_REF]The Guild Handbook of Scientific Illustration[END_REF].

The detailed algorithm can also be reviewed in the shader code of the demo.

Acknowledgments

We thank, in particular, Elena Piñar for investing her time and creating the stippling examples for us. We also thank Ross Maciejewski who provided the stipple statistics script and Moritz Gerl for his help with Matlab, and the reviewers for their nice suggestions on improving the paper. Finally, we acknowledge the support of the Spanish Ministry of Education and Science to the projects TIN2007-67474-C03-02 and TIN2007-67474-C03-01, and the Ministry of Innovation, Science, and Business of the Junta de Andalucía to the project PE09-TIC-5276.


Matthew Van Der Zwan

Wouter Lueks

Henk Bekker

Tobias Isenberg

Illustrative Molecular Visualization with Continuous Abstraction

Keywords: Descriptors (according to ACM CCS): I.3.m [Computer Graphics]: Miscellaneous-Scientific visualization, molecular visualization, illustrative visualization, dedicated seamless abstraction

Molecular systems may be visualized with various degrees of structural abstraction, support of spatial perception, and 'illustrativeness.' In this work we propose and realize methods to create seamless transformations that allow us to affect and change each of these three parameters individually. The resulting transitions give viewers a dedicated control of abstraction in illustrative molecular visualization and, consequently, allow them to seamlessly explore the resulting abstraction space for obtaining a fundamental understanding of molecular systems. We show example visualizations created with our approach and report informal feedback on our technique from domain experts.

Introduction

Molecular visualization is of tremendous importance for understanding processes that are relevant in fields such as material sciences, genetics, pharmacy, immunology, and biology and chemistry in general. Researchers in these domains are trying to cope with an ever increasing complexity of molecular data while trying to gain insight in the structure and function of large molecules such as proteins. In our work we exlore the application of illustrative visualization techniques to this field of molecular visualization which is particularly suitable for illustrative visualization approaches because no 'photorealism' exists at the sizes of atoms [START_REF] Goodsell | Illustrating Molecules[END_REF].

Research into the structural properties of small and large molecules is increasingly gaining importance. The function and interaction of molecules is often primarily analyzed based on an understanding of this structural information. Researchers have a number of tools at their disposal to visualize the collected molecular data [OGF * 10], for example PyMol, VMD and Chimera. Visualizations are created by blending (parts of) the molecule shown in different structural representations. However, the connection between these representations is often difficult to explore by switching/blending alone. It is this problem that we address with our work.

Based on the structural data that is available in large molecular databases [BWF * 00, BBB * 02] and by taking inspiration from traditional hand-made illustration styles [Hod03], we describe how to visualize continuous transitions between different stages of structural abstraction as well as aspects of spatial perception and 'illustrativeness' (e. g., Fig. 1). All of these parameters can be controlled independently and continuously in real-time to enable users to interactively explore the structure of complex molecules.

In particular, we extend a previous GPU-based approach for molecular visualization [START_REF] Tarini | Ambient Occlusion and Edge Cueing to Enhance Real Time Molecular Visualization[END_REF] such that we can transition from solid geometry to a planar one and provide means to move to a smooth, spline-based representation. In addition, we support a seamless interpolation between 'photorealistic,' cel shading, and stylized black-and-white depiction. Finally, we integrate techniques such as object attenuation and halos for all these abstraction stages.

In summary, our main contribution is the continuous abstraction space for illustrative molecular visualization. This abstraction space facilitates an interactive and seamless exploration of structural information as well as depiction styles for molecular data in which each of the aspects can be controlled individually. We discuss the realization of the transitions and their implementation using GPU techniques and report on informal feedback from domain experts.

In the remainder of this paper we first review work related to our own in Section 2. Next, we discuss abstraction techniques for molecular visualization and identify aspects that we combine to form a continuous abstraction space in Section 3. In Section 4 we then describe how we realize our molecular visualization using this abstraction space. Afterward, we present results of our technique and report informal feedback on our technique in Section 5. Finally, Section 6 concludes the paper and discusses aspects of future work. * 10] give a quite comprehensive overview of the different techniques that are being practically applied to the domain of molecular visualization (for an overview of visualization techniques also see [LLC * 11, Section 4.3]), as well as of the visualization tools available for researchers. They highlight non-photorealistic (or illustrative) visualization techniques as very effective methods to depict the overall shape and form of molecules, in particular for presentation to others and for publication. While O'Donoghue et al. mention and show an image of using "flat colors and outlines" as an example for such non-photorealistic depiction, others have introduced more advanced methods of illustrative visualization in this field which we describe next.

Related Work

O'Donoghue et al. [OGF

In an early example, Goodsell and Olson [START_REF] Goodsell | Molecular Illustration in Black and White[END_REF] describe a number of simple techniques to visualize the molecular surface using the parallel hatching and silhouettes techniques that were the state of the art in the early 1990s and also discuss first approaches to some structural abstraction using cylinders and cut-aways. A similar tool for grayscale shaded illustrations of detailed and abstracted proteins was described by Kraulis [START_REF] Kraulis | MOLSCRIPT: A Program to Produce both Detailed and Schematic Plots of Protein Structures[END_REF]. In more recent approaches, Lampe et al. [START_REF] Lampe | Two-Level Approach to Efficient Visualization of Protein Dynamics[END_REF] use a two-level approach on the GPU to illustrate slow dynamics of molecules, while Weber [START_REF] Weber | ProteinShader: Illustrative Rendering of Macromolecules[END_REF] explores the use of texture-based approaches using shader programming to generate pen-and-ink renderings of molecules in real-time. Weber focuses, in particular, on producing cartoon-style illustrations for publication and on permitting users to apply different types of abstraction to different parts of the molecule. While we also rely on shader programming for fast rendering, we concentrate, in particular, on enabling the seamless transition between different rendering and abstraction styles, in addition to being able to apply different styles to different parts of the model.

One particularly relevant illustrative visualization of large molecules was presented by Tarini et al. [START_REF] Tarini | Ambient Occlusion and Edge Cueing to Enhance Real Time Molecular Visualization[END_REF]. They use ambient occlusion as an approximation of global illumination as well as additional techniques such as halos to improve the perception of the spatial structure of large ballsand-sticks and space-fill models. We also employ Tarini et al.'s imposter-based rendering with ambient occlusion and halos but focus on how to transition along the axes of structural abstraction and visual styles in an integrated fashion.

An approach that is different from using space-fill, ballsand-sticks, and related techniques that show the inner structure of a molecule is to visualize its outer surface [CG07, CPG09,KBE09,CWG * 10] which is important to understand the interactions between different molecules. For example, Cipriano et al. [START_REF] Cipriano | Molecular Surface Abstraction[END_REF] not only examine the illustrative depiction of the molecular surface but also explore abstraction as well as the placement of decals to represent features that have been removed through the simplification. While we do not employ surface-based visualization techniques it would be possible and interesting to combine them with the visualizations of internal structure that we explore. Inspiration for our illustrative visualizations also comes from traditional illustration techniques [START_REF]The Guild Handbook of Scientific Illustration[END_REF]. For example, Goodsell [START_REF] Goodsell | Illustrating Molecules[END_REF] summarizes the state of the art of molecular illustration. He emphasizes the existence of traditional schemes including the space-filling diagram and the balls-and-sticks model as well as of structural abstractions such as the ribbon diagram [START_REF] Richardson | Schematic Drawings of Protein Structures[END_REF], all of which we also support with our technique. Interesting for our work are, in particular, Goodsell's black-and-white examples. Here he uses traditional black-and-white shading techniques such as hatching and stippling to portray the atom types for print reproduction which are otherwise often rendered in specific color schemes. We use a similar approach but show how to seamlessly transition between the two extremes.

A Continuous Abstraction Space

In their survey of molecular visualization [OGF * 10], O'Donoghue et al. note the importance of being able to get an overview of a molecule's structure and point out that both non-photorealistic/illustrative visualization techniques as well as structural abstraction such as the ribbon diagram very well support this goal. They also remark, however, that being able to "see where sequence features are located in the three-dimensional structure can be of substantial practical value." This means that being able to mentally integrate both detailed views with abstracted and potentially stylistic depictions is essential, which to date is usually only possible by switching or α-blending between different visualizations.

Abstraction in the context of molecular visualization typically refers to structural abstraction. Here, various forms of depicting the structure of a molecule are commonly used (e. g., see [START_REF] Goodsell | Visual Methods from Atoms to Cells[END_REF]): the space fill diagram (Fig. 2(a)) which depicts each atom using its van der Waals radius, the ballsand-sticks model (Fig. 2(b)) which uses smaller radii and adds bonds as cylinders, and the ribbons model [START_REF] Richardson | Schematic Drawings of Protein Structures[END_REF] (Fig. 2(e)) which further abstracts parts of the molecule to ribbon helices and sheets so that secondary, tertiary, and quaternary structures become apparent. A stage between the balls-and-sticks and ribbon models is the licorice visualization (Fig. 2(c)) that only shows the bonds, colored according to the typical colors associated to atom types. The removal of less important parts of this structure leads to abstractions that emphasize the molecule's backbone [START_REF] Namba | Enhancement and Simplification of Macromolecular Images[END_REF] (Fig. 2(d)). These structural abstraction stages form a natural progression (as in Fig. 2) which we can place along an axis of structural abstraction, and for which we later define transitions in order to support the mental integration.

Besides the mentioned stages there exists a variety of additional structural abstractions. In addition to the previously mentioned surface visualizations [CG07, CPG09, KBE09], the inner structure of molecules can be abstracted with cartoon views where β -sheets are depicted as arrows [START_REF] Delano | PyMOL User's Guide[END_REF] or simplified 2D schematics (see, e. g., [OGF * 10, Fig. 4(g,h)]). Other abstraction types are coarse-grained abstractions (e. g., [MRY * 07]) that combine several atoms into larger pseudoatoms to facilitate the simulation of very large systems. We currently do not support these abstractions because most represent a fundamental paradigm shift, while our work focuses on the aspect of dedicated control of abstraction. However, one can envision potential extensions such as transitions from the space fill diagram to the surface visualization or from the ribbon visualization to coarse-grained models, potentially leading to more than one possible abstraction path.

Aside from structural abstraction, stylistic rendering is also frequently applied in molecular illustration and visualization (e. g., [Ric85,Goo03,Goo05,Web09,OGF * 10]). This type of depiction can also be considered to be a type of abstraction. By reducing the detailed shading, for example, il-lustrative depictions of molecules can abstract from otherwise overwhelming detail and instead highlight the overall shape [Goo05, OGF * 10]. Illustrative depiction can also support the use of structural abstraction by emphasizing through stylization the fact that abstraction has been applied [START_REF] Cipriano | Molecular Surface Abstraction[END_REF]. Finally, we consider the use of visualization techniques that more or less support the perception of the spatial shape (ambient occlusion and halos [START_REF] Tarini | Ambient Occlusion and Edge Cueing to Enhance Real Time Molecular Visualization[END_REF] or halos and line attenuation [START_REF] Everts | Depth-Dependent Halos: Illustrative Rendering of Dense Line Data[END_REF]) to be a third axis of abstraction that can be applied in molecular visualization. These techniques selectively reduce detail and emphasis in certain places such as the inside of a molecule or visually distant parts and, therefore, introduce localized abstraction [START_REF] Luft | Watercolor Illustrations of CAD Data[END_REF].

Researchers work with visualizations in diverse combination of the mentioned abstractions, both structurally and visually [OGF * 10]. Higher levels of abstraction, e. g., are well suited to provide an overview, but at the same time these lack detail [START_REF] Cipriano | Molecular Surface Abstraction[END_REF] which may be required for other tasks. Moreover, it is not only desirable to be able to combine views of different levels of scale or abstraction [START_REF] Goodsell | Visual Methods from Atoms to Cells[END_REF], but in fact to be able to seamlessly transition between abstraction levels. This seamless transition is what we are addressing in this work. For this purpose we define an abstraction space whose main axis is that of structural abstraction; this axis is augmented by changes to the visual style of the visualization: 'illustrativeness' and support of perception of spatiality. We need each of these independent axes to be continuous (i. e., not to contain visible 'jumps') to support the desired seamless navigation and exploration. This continuity implies that we need to define an order between the previously discrete stages within each axis and meaningful transitions between adjacent ones. This order is given for structural abstraction as depicted in Fig. 2, and understandable stages in-between these discrete stages can easily be envisioned.

More formally, the abstraction space is a space F of functions. Every function

f (t s ,t p ,t i ) ∈ F with t s ,t p ,t i ∈ [0, 1] consists of a function triple ( f s (t s ), f p (t p ), f i (t i ))
, where f s determines the degree of structural abstraction, f p the support of spatial perception, and f i the 'illustrativeness.' Each of these functions f k has several discrete levels t k,n , evenly spaced in [0, 1], associated to it that mark known styles. Each f k needs to specify how to seamlessly transition between f k (t k,n ) and f k (t k,n+1 ). A mapping A assigns to each amino-acid a in the protein a tuple (t s (a),t i (a)), thus determining its style. The parameter t p can only be controlled globally.

Interactive Visualization with Seamless Abstraction

To achieve our goal of seamless structural and visual abstraction we first describe how to achieve the continuous transition of f s between the structural abstraction stages t s,n before outlining the realization of the visual abstractions f p and f i .

From Space Filling to Ribbon Visualization

The seamless structural transition from space-fill (t s,0 ) towards balls-and-sticks (t s,1 ) is straightforward, both conceptually and in its implementation. Spheres represent atoms with their van der Waals radii in the space fill model, while smaller spheres (whose sizes do not have a physical meaning) with additional cylinders to represent the bonds are used for balls-and-sticks. Thus, we can gradually abstract from the space fill to the balls-and-sticks representation by reducing the spheres' sizes, hence revealing the bond-representing cylinders that were previously hidden by the larger spheres. A further shrinking of the spheres until they vanish results in the licorice representation (t s,2 ). We confirmed for the resulting animation with our collaborating domain experts that they create nothing but the expected visualizations.

The next discrete stage in the common structural abstraction sequence shows only the protein skeleton (t s,3 ). The core of an amino acid consists of a sequence of a nitrogen atom followed by two carbon atoms. The protein skeleton is the sequence of these core atoms. Therefore, to transition from the licorice representation to the protein skeleton we need to remove those bonds that are not part of the core atoms in a structured and continuous fashion. For this purpose we define the rank of an atom as the minimum distance to the protein skeleton along the bonds. All atoms in the protein's skeleton are assigned rank 0, those immediately adjacent rank 1, etc., up to a maximum rank for the molecule. Since a bond connects two atoms, each endpoint has a naturally defined rank as well. Hence, we can also compute a rank for every point on the cylinder by linear interpolation. This rank allows us to specify the continuous transition from the entire molecule to only the backbone using a rank threshold t: only the part of every cylinder with rank ≤ t is shown. Again, we sought feedback from our collaborating chemists about this transition that continuously removes the bonds, with bonds distant from the skeleton being removed first. Also in this case the chemists confirmed that this animation does not introduce unwanted artifacts of structural or other nature.

The final discrete structural abstraction level supported by our approach is the ribbon diagram (t s,4 ) in which helices represent abstract spiral structures (α-helices) in the protein backbone. Thus, in order to achieve a continuous abstraction from the backbone we need to smoothly transition between a cylinder representation with sharp bends and smooth lines and ribbons. This transition comprises two aspects: to shift from the solid geometry of the cylinders in the backbone to the planar geometry of the lines and ribbons and to linearly interpolate between polylines along the cylinders in the backbone and the smoothed ribbon representation. An additional challenge that relates to both aspects is that the planar geometry of the lines and of the helices in the ribbon representation differs: the line normals are view-aligned, while the helix normals point toward the primary axis of the helix.

To realize both aspects of the transition we obtain information about the molecule's geometry and structure from the PDB [BWF * 00, BBB * 02]. We calculate the position of the smoothed ribbons by fitting cubic splines through the atoms of the protein skeleton. We achieve a smooth and interactive visualization by extending Tarini et al.'s [START_REF] Tarini | Ambient Occlusion and Edge Cueing to Enhance Real Time Molecular Visualization[END_REF] shader-based imposter technique. These impostors are viewdependent planes upon which the (shaded) shape of the depicted primitive is rendered (spheres for atoms and cylinders for bonds)-corresponding to how a viewer would see the real object. Depth-displacement is applied corresponding to the primitive's geometry to obtain a correct 3D visualization where cylinders are depicted as halfed pieces of a tube with open ends. Because these open ends are not acceptable in the abstraction stages without large spheres we augment Tarini et al.'s technique to include endcaps.

In an early stage during the transition from the backbone to ribbon visualization we change the backbone into smooth curves. We achieve this change by rendering the backbone using very small generalized cylinder patches which initially have a circular cross-section. During the transition from solid to planar geometry we change this cross-section into an ellipse whose major axis (representing the width of the bond) remains constant in length and which is oriented parallel to the viewing plane. Thus, we realize the transition of the impostors to view-aligned planes by shrinking the minor axis to zero. Because this is implemented using impostors we effectively only have to adjust the depth displacement to give rise to the planar patches. Later during the abstraction we also transition the line segments that represent spiral skeleton structures into helices. In addition to their spatial relocation and in contrast to non-helix lines we also smoothly interpolate the normals of the helix planar patches towards normals perpendicular to the helix direction.

We implemented this last part of the structural abstraction through a linear interpolation from the bond geometry to cubic splines fitted though the atoms' locations, and our collaborating computational chemists noted that this behavior matches their expectations of what such a transition would look like. Moreover, we do not use the impostor technique for helices [START_REF] Bajaj | TexMol: Interactive Visual Exploration of Large Flexible Multi-Component Molecular Complexes[END_REF], thus achieve a more correct ribbon representation due to using correct locations. In addition, we still maintain interactive rendering because the structural abstraction leads to an overall simplification of the geometry with respect to the space fill or balls-and-sticks levels.

Transitions of Visual Style

In addition to the seamless control of structural abstraction we also provide two means of changing the visual style. The reasons for providing such means lie not only in their use in traditional molecular illustration [START_REF] Richardson | Schematic Drawings of Protein Structures[END_REF][START_REF] Goodsell | Illustrating Molecules[END_REF][START_REF] Goodsell | Visual Methods from Atoms to Cells[END_REF] and recent visualization approaches [TCM06, Web09, OGF * 10] but is also motivated by the fact that visual abstraction supports effective communication of knowledge [START_REF] Rautek | Illustrative Visualization: New Technology or Useless Tautology[END_REF].

To enhance the spatial perception of molecules at various structural abstraction stages we employ ambient occlusion [START_REF] Tarini | Ambient Occlusion and Edge Cueing to Enhance Real Time Molecular Visualization[END_REF], haloing [ARS79, BG07], object attenuation, and change of perspective projection. While some of these techniques can be used in a similar manner for all structural abstraction stages, ambient occlusion can only be applied to space fill and balls-and-sticks because both have an inherent dense geometry, but cannot be used in the less dense, higher levels of structural abstraction. For the computation of ambient occlusion (AO) we follow Tarini et al.'s [START_REF] Tarini | Ambient Occlusion and Edge Cueing to Enhance Real Time Molecular Visualization[END_REF] approach. To deal with changing scenes, we precompute the AO for spheres at t s,0 and interpolate toward full illumination at t s,1 , while for cilinders we precompute halfway between t s,0 and t s,1 and interpolate first from no illumination then toward full illumination. In addition to AO we employ depthdependent halos [TCM06, EBRI09], either black or white when applied to 3D objects (relating to shadow or fog, resp.) or just in white for line-based representations. Finally, with dolly zoom (Vertigo effect) and object attenuation (change of object size depending on distance to the viewer) we provide two alternatives for enhancing depth perception from perspective, the latter often being employed in illustrative visualization (e. g., [START_REF] Everts | Depth-Dependent Halos: Illustrative Rendering of Dense Line Data[END_REF]).

To arrange all these effects along a continuous onedimensional axis we first apply AO simultaneously with dolly zoom / object attenuation and only afterward add halos (Fig. 3). The simultaneous use of AO and zoom / attenuation ensures that the control changes spatial perception at all levels of structural abstraction, even for levels where no AO is possible (e. g., Fig. 4). Moreover, the late halo application prevents the halos from obscuring attenuated objects.

A second means of changing the visual style of molecular visualizations is inspired by the varying styles and conventions in molecular illustration. One such established convention uses photorealistic shading (originating from physical plastic models) and assigns specific colors to the chemical elements and chains in the ribbon model. At the other end of the spectrum are purely black-and-white illustrations [START_REF] Goodsell | Illustrating Molecules[END_REF], and in-between the two extremes stylistic shading techniques are used [START_REF] Goodsell | Visual Methods from Atoms to Cells[END_REF]. Color conventions are also extended to structural abstractions by interpolating the atoms' colors along the connecting bonds or lines, while different colors depict the front-and back-sides of helices.

We provide a similar control along an 'illustrativeness' axis. As we move away from traditional (plastic) shading (Fig. 5(a)) we first transition to cel shading [Dec96, Leb96, LMHB00] and change the colors to their pastel equivalents (using palettes from ColorBrewer [HB03, Bre09]; Fig. 5(b)) to create a cartoon look. The darker rims that result from the cel shading, at this stage, begin to create what Goodsell calls "illustrative outlines in molecular structures" [START_REF] Goodsell | Visual Methods from Atoms to Cells[END_REF]. We create the black-and-white end of the spectrum (Fig. 5(c)) using slice-based hatching [Lei94, DHR * 99, SEI10] to mark the atom types. As in the hand-drawn examples [START_REF] Goodsell | Illustrating Molecules[END_REF] we hatch the helix back sides (following Weber [START_REF] Weber | ProteinShader: Illustrative Rendering of Macromolecules[END_REF]; see Fig. 4) and add silhouettes [IFH * 03] to spheres and ribbons.

Results and Feedback

Below we discuss a selection of example visualizations, mention rendering performance, and report informal feedback.

Visual Results and Discussion

Individual transitions along each of the axes of the abstraction space have been used to illustrate the concepts earlier in the paper. For example, Fig. 2 shows the effect of changing the structural abstraction while Fig. 3 and 4 demonstrate the support of depth perception and Fig. 5 depicts the effects of 'illustrativeness.' The use of different degrees of illustrativeness facilitates the selective emphasis or de-emphasis of selected parts of the structure. The desired amount of emphasis can be selected by continuously transitioning from traditional colored shading via a cartoon shading to a black-andwhite depiction. Fig. 6(a) demonstrates this focus+context effect in which the larger part of the molecule was deemphasized using an intermediate stage between the blackand-white and grayscale levels, while using ambient occlusion and halos support the spatial perception of the context shape. The chosen intermediate level of illustrativeness together with the use of halos for the context also has the effect that individual atoms merge visually so that the molecule's overall shape is more visible [START_REF] Goodsell | Visual Methods from Atoms to Cells[END_REF]. The focus is, instead, created using more structural abstraction (ribbon view) and less illustrativeness to add color. An example with less emphasis on the abstracted chains in focus is shown in Fig. 1.

Using the same molecule and the same view as in Fig. 6(a), Fig. 6(b) shows the application of three different degrees of structural abstraction, combined with different levels of illustrativeness. The background uses licorice with b/w, while in the foreground one sub-chain is shown using the balls-andsticks model which is contrasted with two sub-chains in ribbon style to both its sides. In this specific case this allows viewers to compare the representations with each other.

The visual emphasis resulting from the use of different degrees of illustrativeness is illustrated in Fig. 6(c). This effect can easily be used to guide a viewer's attention, in Fig. 6(c) to the part shown in color, also allowing people to compare different parts of a molecule. The intermediate stage of illustrativeness between black-and-white and grayscale, in this case, ensures that the atom type normally communicated using the well-established color scheme is not lost altogether (as it would be in a pure grayscale visualization) while at the same time avoiding a high-contrast black-and-white only impression (which would result from using the most extreme illustrativeness). An example with less visual emphasis to a specific part is shown in Fig. 6(d) where the only changes with respect to Fig. 6(c) are in the amount of illustrativeness assigned to the two parts.

Performance

To illustrate the performance we give rendering frame-rates on a 2.4 GHz Intel Core2Quad with 3GB memory and a NVIDIA GeForce 8800GTS 512MB card run under 64bit Linux in Table 1, for a range of molecule sizes. The frame rates are given for the molecules rendered at approximately 920 × 720 pixels for the two extremes, not abstracted at all (slowest performance overall) and full abstraction on all axes (requiring the rendering of less primitives).

Informal Feedback

We collected informal feedback from three computational chemists to whom we first demonstrated our implementation, pointing out the different abstraction axes and the stages they allow the visualization to go through. We also pointed out the fact that the transitions are continuous and that structural abstraction can be applied by protein sub-chain.

The experts pointed out that, while the individual structural abstraction levels can be generated with their current tools such as VMD [START_REF] Dalke | VMD -Visual Molecular Dynamics[END_REF], the seamless transition between the levels that we provide cannot be produced. Instead, tools such as VMD only permit the α-blending between different visualizations. The chemists see the advantages in our approach, therefore, in that it is possible to transition between adjacent levels without having to specifically define these levels first, and in the continuity of the transition without jumps-the latter specifically for teaching. They speculated that they may be able to observe aspects in intermediate stages that would not be observable in the discrete stages or blended versions. One expert commented that the structure was more apparent in the toon-shaded version than in the traditional shading, and that he likes object attenuation and dislikes perspective stretching in VMD. Comparing to PyMol, he also commented that it is very powerful but also requires much more training to achieve nice results.

The chemists also suggested numerous possibilities for extension. They were interested in the interactive selection of sub-strands and the individual control of their abstraction parameters. Also, it was important that very large molecular systems can be treated, including those ≥ 10 6 atoms such as large protein complexes. In addition, they would like to visualize molecular systems that also include solvents and other molecules, time-dependent data with varying time resolutions, and the possibility to show scripted animations.

Conclusions and Future Work

There are many illustrative visualizations that employ abstraction to some degree [START_REF] Rautek | Illustrative Visualization: New Technology or Useless Tautology[END_REF], and Rautek et al. are right to argue that abstraction is fundamental to illustrative visualization. However, many existing approaches either use some form of stylization (called "low-level visual abstractions" by Rautek et al.) which inherently abstract or apply means such as focus+context or distortion (called "high-level visual abstractions" by Rautek et al.). In contrast, we explore the interactive and dedicated application of abstraction in illustrative visualization along three different axes that all permit simultaneous and seamless control. This seamless abstraction space permits viewers to make mental connections between detailed and abstracted levels and, thus, to better understand these relationships and to interactively adjust the visualization to their current needs or specific data. Moreover, high-level visual abstractions are also possible with our approach (e. g., focus+context in Fig. 6(a)), but with a seamless control of the degree of the effect.

With our technique and its application to molecular visualization we have started to explore this dedicated control of abstraction, but there are more possibilities for realizing this control. There are a number of design decisions that we made that could have been made differently, and whose specific settings could be adjusted. Also, we concentrated on the visualization of biomolecules due to the availability of the data, while most of our techniques can also be applied to other types of molecules more generally. In that case we would need to limit ourselves to structural abstraction only from the space fill to a licorice visualization (skipping the backbone and ribbon levels) and may need to add other structural abstraction types depending on the domain.

There are a number of limitations in the current realization that need to be addressed in the future. In addition to increasing the supported size of molecules beyond about 10 5 atoms it would also be helpful to add abstraction for tertiary and quaternary structures or coarse-grained models [MRY * 07] as well as surface views and to include larger molecular systems consisting of several molecules and their interactions.

Figure 1 :

 1 Figure 1: Protein with selective structural abstraction, medium 'illustrativeness,' and support of spatial perception.

Figure 2 :

 2 Figure 2: Structural abstraction stages: (a) space fill, (b) balls-and-sticks, (c) licorice, (d) backbone, and (e) ribbon.

Figure 3 :Figure 4 :

 34 Figure 3: Stages of support of spatial perception: (b)-(c) ambient occlusion and object attenuation and (d)-(e) added halos.

Figure 5 :

 5 Figure 5: Abstraction along 'illustrativeness': (a) 'photorealistic,' (b) cel shading, and (c) black-and-white.

Figure 6 :

 6 Figure 6: A selection of results: in (a) a focus is placed on the secondary structure of a chain while showing the back part of the molecule as context, (b) shows the same molecule but now with three structural abstraction levels: balls-and-sticks, ribbon, and licorice; the visualizations in (c) and (d) demonstrate how illustrativeness can be used to guide attention.

Fig

  Fig. atoms bonds p bonds s FR 1 FR 2 2-5 796 152 647 328 1,151 1, 6(a, b) 12,605 4,645 8,300 93 341 n/a 58,674 24,024 35,063 23 79

Table 1 :

 1 Performance measures: bonds p & bonds s are primary and secondary bonds, FR 1 & FR 2 are average frame rates (fps) in non-and fully abstracted modes, respectively.

	.	atoms bonds p bonds s FR 1	FR 2
	2-5	796	152	647	328 1,151
	1, 6(a, b) 12,605	4,645	8,300	93	341
	n/a	58,674	24,024 35,063	23	79

© 2011 The Author(s) Journal compilation © 2011 The Eurographics Association and Blackwell Publishing Ltd.