Maria Roussou
email: m.roussou@cs.ucl.ac.uk

George Drettakis
email: george.drettakis@sophia.inria.fr

Can VR be Useful and Usable in Real-World Contexts? Observations from the Application and Evaluation of VR in Realistic Usage Conditions

This paper presents our observations from the use of high-end projection-based VR in different real-world settings, with practitioners but also novice users that do not normally use VR in their everyday practice. We developed two applications for two different content domains and present case studies of actual experiences with professionals and students who used these as part of their work or during their museum visit. Emphasis is given on usability issues and evaluation of effectiveness, as well as on our thoughts on the efficacy of the long term deployment of VR under realistic usage conditions, especially when the technology becomes mundane and the content takes precedence over the display medium. We will present an overall assessment of our experience, on issues relating to usability and user satisfaction with VR in real-world contexts.

Introduction

The majority of Virtual Reality (VR) applications developed today consists of research products that are either industrial prototypes created within very specific contexts or are used for presentation purposes. Despite the promise and the development activity of over two decades, there has been a considerable lack of real-world applications. The issues regarding the deployment of VR in everyday work contexts have been discussed many times and continue to revolve around the familiar practical difficulties: setting up special and costly hardware within facilities that are not easily transportable, requiring special teams of developers and maintenance staff, but also providing the high-level tools that will support users in their complex tasks (Neale, Cobb, & Wilson, 2002) and can succeed in establishing a collaborative VR work environment amongst individuals of different disciplines (Mackay & Fayard, 1997).

Experienced practitioners in the field of VR have indicated that to work effectively in a Virtual Environment (VE), the application content must include the ability to access or change environmental/system/meta parameters, create and manipulate particular objects, perform analyses, and export changes to permanent storage (Sowizral et al., 1995). While the current state of VE development has advanced its techniques to support these tasks, rarely does one find complete VEs that achieve both a high-quality photorealistic real-time environment and the level of interactivity required to carry out sufficiently complex real-world tasks.

Our goal was to create a VE infrastructure supporting both audio and visual realism and a high level of interactivity, and to situate and evaluate its utility in an appropriate real-world application context. We chose two different yet, in many ways, interrelated domains of application: the world of architectural design and urban planning (UP), where realism and interactivity are inherent requirements of the work process, and the domain of archaeological reconstruction, especially when this is used to educate on cultural heritage topics. A detailed user requirements analysis with the different end-users, that is architects, urban planners, archaeologists, educators, and children (Roussou, Sideris et al., 2004), confirmed the suitability of our choices and led to a detailed study of the existing workflow in these domains. A key element of this work has been our close collaboration throughout the entire project with the end-users of real-world projects in the two domains. The architectural and urban planning project involves the redesign of public spaces as part of the construction of a new Tramway in the city of Nice in France. The archaeological project involves the reconstruction of the ancient Doric temple of Messene in Greece.

Following our initial user needs analysis that guided our choices and design, we proceeded with the development of a complete VE for each case, which was continuously informed by the participation of the end-users of the real projects. Additionally, development and evaluation advanced together in order to determine the elements required to make the VEs useful in the context of the real-world project we were fortunate to connect to. The resulting VEs provide, in the one case, a design and brainstorming tool for architects and decision makers that also serves as a consensus building tool and as a presentation tool for the project; and in the other case, a multi-modal and engaging learning environment for archaeologists, educators and students.

In addition to the design approach, our work included in-depth formative and preliminary summative evaluation. Evaluation was performed based on observation, questionnaires and interviews with architects that took place both in a controlled lab experiment setting as well as in the ``field'', i.e. the actual work environment where the VE was used in decision-making meetings of the urban planning project. Similarly, observation, questionnaires and interviews were used with the archaeologists, educators and the children users of the cultural heritage production.

Related Work

Virtual reality development for architectural design and urban planning applications can be roughly grouped into two categories: applications that display detailed 3D CAD models of architectural spaces/structures and rapid prototyping systems. In the first case, the challenge has been to visualize large data sets in as photorealistic a fashion as possible. These environments are mostly used for presentation, recreation, and educational purposes (e.g. review of architecture before it is actually built, cultural heritage reconstructions, 3D entertainment rides, etc.) where complex 3D spaces are constructed so they can be explored in walk-throughs (Brooks, 1986); (Houston, Niederauer, Agrawala, & Humphreys, 2004). The majority of these projects allow for little interactivity beyond the user's ability to freely navigate about the environment.

On the other hand, the virtual prototyping environments allow immersive VR to be used in earlier phases of a design process and are thus designed to incorporate a higher level of interactivity and object manipulability. In most cases, these capabilities are implemented at the expense of visual realism, as they have been developed by computer scientists in order to further advance research in VR tools. Furthermore, most of these environments support only trivial user tasks and thus cannot be used in real-world situations yet. Nevertheless, many interesting ideas have been introduced by architectural prototyping projects that we can draw from. The CALVIN project (Leigh, Johnson, Vasilakis, & DeFanti, 1996), for example, introduced the idea of different perspectives, the mortal (ground-level) viewpoint and the deity (global above-ground) viewpoint, either of which users can assume to interact collaboratively in designing a space in VR. Other projects, such as ARTHUR (Broll et al., 2004) or BUILD-IT (Rautenberg et al., 1998), have developed novel interfaces to support and augment the collaborative workspace or to provide rapid sketching tools that facilitate the understanding of spatial features in urban design and consultation (Seichter, 2003).

The core idea from the beginning of our project has been to combine the strengths of the above two categories of virtual environments, namely to achieve the realistic visualization/auralization of a large space coupled with the ability to use it early on in the design process as an interactive work tool. Essentially, our goal is to create a virtual space that can complement and enrich people's day-to-day work practice in a useful and meaningful way, an application that would allow both design and review to take place in a VE. In order to achieve this we chose two ongoing real-world projects which formed the case studies described next.

Case Study #1: VR in a Real-World Urban Planning Context

The city of Nice and the Greater Nice-Cote d'Azur Urban Community (CANCA) recently decided to build a Tramway. The Mission Tramway project involves 8 km of rail in the densest parts of the city, requiring the redesign of several open spaces such as the main city squares, "Place Garibaldi" and "Place Massena". We established a working relationship with the officials and the company of architects in charge of the project.

The basic premise of our VE design approach has been to engage the users from the first steps of the design. Putting such an approach into practice requires collecting and analyzing as much information about our users as possible, through a detailed user requirements process and a deep understanding of how they work. Hence, a preliminary survey was first undertaken with architects, chief engineers and decision makers of the project, followed by a study of the traditional workflow employed (Drettakis et al., 2005). We then determined the elements required to make the VE useful in the real-world setting, choosing appropriate elements to develop a rich and audio visually realistic VE (Roussou, Drettakis, Tsingos, Reche, & Gallo, 2004). This preliminary work also guided the development of an appropriate interface and an evaluation methodology to test the overall usability of the system.

The result of our working relationship led to a closer collaboration with the architects on the re-design of "Place Garibaldi" (a central square in the city of Nice), and enabled us to gain access to all the project data. The architectural design of this square was of major importance to the city, since it is considered a historic landmark. As such, many stakeholders participated in the decision making process, including the mayor, the officials of the city council in charge of open spaces and public works, and other authorities at a national level who generally have a definitive say in any modification of a historical space. There is also a public hearing which occurred at the beginning and continued throughout the design process. Our collaboration was founded on the principle of mutual benefit. We were interested in studying and understanding the workflow to allow us to design novel VE tools that combine realism and interactivity, and to apply them in a real-world setting. The architects and decision makers of the Mission Tramway were interested in using the resulting interactive VE as an aid in decision making and brainstorming, as well as an impressive means for presentation of the project.

The VE that was finally developed included the ability to experience the reconstructed square from multiple different views (top-down, perspective or "balcony" view, and ground-view) and work within all these views. The user activities allowed included selecting, resizing, and positioning different types of trees, benches, umbrellas, and other elements, as well as different transport modeling scenarios that aided in visualizing the overall effect that the tram would have on the square. Finally, the visualization elements included image-based textures for facades, realistic vegetation, shadows, 3D sound, crowds, and, of course, the representation of the tramway and traffic (Drettakis, Roussou, Tsingos, Reche, & Gallo, 2004).

Case Study #2: VR in Support of Archaeological Research and Education

The second case study concerns the archaeological site of ancient Messene in Greece, specifically the study of the excavated Doric temple of the site. The temple is preserved in poor state, but there are a considerable number of architectural members found in the adjacent area, all of them well documented and interpreted by the Society for Messenian Studies, the archaeologists responsible for the excavation of the site. We worked with the archaeologists of the Society in order to identify their needs and also to identify the users who would benefit from a VE that will visualize the process of an archaeological reconstruction (Figure 1). As a result, restoration architects and archaeologists (especially archaeology students) were identified as the domain experts, who would use the VE as a tool for the exploration and validation of varied reconstruction hypotheses. To date, the tools available for this purpose are usually low-tech, low-accuracy models that cannot give the correct impression of scale and context. We also worked with the museum educators of the Foundation of the Hellenic World, a cultural heritage center in Athens with a CAVE-like display open to the public, who expressed the need for a similar VE that could be used in the context of a museum learning activity. In this case, adult and younger museum visitors, attracted by the interactive learning aspects of VR, would use a highly realistic interactive environment to learn more about history and archaeology.

The VE that was developed accurately represents a representative section of the temple, which the user must reconstruct. The user's activity in this VE resembles creative child's play with a construction kit: users of the environment must select the correct architectural members and position them appropriately. The process of virtually "building" parts of the temple provides the opportunity to actively experiment with different possibilities and solutions during the virtual reconstruction and to explore alternative scenarios. The environment also includes an instructional component, which provides novice users with information and the terminology used for each part of the reconstruction. Due to the tactile nature of the task, we decided to use a Haptic Interface, designed by PERCRO, as the main interface of interaction between the user and the VE.

Evaluation

Our evaluation methodology draws from the structured framework proposed by (Gabbard, Hix, & SwanII, 1999); (Bowman, Gabbard, & Hix, 2002) for the design and evaluation of user activity in VEs. This includes the combination of user needs analysis, user task scenarios, usability evaluation and formative evaluation, and preliminary summative evaluation.

The user needs analysis was carried out at the very beginning of the project and led to the definition of the user task scenarios that were used in the evaluation sessions. Usability evaluation forms a central tenet of our evaluation methodology, as it involves observing the users of the VE in order to determine if the VE aids or hinders them in reaching their intended goals. Given the nature of our project, we chose to limit our testing to a small number of users and follow an in-depth qualitative approach. One of the reasons for this is the obvious practical difficulty in evaluations within real-world situations, i.e., getting busy, highly qualified professionals to agree in participating in experimentation, which requires a significant investment in time. Other reasons include the highly experimental nature of the prototypes and the use of innovative and relatively inaccessible equipment (tracked immersive VR displays). We believe that for this project, case studies, where small groups of users are studied in depth, are more useful for gaining insights into the effectiveness and efficiency of our system, and come as a natural continuation to the design process that preceded the evaluation. The methods used in our evaluation included direct observation, a post-experiment questionnaire and postexperiment interviews.

Direct observation. Users performed the various tasks whilst being observed by a facilitator. Users were encouraged to use a think-aloud protocol (Ericsson & Simon, 1985) to explain what they are doing, to ask questions and to give information. The facilitator used an interactive style, asking users to expand upon comments and activities. Sessions were also videotaped for further analysis.

Questionnaire. A usability questionnaire was developed to identify the user's perception of the effectiveness and efficiency of the system and their level of satisfaction with the interaction. Our questionnaire was constructed by merging a number of standard user satisfaction questionnaires, such as the approaches provided by Perlman (Perlman, 2004) and others (Davis, 1989). The questionnaires include questions that require answers on a 1-7 Likert scale (Likert, 1967).

Interviews. An informal interview following the experience was used to help identify the various issues that occurred during the experience and that could not be captured by the questionnaire. The interview was particularly important for understanding the issues involved in the in situ usage of the system, where the use of a questionnaire does not make sense.

Evaluation of Case Study #1

Controlled Evaluation and Situated Use

The "Place Garibaldi" VE was evaluated both in the laboratory (controlled) and in the users' natural work environments (situated). Usability testing was conducted in the controlled setting of our laboratory with three of the collaborating professional architects, all directly implicated in the Nice Tramway project, and specifically in the design of the new Garibaldi square. Prior to the sessions with the architects, we ran pilot studies with engineers who had no previous VR or computer graphics experience. The main experiment was preceded by a simple training environment in order to allow the user to learn the interface. The experiment took place on a Barco Baron workbench. Each subject was head-tracked, wore active stereo glasses, and used a tracked game controller for interaction (Figure 2).

The users were asked to carry out a set of predefined tasks. The tasks used for the usability evaluation accurately represented the intended actual use of the application and occurred within a realistic scenario. The overall goal in using the VR tool, as defined by the users themselves from the beginning of the project, was to "create, define the appropriate size and position of design elements (different umbrellas and benches), and to evaluate the occupation of space and overall aesthetic effect on the new design of the square".

The main experiment VE included the entire environment of the new design of Place Garibaldi. The task was to place, size and arrange the umbrellas and stone benches in regions close to the orange trees. The user was asked to first position the elements, then determine the size (height and size) of the umbrellas and the placement of the benches, and finally to evaluate the "presence" and "occupation" of the square.

Each user was presented with the top-view at the outset, and was told to create and place umbrellas and benches, starting with the lower right corner. The user would insert a primitive (umbrella or bench), and typically position it in top-view. The users were reminded that they could switch freely between views. In a typical session, the user would place a set of umbrellas, either aligned or not, to populate the space. They would then switch to perspective view to correct or adjust placement, and to resize the width and height of the umbrellas. Balcony view was often used to judge the design.

In addition to the controlled lab experiments, the system has been used at several different occasions in the context of the real project. This was part of the agreement, in which the authorities provided us access to the architects and the data in exchange for their use of our system. We visited the authorities and the architectural offices at several occasions. One of these meetings concerned the discussion of a planned proposal for the choice of the type of trees to be used in the square. The choices included either the 3 meter-high orange trees or the 8-meter high oak. The working group was comprised of 3 high-ranking city officials, in charge of public spaces and urban planning, 4 city middle-level managers who were mainly architects concerned with the overall view of the project, 2 officials from the Mission Tramway, and the architect in charge of the overall project. The two scenarios (orange trees and oak trees) were mapped onto two different buttons on the handheld device. The members of the working group used the tool to explore the different views and locations provided for the virtual square and to make different choices. They chose one or another type of tree in different places in the square in order to test spatial relationships, the placement of objects, but also the different effect of shadows etc.

Another instance of field deployment involved the use of the full VE system (tracked stereo-vision workbench) for a brainstorming session. Two of the main architects and the designer of the project participated in this session and used the system as an opportunity to discuss issues concerning the design of the square, most notably the choice of trees and ground elements.

Observations

After completing the controlled lab experiment, each of the three architects was asked to complete the questionnaire and participated in the post-experiment interview that followed. We studied the videotaped sessions in order to better reflect on the important issues and difficulties that were expressed by the users during their interaction with the system. Similarly, we studied the data we had collected from our observation of the city officials and architects that used the VE as part of the in situ experience.

In terms of learnability and ease of use, all participants ranked the system as easy to learn (6 or 7 on the Likert scale) and stated that they were able to use the tool without difficulty. We were particularly pleased with this result, since the majority our subjects had no prior experience with interactive 3D systems or video games.

In terms of effectiveness and efficiency, there was a uniform approval (6 or 7 on the Likert scale) of the utility of the tool and the fact that the system would improve productivity in the workplace. The top view, although familiar to the users through the use of conventional paper renderings, was judged moderately useful in its VR version (3-5 on the Likert scale). From observing the videos and the interviews it became clear that the precision of manipulation when using this view was insufficient. In retrospect this was to be expected, since the distance from the object being manipulated is too large. Clearly, the best solution to this is a mixed 2D-3D interface, where a "pen-like" interface could be used to directly place objects onto the top-view, as is currently done in existing CAD tools. In fact, the need to have the same interface as CAD tools for these tasks was explicitly mentioned by one user who also asked to have menus present in the other views (balcony view and ground view). On the other hand, another user stated that one of the main values of the system was that it removes the "break" which exists between traditional 3D CAD systems and the resulting design. Additionally, one participant stated that the perception of ambience and scale were extremely useful and important for an architect in the evaluation of an urban planning project, and found that the tool had great potential for brainstorming and interactively trying out different alternatives.

In terms of VE interface features, the "balcony" view was used extensively and was greatly appreciated by all participants. All users agreed that this was a particularly useful view of the environment, and that it helped in their judgment of the resulting design, but also during the design. This is attributed to the fact that the VE environment followed the real design process, since the choice and successful design of these views was based on our observation and understanding of how the users actually work, and their continuous input throughout the design-developmentevaluation cycles. Finally, in terms of satisfaction, all users stated both in the questionnaire and in the interviews that they liked the tool.

Evaluation of Case Study #2

Sessions with Content Experts and Museum Visitors

We tested the archaeological activity environment with expert and novice users in the context of a museum, with three different categories of users: adult novice users (museum visitors), young novice users (museum visitors between 9 and 14), and adult domain experts (archaeologists and educators). All studies took place in the Foundation of the Hellenic World's cubic immersive (CAVE®-like) display, during or after normal museum hours. All novice users (adults and children) were family visitors who spent their day at the museum.

Overall, we ran complete sessions with a total of 14 adults, 7 of which were novice users and 7 content domain experts (Figure 3), and with 7 children between 9 and 14 years of age (Figure 4). In addition, we collected opinion questionnaires concerning the haptic interface from 25 more museum visitors after their experience with the environment, particularly the use of the haptic interface.

The instruments used were questionnaires and informal interviews. A usability and presence questionnaire was used after the experience for all users. Additionally, for the non-expert users, a pre-test questionnaire was used to test prior knowledge and then a similar post-test questionnaire to see if there was a change in their knowledge, as a result of the virtual experience. We also collected general visitor opinions about the Haptic Interface with visitors of the museum who used it during normal museum hours.

Observations

The evaluation of the virtual reconstruction case study with the content experts (archaeologists and educators) involved primarily the usability of the system and its potential as an educational work tool. The archaeologists we worked with and the majority of the archaeologists we evaluated the VE with, were very positive about the environment and its potential in educating restoration trainees, mostly because of its ability to present the content in a photorealistic and accurate manner and, most importantly, in the correct natural dimensions. However, most users pointed out that in order for the environment to be used in a real-world workspace (provided that all other practical issues were resolved), the representation of much more detail would be required, as well as the ability to simulate specific restoration techniques, such as filling in missing parts with plaster or treating aging. Many comments concerning the potential of the haptic interface and suggestions for improvement were also collected.

The evaluation of the case study with novice users aimed at determining whether interaction within the VE helps the user to gain a better sense of the process of archaeological research and learn about the positions, dimensions, and interrelationships between architectural members. The focus of the investigation has been on the potential for cognitive change, involving the measurement of the interaction effects on the user's understanding of the somewhat abstract concepts eluded by the task. However, an important aspect which cannot be separated from the evaluation of learning, especially when working with children, is the measure of affect (fun, engagement), as well as the potential pedagogical value of the system. Within this evaluation framework (cognitive, affective, pedagogical), we also looked at usability issues, involving mostly the learnability and ease of use of the system. The analysis of these studies is currently underway.

Conclusions

The above are preliminary observations derived from different types of experimental sessions and focus groups, with unavoidably small user sets. Although preliminary, with thorough analysis not yet completed, we consider these to be rich results because they provide insights and involve an in-depth observation of how actual non-IT expert users may be able to use Virtual Reality as a central tool in their work processes (for example, the in situ use of a VE in the decision making process of a real project).

Conclusions at this point can only be general, the main one being that by using a user-centered design approach and a focused evaluation process we were able to tailor the development of the VEs to the real needs of our end-users and to increase the validity of our environments as practical yet rich work tools. In terms of usability "engineering", the active involvement of the users in our projects confirmed once again that an informed human-computer interaction approach to application development is central in ensuring that the VE will meet the practical need for useful, usable, and successful multi-modal systems.

Many problems remain, of course. Firstly, these case studies are still far from proving that VR can be used in a realworld context with non-expert users on a long-term basis. To date, there are examples of VR practice in industries, such as the automotive or oil and gas industry, where immersive systems are used in the workplace. However, these workplaces still need to employ special laboratories and scientists in order to support the use of VR. When we talk about VR systems used in the workplace, in an educational setting, in a leisure-based context, or for any other kind of widespread public use, we envision use that resembles (in terms of its simplicity and straightforwardness) that of a home or office PC. For this to happen, the practical difficulties of VE development for real applications cited in the introduction, as well as issues of cost, distribution, space, and maintenance still hold and must be resolved. Secondly, our involvement in a real-world project clearly demonstrated the need for significant acceleration and streamlining of the capture process for real-world content (buildings etc.). Also, the frequently changing directions and needs in such projects underlined the need for better authoring tools for the creation of interactive scenarios. Both of these improvements would allow a faster turnaround time, enabling an improved response of the development team to the users and thus better user involvement in the process. Thirdly, in the cases described, there are still particular interface details that must be added, modified, and improved in order to make the environments more usable, easy to learn, and efficient. Although none of these interface improvements are particularly hard to resolve, they do require further software development which means specialized scientists and artists who will develop specifically for each case. VR hardware, software, and authoring tools have evolved greatly thus making the process of developing a VE much simpler than before. However, this still requires special programming skills, experience, time and effort, all of which can not be predicted and depend on the complexity of each application and case.

Overall, although the approach we followed of engaging users in the design is time and resource intensive, we consider it to be worthwhile and will continue to explore the application of HCI models into the design of VEs. We believe that this approach provides a flexible way to deal with the variety of situations presented within real world projects and, thus, provides promise for the development of environments and meaningful tools that can be of real value to their users and can advance the widespread use of VR.

Figure 1 :

 1 Figure 1: Working with archaeologists to define the virtual reconstruction task (left). Cross section of a temple, indicating the different members that the user manipulates in the VE (right).

Figure 2 :

 2 Figure 2: View of the urban planning VE on a curved presentation display (left) and an architect using it on an immersive workbench (right).

Figure 3 :

 3 Figure 3: Archaeologists and educators participate in the evaluation of the virtual reconstruction tool, using a haptic Interface device designed by PERCRO (http://www.percro.org/).

Figure 4 :

 4 Figure 4: Young users, between 9 and 14 years of age, use the Haptic Interface to interact with a virtual reconstruction environment during their museum visit.

Acknowledgments

We thank all the end-users who worked with us, providing access to data and to their work processes, and for taking part in the studies: the CANCA, the City of Nice and the Mission Tramway, the Society for the Study of Ancient Messene, and the archaeologists, museum educators, and visitors who participated in the studies at the Foundation of the Hellenic World. We are indebted to the AURA architects Marc Dalibard, Andre Jollivert and Emilie Hélardot, who generously provided their valuable time and effort for the success of this project. Please note that the images presented in this work do not engage any of the authorities in any way, and that the decision process for the design of the Place in the urban planning project is still far from complete. This work was performed as part of CREATE, a 3-year RTD project funded by the 5th Framework Information Society Technologies (IST) Programme of the European Union (IST-2001-34231), http://www.cs.ucl.ac.uk/create/. Special thanks are due to all the technical staff and students of the partner organizations, who developed the systems and helped in the evaluation procedures.

Finally, special thanks are due to Asimina Vasalou for her invaluable help in presenting this work.

Bowman, D., Gabbard, J. L., & Hix, D. (2002). A Survey of Usability Evaluation in Virtual Environments:

Classification and Comparison of Methods. PRESENCE: Teleoperators and Virtual Environments, 11(4), MIT Press, 404-424. Broll, W., Lindt, I., Ohlenburg, J., Wittkamper, M., Yuan, C., Novotny, T., et al. (2004). ARTHUR: A Collaborative Augmented Environment for Architectural Design and Urban Planning. Journal of Virtual Reality and Broadcasting, 1(1), 1-10. Brooks, F. P. J. (1986). Walkthrough: A dynamic graphics system for simulating virtual buildings. In Proc.

Workshop on Interactive 3D Graphics, 9-21. Davis, F. D. (1989). Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology.

MIS Quarterly, 13(3), 318-340. Drettakis, G., Roussou, M., Asselot, M., Reche, A., Olivier-Mangon, A., Tsingos, N., et al. (2005). Participatory Design and Evaluation of a Real-World Virtual Environment for Architecture and Urban Planning (INRIA Research Report No. 5479). Sophia-Antipolis: INRIA. Drettakis, G., Roussou, M., Tsingos, N., Reche, A., & Gallo, E. (2004). Image-based Techniques for the Creation and Display of Photorealistic Interactive Virtual Environments. In Proc. Eurographics Symposium on Virtual Environments, Grenoble, France, ACM. Ericsson, K. A., & Simon, H. A. (1985). Protocol Analysis: Verbal Reports as Data. Cambridge, MA.: MIT Press. Gabbard, J. L., Hix, D., & SwanII, J. E. (1999). User-Centered Design and Evaluation of Virtual Environments. IEEE Computer Graphics and Applications, 51-59. Houston, M., Niederauer, C., Agrawala, M., & Humphreys, G. (2004, August). Visualizing Dynamic Architectural Environments. Communications of the ACM, 47(8), 55-59. Leigh, J., Johnson, A. E., Vasilakis, C. A., & DeFanti, T. A. (1996). Multi-perspective Collaborative Design in Persistent Networked Virtual Environments. In Proc. VRAIS, IEEE. Likert, R. (1967). The human organization: its management and value. New York: McGraw-Hill. Mackay, W. E., & Fayard, A.-L. (1997). HCI, Natural Science and Design: A Framework for Triangulation Across Disciplines. In Proc. Designing Interactive Systems (DIS), Amsterdam, ACM Press, 223-234. Neale, H., Cobb, S., & Wilson, J. R. (2002). A Front-Ended Approach to the User-Centred Design of VEs. In Proc. IEEE Virtual Reality 2002, Orlando, Florida, IEEE Computer Society, 199-206. Perlman, G. (2004). Web-Based User Interface Evaluation with Questionnaires. Retrieved May 25, 2004, from http://www.acm.org/%7Eperlman/question.html Rautenberg, M., Bichsel, M., Fjeld, M., Leonhardt, U., Krueger, H., & Meier, M. (1998). BUILD-IT: A Planning Tool for Construction and Design. In Proc. ACM SIGCHI 1998 (CHI '98: Human Factors in Computing Systems), ACM Press, 177-178. Roussou, M., Drettakis, G., Tsingos, N., Reche, A., & Gallo, E. (2004). A User-Centered Approach on Combining Realism and Interactivity in Virtual Environments. In Proc. IEEE Virtual Reality 2004, Chicago, IL, IEEE, 251-252. Roussou, M., Sideris, A., Loscos, C., Dettori, A., Drettakis, G., Lombardo, J.-C., et al. (2004). Requirements Analysis on Cultural Heritage -Education and Urban -Architectural Planning and Design Case Studies (Technical Report No. RN/04/09). London, UK: University College London. Seichter, H. (2003). sketchand+. In Proc. CAADRIA, Bangkok, Thailand, 209-219. Sowizral, H., Angus, I. G., Bryson, S., Haas, S., Mine, M. R., & Pausch, R. (1995). Panel session on Performing Work within Virtual Environments. In Proc. 22nd International Conference on Computer Graphics and Interactive Techniques, ACM Press, 497-498.


Carsten Dachsbacher

Tobias Bolch

Marc Stamminger

Procedural Reproduction of Terrain Textures with Geographic Data

Surface textures of high resolution and quality, either acquired from aerial or satellite imagery or computed using procedural models, are crucial for photorealistic terrain rendering. Procedural models provide a compact representation and can be evaluated at run-time. In this paper we present an extension to an existing, GPU-friendly procedural texturing model, such that it can be fitted semiautomatically to real-world data. In order to increase realism and to account for geographic conditions, we also include temperature, solar radiation and rainfall distributions -simulated or modeled using measured data from gaging stations -into the reproduction process. The original surface texture is no longer required for rendering: instead a new texture of arbitrary resolution is synthesized at runtime. In addition to the compact procedural model we store elevation data, anyway required for the terrain rendering, and low-resolution geographic data. We show results of our method applied to a comparatively little cultivated region in Central Asia.

Introduction

Surface textures of high resolution and quality are crucial for photorealistic terrain rendering. Basically they can originate from two sources: either they are acquired as real-world data, e.g. from satellite imagery, or they are created from the input elevation data using appropriate procedural models. In both cases the memory consumption of the acquired or precomputed textures can be tremendous if great viewing distances or high surface detail for close views is provided and therefore adequate implementation strategies have to be applied to achieve interactive rendering.

Procedural models for terrain texturing can be used to create completely artificial, yet realistic landscapes. In this paper we propose means to mimic real terrain textures (left image in the teaser) by guiding a procedural model with real-world data. We incorporate further geographic input data in addition to elevation data, namely rainfall, solar radiation, and temperature (center images in the teaser). The terrain surfaces visible in a georegistered satellite image are classified and the geographic preconditions for their appearance are estimated. This information can be used to reproduce the input texture with the advantage of a compact procedural description (right image in the teaser). Using a model that can be evaluated at arbitrary resolutions, we can also render close-ups of the terrain by adding procedural detail.

This paper is structured as follows: in Section 2 we give an overview over related and previous work and Section 3 gives an introduction to geographic considerations for realistic terrain texturing. Section 4 recapitulates the basics of the procedural model which serves as a starting point for this work. Section 5 describes our texture reproduction by guidance of the procedural model followed by results and conclusions in Section 6.

Previous Work

Diverse caching and compression strategies have been developed to deal with huge amounts of precomputed or acquired texture data. Two approaches with hardware support are clipmaps [SGI] (on SGI's InfiniteReality) and S3 texture compression (see e.g. [START_REF]MICROSOFT CORPORATION: DirectX 9.0 SDK[END_REF]) supported by all contemporary graphics hardware. Other solutions to this problem are solved on the software side using adequate caching or paging strategies: typically a large terrain is rep-resented by a quadtree-like data structure and the required texture (resolution) for each visible node is determined and the appropriate texture is loaded or generated. Examples for this technique have been presented among others by Blow [START_REF] Blow | Implementing a Texture Caching System[END_REF][START_REF] Blow | Terrain Rendering at High Levels of Detail[END_REF] and Ulrich [START_REF]Super-size it! Scaling up to Massive Virtual Worlds' course at SIGGRAPH[END_REF].

However, in various settings, it is not possible to use acquired real-world data for terrain rendering. It may be that the texture resolution is too low for close-up views or that the images include human housing or agriculture although undisturbed, natural scenes are to be rendered. Apart from that, it is often the case that storage or transfer of the textures is not desirable -then, procedural models can be used to compute textures.

Many texturing models, mainly those often used in computer games and other interactive applications, are computationally rather cheap but lack realistic appearance. For example procedurally precomputed low-resolution texture maps combined with detail or bump maps are not sufficient for a plausible appearance of soil, especially due to the visible repetitions. Although Wang tiles [START_REF] Cohen | Wang tiles for image and texture generation[END_REF] can solve this problem they require a set of precomputed texture tiles and cannot account for varying terrain features, like steep slopes and plains. Precomputing procedural textures at high resolution requires paging and caching algorithms, but even those have their limitations. Widely used approaches in computer games are texture splatting, that is compositing of a terrain texture by blending a set of tileable textures (e.g. for rock, sand and grass surfaces) in the frame-buffer (see Bloom [START_REF] Bloom | Terrain Texture Compositing by Blending in the Frame-Buffer[END_REF] for details) or texture bombing (see [START_REF] Glanville | Texture Bombing[END_REF]). A procedural model which can be computed efficiently at arbitrary resolutions using programmable graphics hardware and which accounts for terrain features was introduced by Dachsbacher et al. [START_REF] Dachsbacher | Cached Procedural Textures for Terrain Rendering[END_REF]. It is comparable to the model used in Terragen [START_REF] Fairclough | Terragen v0.9 (Software)[END_REF] in terms of visual quality and flexibility, but can be evaluated at run-time at interactive frame rates.

For reproducing textures, the input data has to be interpreted. Premoze et al. [START_REF] Premoze | Geospecific Rendering of Alpine Terrain[END_REF] propose methods for classifying surface types and for removing shading and shadowing artifacts from aerial imagery. Using this information, new views of the terrain for arbitrary times of day and atmospheric conditions can be rendered.

Much research has been spent on terrain rendering, level-of-detail algorithms and textur-ing.

As these topics are not closely related to this work, we direct the interested reader to http://www.vterrain.org which is a valuable source of links and information.

Geographic Considerations

Apart from efficient computation, the quality of a procedural model depends mainly on the realism it achieves. Those procedural models taking terrain features into account so far only consider terrain elevation and slope (e.g. the Terragen [START_REF] Fairclough | Terragen v0.9 (Software)[END_REF] model). We propose to include additional, geographic input data, e.g. temperature, solar radiation and rainfall, into the procedural model. This enables us to steer a procedural model such that it reproduces a given input texture. This is not possible when regarding only input elevation data, as terrain surfaces and particularly vegetation also depends on other factors. The geographic quantities can either be acquired from appropriate gaging stations or computed by simulations.

In this work we concentrate on generating textures only, which implies a certain viewing distance. For close views, vegetation and small scale detail, e.g. rocks, have to be represented geometrically. Deussen et al. [START_REF] Deussen | Interactive visualization of complex plant ecosystems[END_REF] and Colditz et al. [START_REF] Colditz | Real-time Rendering of Complex Photorealistic Landscapes Using Hybrid Level-of-Detail Approaches[END_REF] propose methods to handle the complexity of such ecosystems for rendering. To account for additional input data for texturing, it is important to know how it influences the appearance of surface types. We refer the interested reader to physical geography and biogeography textbooks (e.g. [START_REF] Mcknight | Physical Geography: A Landscape Appreciation, 8th edition[END_REF][START_REF] Pidwirny | Fundamentals of Physical Geography -Online Textbook[END_REF]) and present a collected summary of aspects that proved to be decisive.

Erosion simulation can be used to determine where solid rock and where loose sediments exist. However, erosion itself is a quite complicated process, if vegetation and non-uniform material is considered. The basis of erosion and rockfall is weathering or decomposition, e.g. due to frost weathering, chemical decomposition or plant roots. Rockfalls are common for high mountains and exhibit brighter tear-off edges, due to non-weathered stone. Besides from terrain slopes, soil properties and rainfall, erosion also depends on vegetation, as plants prevent from soil loss. Vice versa, the plant distribution also depends on surface materials and thus on the result of erosion.

The appearance of plants in general depends on few parameters: they require sufficient water sup- ply, an adequate soil to strike roots and a certain amount of solar radiation for photosynthesis. On the other hand the solar radiation is an important steering factor for the evaporation; thus it is also a factor that limits the growing of plants. The sunshine duration can be computed for a given terrain and location on the earth, provided that an average cloud cover can be assumed. Computing the irradiation for a surface location from the solar constant (the amount of energy received at the top of the earth's atmosphere, 1368W/m 2 ) is explained by Stickler [START_REF] Stickler | Solar Radiation and the Earth System[END_REF]. Concerning the availability of water, it is important to note that it does not depend solely on direct rainfall, but also on subterranean streams that are hard to simulate. The rainfall distribution depends mainly on large-scale terrain features. The precipitation normally rises with the elevation due to air cooling and therefore high mountains receive more rainfall than surrounding planes. In addition, an important aspect is the rainfall 'shadowing' from mountains in the main wind direction. The occurrence of perennial snow and ice also depends on certain climatic conditions. Procedural models for height field generation and real world elevation data usually do not exhibit soil type information. For this, the distribution of soil materials is usually computed within the procedural texturing model assuming that e.g. rocky surfaces appear at steep slopes. The Cached Procedural Texturing (CPT) model [START_REF] Dachsbacher | Cached Procedural Textures for Terrain Rendering[END_REF] is constructed such that one surface type is the precondition for the existence of another: for example, a surface type representing fertile sediments is the precondition for grass and other plants to appear. So far, however, only altitude and slope criteria have been considered for the distribution of soil materials.

This paper describes how to extend and fit the CPT model semi-automatically to real-world data, e.g. acquired satellite images. By this, the original surface texture is not required for the rendering and is replaced by a very compact procedural approximation which can be evaluated directly at run-time using stored low-resolution geographic data.

Cached Procedural Textures

As our reproduction of natural terrain textures is an extension to the CTP method, we recapitulate the basic ideas in this section. This technique generates surface textures from input elevation data while the surface appearance is described by hierarchically ordered surface layers. A surface layer represents a soil or vegetation layer and can contribute to the final texture color at a certain location when constraints for appearance (e.g. a certain range of elevation) depending on user-defined layer parameters and input elevation data are met. Originally, this method was designed to compute artificial terrain textures with few, intuitive parameters.

The method can be best explained, when assuming that we compute a texture for the whole terrain with a 1:1 texture-to-heightmap texel ratio. At first, the terrain texture is initialized with a base color representing the underlying rock material. Then one surface layer at a time is processed: for each texel of the terrain height map the layer's constraints (see Section 4.1 and 4.2) are evaluated. The surface layer only contributes to the terrain color and may change the terrain surface (modifying its elevation data), if its constraints are met. If one of the constraints is unmet, the layer is discarded at this texel. The next surface layer operates on the output color and elevation data of the previous step. After applying all layers, the final height is used to compute the terrain's surface normals for lighting computations and for displacement mapping. This technique thus produces modified elevation data through displacements, a corresponding normal map, and a color texture that represents the final terrain color.

In order to provide level-of-detail rendering a quadtree data structure is used for spatial subdivision of the terrain domain. For geometric level-ofdetail a Chunk-LOD approach [Ulr02] is used. As terrain closer to the viewer requires higher texture resolutions than terrain further away, the algorithm operates on texture tiles (small textures associated a quadtree node) stored in a texture atlas [START_REF] Wloka | Improved Batching via Texture Atlases[END_REF].

The generated surface textures are cached in the atlas to exploit frame-to-frame coherency. Tiles may become invalid as the current view changes or as the current view requires them to be of different resolution. With caching, only a few or even no texture updates are required for rendering a new frame.

Surface Layers and Attributes

Whether a surface layer contributes to a surface location or texel x of the terrain texture depends on the evaluation of its constraints. A layer appears only in a certain range of height and slope values and the amount of its contribution to the texture color is called coverage C(x) ∈ [0; 1]. For efficient evaluation these coverage functions are computed from simple, clamped hat functions (provide continuous transitions) hA(x) and hS(x) for height and slope constraints (see Fig. 2). Please note, that hA and hS are functions of scalar input and output -however, we will apply them to input data retrieved from the elevation data H(x) and define h ′ A (x) = hA(H(x)) and analogous for h ′ S . In addition to these hat functions the coverage value is slightly modified by a noise function to diversify the terrain texture which may be of much than the input height field. Each surface layer stores parameters for evaluating the hat and noise functions, and a RGB color value representing the surface material or vegetation (see Table 1). These parameters are chosen by the user and allow an intuitive modeling of surface texturing.

Evaluation

The hierarchical structure of surface layers, also defined by the user, specifies the order of evaluation of the surface layers (Fig. 4). Layers are evaluated in depth-first order, and layers whose parents fail to satisfy their constraints cannot contribute. A layer cannot contribute more than its parent layer and to achieve this, the coverage C(x) of a layer is bounded by its parent's coverage CP (x). Please note, that coverage is only propagated to a layer's children, the modified elevation data is always taken as input for the next layer (Fig. 4). The root node is special in that it has no conthe base material of the terrain is present everywhere. The input to the root node is the original terrain elevation data and as the target resolution of the terrain texture is usually higher, the elevation data is upsampled by cubic B-spline interpolation. Simpler bilinear interpolation leads to piecelinear patches and causes undesirable texturing and shading artifacts.

The constraint hat functions can be efficiently computed by the clamped minimum of two linear functions. As a consequence the native instructions of a GPU's pixel shader allow a very simple implementation and a parallelized evaluation for multiple constraints. In order to provide the option of making the surface layer distribution more diversified, the noise function N (x) with a codomain of [-1; 1] is scaled, biased and used to modify constraint results. The coverage C(x) is then computed from all constraint functions (denoted by Fi here, F0 = h ′ A , F1 = h ′ S and n being the number of constraints) and the parent's coverage:

C(x) =clamp ( [bs + ssN (x)] n-1 Y i=0 Fi(x) ) CP (x) with clamp(x) = max {0, min {1, x}} (1)
Finally, the new color due to application of this layer and the new height value H ′ (x), modified by a displacement map DS(x) (representing surface layer characteristics like bumps or cracks) is determined:

RGB(x) = (1 -C(x))RGBin(x) + C(x)RGBS H ′ (x) = H(x) + C(x)BSDS(x) (2)

Mapping the Real World

In this section we describe a method to estimate surface layer constraints and properties for the procedural texturing method from real-world data, i.e. satellite color images, elevation and further morphometric and climatic data. For this, we need registered input data as shown in Fig. 3. Once a surface layer description is constructed from realworld data, we can render arbitrary views of the terrain or images from other terrain parts for which elevation and (simulated) geographic data exists.

Acquisition of Input Data

First of all, we describe how the input data in our example has been acquired. Computations may vary in detail as specialized models exist for simulating climatic conditions in different geographic regions; the utilization of the resulting data for our texturing method, however, always remains the same.

To regionalize the mean annual air temperature and the precipitation (Fig. 3) in a first step we dithe study area into two parts for temperature and five parts for precipitation, in order to obtain homogeneous areas with similar climatic conditions, e.g. the foothills and the northern slopes the first mountain ridge and the intramountainous basin of lake Issyk-Kul. In a second step, we computed vertical gradients for each area based on a regression model using the existing climate stations in order to get specific values about the temperature decrease and precipitation increase with the altitude. For the Issyk-Kul basin, a significant precipitation increase from west to east occurs, because the air masses obtain humidity while flowing over the large water body. Therefore here a horizontal gradient was also calculated. Then the temperature and precipitation conditions were regionalized using the digital elevation model from the Shuttle Radar Topography Mission (SRTM, resolution 90 m) for each area separately and finally merged using ArcGIS 9.0. The results for the temperature were promising, but not entirely so for the precipitation. In order to integrate the important luv and lee effects of the precipitation, this model was combined with [START_REF] Öhner | Environmental Change Modelling for Central and High Asia: Pleistocene, Present and Future Scenarios[END_REF] for High Asia, which includes these effects, based on the calculated main wind direction using a circulation model. However this model has a lower resolution and integrates only three climate stations in northern Tien Shan and Issyk-Kul basin. The radiation has a high impact on the surface conditions, e.g. the vegetation belts and the snow line are shifted to higher altitudes than on the northern slopes due to the dryness on the southern slopes caused by high radiation input. Therefore the potential direct solar radiation (the direct radiation which the earth surface would receive in cloudy conditions) was modeled, in order to obtain its differentiation. This model has an iterative approach taking the astronomic and atmospheric standard parameters (such as the geographic position or the atmospheric extinction) into account. For more information we refer the interested reader to [BL05, TH04, Bol06].

Acquiring Surface Layer Descriptions

Several processing steps are involved to acquire the surface layer description. These are presented in chronological order in the following sections. To capture the complexity of nature, we do not represent the surface layer constraints by simple hat functions, but by arbitrary constraint functions. These functions are acquired by sampling the realworld data and stored for a later evaluation. Our texturing parameter estimation has to perform the following steps:

• Identification and classification of surface materials visible in the satellite image. • Evaluation under which conditions a class of surface material appears, i.e. for example, which elevation, slope, and temperature have to predominate at a certain location that the terrain is covered by snow and ice. • Replace previously simple hat functions of the procedural model by sampled constraint functions obtained from the evaluation step.

Surface Type Classification

Although multi-spectral satellite images provide much more information than just RGB color, e.g. infrared channels (in fact often the RGB representations are computed from other frequency bands) we restrict our experiments to this limited data. By this, we can also use data from aerial photography which most often does not offer additional data channels. First, the surface types in the RGB color satellite image are to be classified and each surface type, e.g. rock, sand or grass, will initially be represented by a single surface layer. Premoze et al. [START_REF] Premoze | Geospecific Rendering of Alpine Terrain[END_REF] present a method for removing shading from aerial orthoimages and classifying surface cover considering surface orientation and application of a maximum likelihood Bayer classifier. However, we found that for our purpose and for rendering distant views we can achieve a satisfying classification more easily. The underlying idea is to transform the color information from RGB into color spaces, that allow an easy, yet sufficient, classification of surfaces. Experiments showed that color models providing luminance and two chromaticity values, like the CIE XYZ, YIQ or YUV models, are not practical, as the color separation is not sufficient. On the other hand, the hue-saturation-value (HSV) model in combination with the original RGB representation proved to work very well. Figure 5 shows the satellite image converted into HSV color space and Figure 7 shows the respective histogram of hue and saturation values as a density plot. Please note, that colors, when converted from RGB to HSV, may end up in undefined hue values due to color quantization. For this, (near) gray colors are identified in RGB space and their hue value is set to a value (here it is set to the hue of purple colors that do not originally appear in the satellite image). The resulting hue-saturation histogram is well suitable to classify the terrain surface into rock, vegetation, water and snow, and ignores shading due to surface orientation. In our experiments we defined the surface clusters manually by dividing the histogram into four subsets. This step does not require geographic knowledge -the clusters are simply chosen according to the hue-saturation histogram. The application of automatic clustering methods, e.g. learning vector quantization or selforganizing maps (see Kohonen [START_REF] Kohonen | Self-Organizing Maps[END_REF] for details), remains subject to future research.

Surface Layer Hierarchy

Building a meaningful surface layer hierarchy automatically from the input data cannot be easily accomplished. This is because the input satellite image, does not provide information on the constitution of the terrain surface, from which a meaningful surface layer hierarchy can be derived -only color values are provided. The evaluation of the layer description, however, also depends on the topology of the layer hierarchy.

Considering these problems, we suggest to use a simple heuristic which imposes certain requirements on the constraint functions described in the next section. Initially we start with a root node and assign the color of the most widespread surface type to it. Then we create a level-1 node for each surface type classified in the previous step. All of these sibling nodes are children of the root node and we assign the average color of each surface type class to the nodes. Parameters for the texture diversification for close views through noise functions remain user-defined. Please note, that colors taken from the satellite image underlie atmospheric scattering and thus colors exhibit a shift to blueish color tones.

Estimating Distribution Functions

The last processing step is the determination of conditions which have to predominate that a certain type appears in the texture. In Section 4 these conditions were user-defined by few intuitive quantities and simple hat functions have been used for evaluation. In real-world, however, such simple functions will not suffice to capture the complex distribution of surface materials. Our goal is to acquire constraint functions from the input data automatically which allow us to reproduce the input satellite image. We compute these functions Fi(x) ∈ [0; 1) (analogous to Eq. 1) for each input quantity (height, slope, rainfall, temperature and radiation) to obtain P (x) = Q n-1 i=0 Fi(x). This product of constraint functions P (x) is the main criteria for the appearance of a surface layer and can be considered as amount of contribution to the texture color. However, surface layers are evaluated sequentially (due to the simplified layer hierarchy, see Section 5.2.2) and one surface layer may overwrite the contribution of its predecessor.

Consequently, the problem is to acquire Fi(x) for each surface layer from the respective input quantities such that P (x) approaches 1 if, and only if, the layer is visible at surface location x. Under the assumption, that the surface color can be predicted from the input quantities, we propose to use the following heuristic, which proved, judged subjectively, to produce good results. Please note, that we created one surface layer per surface type (as identified in the clustering step) and we estimate all Fi(x) independently from each other. At first, we scale all input quantities such that their domain is [0; 1). Next we compute a histogram for each input which denotes how many pixels exhibit the surface type for a certain input value. For example, how many pixels of the satellite image belong to the vegetation cluster and appear at a certain terrain elevation. We found empirically that appropriate distribution functions can be obtained by scaling (and clamping) the histogram such that its mean value equals to 0.5. Resulting functions for the vegetation layer are shown in Fig. 6. The distribution functions are stored as textures for use in the procedural texturing algorithm. The evaluation the analytic functions is replaced by sampling textures.

A diversification of the landscape coloring is achieved by adding more surface layers. To do that without complicating the constraint function estimation, we add child nodes to those level-1 nodes which represent larger clusters in the hue-saturation histogram (Fig. 7), e.g. the greenish vegetation cluster. These nodes are created with slightly, randomly modified attributes for noise diversification and colors from within the surface type cluster.

Due to the lack of a meaningful layer hierar-chy, the estimated constraint functions appear rather complicated. If a layer hierarchy based on geological observations would be available, then the histograms would be less disconnected and smoother. For example, assume a fertile soil type appearing at all heights. A vegetation layer on top of it may appear only at a certain height interval (due to other constraints for example). As a consequence, a simple nearly constant height distribution function for the fertile soil becomes disconnected. This is because the non-existent hierarchy (just one level for the basic distribution functions) does not represent the fact, that the soil layer is the precondition for the vegetation layer. This problem cannot be solved with a simple heuristic and rather requires an interactive approach.

Results and Conclusions

We found that our rather simple method provides a plausible and visually pleasing reproduction of natural terrain textures. For estimating the constraint functions we excluded the terrain parts that were cultivated by humans (especially the area north of the mountain range in Fig. 3, where irrigated farmland exists). It is obvious, that a method like ours cannot reproduce such regions: human cultivation does not follow the same rules as natural distribution of plants and soil. One remaining problem with satellite images is cloud cover. Due to the white colors of clouds they can misleadingly classified as snow cover. At least for images exhibiting sparse clouds this problem can be solved with a simple special treatment of the snow surface type: for most terrain regions a minimal altitude for the appearance of snow can be indicated and all pixels that are classified as snow (due to their color), but do not meet this height criterion are discarded. By integrating shortwave infrared images (e.g. Landsat, Channel 7), snow and ice can be distinguished from clouds. The results achieved with our method are illustrated in Figure 6 showing the same region as visible in the satellite image (Fig. 3). The sea and atmospheric conditions were rendered using appropriate analytic models [PSS99, PA00].

During rendering textures with higher resolution than the input data are generated and the texture diversification due to the noise functions produces at least plausible results without changing the overall appearance of the terrain. For texture generation, we only store the distribution functions and the geographic input data. Each distribution function is represented by 256 equidistant samples stored as an 8-bit texture. The procedural model is implemented as vertex and fragment shaders and entirely executed on graphics hardware. Rainfall, temperature and radiation is stored in a 1024 2 RGB 8-bit texture, the elevation data for our example is 2048 2 with 16-bit precision. Due to tiling and use of a texture atlas, the resolution of the synthesized texture adapts to the view point. The tiled, adaptive texture (occupying 40mb of video memory) used in Fig. 6 corresponds to a 32768 2 fixed-resolution texture. In general, using textures for terrain rendering is only suitable for distant views -for close view, ground detail like shrubs and rocks have to be represented by geometric objects. The execution time for the two automated steps of the fitting process, computing the hue-saturation histogram and surface classification with constraint function estimation, was 2.0sec and 0.7sec on a P4, 3.4GHz processor for a satellite image resolution of 2400×2200 pixels.

Basically a once fitted procedural model, represents a specific clime zone and soil constitution. Thus, changes to the height-field can be made and after redoing the geographic simulations or extrapolating measured data, images of the modified terrain can be rendered with likely realistic texturing.

In the future we would like to exploit more information available from satellite imagery, e.g. infrared bands, to ensure a more detailed and accurate classification. To fully automate this process, appropriate clustering methods have to be applied and validated. Identification of surface layer dependencies and constructing a layer hierarchy accordingly is also subject to research.

Figure 1 :

 1 Figure 1: Applying a surface layer includes modification of the elevation data, normal/slope computations, and finally the determination of the new surface color and height value.

Figure 2 :

 2 Figure 2: Surface layer constraints are represented by clamped hat functions.

Figure 3 :

 3 Figure 3: The surface layer constraints and properties are estimated from the real-world input data for the terrain. Please note that for water surfaces the original elevation data does not capture the sea bottom -elevation data is modified there algorithmically. The Landsat satellite image was taken in August 1999 showing little snow cover and snowfree glacier tongues.

Figure 4 :

 4 Figure4: Traversal of a surface layer hierarchy: height always passes to the layer that is evaluated next, but coverage only feeds back to the layer's children.

Figure 6 :

 6 Figure6: Left: two real-time renderings of the Kazakhstan region with procedural texturing parameters acquired from real-world data. Please note, that the northern lakes and human cultivated regions were not considered during classification and parameter estimation and thus, as a consequence, do not appear in the rendering. Right: the distribution histograms (y-axis: probability) of the vegetation surface type for the different input quantities (x-axis, scaled/biased to domain [0;1)).

Figure 5 :

 5 Figure 5: The RGB satellite image is converted into HSV color space. Using these two color spaces allows a feasible clustering of pixels to surface types.

Figure 7 :

 7 Figure 7: The hue-saturation histogram of the satellite image (which special treatment of gray colors) and the respective color table. The right image shows the classification of surface types for four clusters: water (blue), vegetation (green), rock (brown) and snow (white).

Acknowledgement: this project was funded by the DFG project Interaktive Visualisierung prozeduraler Modelle.


Kyle Hegeman

Simon Premoze

Michael Ashikhmin

George Drettakis

Simon Premože

Approximate Ambient Occlusion For Trees

published or not. The documents may come

Introduction

Synthetic natural scenes contain very large amounts of geometry. As an example, a field can contain several trees, each with hundreds of thousands of leaves, and millions of grass blades. Although it is now possible to render animated versions of such scenes using current graphics hardware, the lighting models used are usually very basic, typically involving point or directional hardware light sources, and simple local illumination.

We are interested in the illumination of such scenes from the skydome. Since such environments usually include a significant amount of mutual occlusion, for example between leaves or grass blades, subtle lighting effects are very visible. Evidently, a true global illumination solution (e.g., using path tracing or photon mapping [START_REF] Dutre | Advanced Global Illumination[END_REF]) would correctly simulate such effects. The prohibitive computational costs of such approaches, especially in the case of scenes with moving objects, precludes their usage in an interactive setting.

One approximation to full global illumination which has become quite popular is ambient occlusion [START_REF] Christensen | Ambient occlusion, image-based illumination and global illumination[END_REF][START_REF] Landis | Ambient occlusion. Course 16: RenderMan in Production[END_REF]], which we present in more detail below. In a nutshell, for every pixel we can compute how much of the skydome is visible, and modulate illumination by this percentage. This corresponds to outdoors illumination on a cloudy day. This approach is easy to understand, and thus has apparently become popular with artists in the production of films or video games. Most ambient occlusion based methods, however, require an expensive preprocessing step. For example, ray-casting in a large number of directions towards the sky for each visible point is frequently used. As a result, this approach cannot be used for interactive viewing of dynamic scenes.

In this paper we propose a simple and efficient approximation to ambient occlusion, which is effective in producing the desired visual effect and allows interactive viewing of complex and dynamic natural outdoor scenes. We emphasize that our main goal is not physical accuracy, but a plausible and convincing effect at acceptable computational cost. Such a goal is particularly appropriate for games applications, where visual quality is important, even if the image is not completely physically accurate.

We develop our approximation based on a statistical argument in a simple geometric configuration, which is similar to tree leaves contained in a sphere, but with several simplifying assumptions. We will show that for such an environment, the approximation is close to the true solution. For more complex configurations our approximation can vary, sometimes significantly, from the true solution. Nonetheless, our method preserves a convincing and compelling visual appearance.

We argue that our approach provides a way to give an interactive "ambient occlusion effect" for complex scenes of grass and trees, including moving objects, such as grass moving in the wind. We believe that our technique can be easily integrated into any interactive application, such as games or VR systems (e.g., driving simulators).

Background and previous work

If a virtual camera is looking in direction ω e , the reflected radiance from the scene at point x (with surface normal n) due to illumination from a skydome is

L(x, ω e ) = ω∈Ω L(x, ω) f r (x, ω e , ω)V x ( ω)( ω • n)dω (1)
where f r is the BRDF of the surface point, and V x ( ω) is the visibility function at the point x in the direction ω, which is 1 if the point sees the sky in this direction and 0 otherwise. If the BRDF f r is Lambertian and we ignore occlusion by objects in the scene (V x = 1), the above equation can be simplified to

L(x, n) ≈ f r (x) ω∈Ω L( ω)( ω • n)dω (2)
with the radiance L(x, ω) is now becoming just a function of the surface normal n for a given point x. Shading a point with surface normal n by illumination from an environment map using this equation involves a simple texture lookup if the above integral has been precomputed and stored in a texture map. This approach has been used in recent work [START_REF] Kautz | A unified approach to prefiltered environment maps[END_REF]], but the above approximation does not take shadows into account. Two earlier ideas of accessibility maps and obscurances are related to ambient occlusion (which we define more precisely below). In the work of [START_REF] Miller | Efficient algorithms for local and global accessibility shading[END_REF]], "accessibility" is computed based on local surface properties such as curvature. This results in an approximation of occlusion due to the local features of an object, giving aesthetically pleasing renderings which greatly enhanced contrast. Obscurance shading [START_REF] Zhukov | An ambient light illumination model[END_REF]] can be seen as a precursor to ambient occlusion. Here visibility is approximated as a function of distance; the result is visually convincing, but does not offer any guarantee of accuracy.

Ambient occlusion was introduced by [START_REF] Christensen | Ambient occlusion, image-based illumination and global illumination[END_REF]] and [START_REF] Landis | Ambient occlusion. Course 16: RenderMan in Production[END_REF]] and incorporates effects of shadowing in Eq. 2. The ambient occlusion approximation is as follows:

L(x, n) = 1 2π ω∈Ω V x ( ω)dω f r (x) ω∈Ω L( ω)( ω • n) (3)
The first integral is ambient visibility which can be precomputed and stored in another texture map. Landis [START_REF] Landis | Ambient occlusion. Course 16: RenderMan in Production[END_REF]] also computes the average direction (or least occluded direction) which is used in the environment map lookup instead of the true surface normal. A hardware version of this was presented in [START_REF] Pharr | GPU Gems[END_REF]].

As we shall see, our approximation uses some specific assumptions concerning trees and grass. The work of [START_REF] Reeves | Approximate and probabilistic algorithms for shading and rendering structured particle systems[END_REF] can be seen as a precursor to our work. In their approach, ambient light on tree leaves is attenuated by an approximation of leaf self shadowing, which is based on the depth of the point being shaded inside the tree. The intuition used is that points deep in the tree are darker by an exponential factor of the depth. Despite some similarity to our approach, this paper does not attempt to explicitly approximate ambient occlusion, and as we shall see (Figure 12), gives significantly less convincing results than our method.

There have been several methods which improve the speed of obscurance or ambient occlusion calculations. [START_REF] Mendez | Real-time obscurances with color bleeding[END_REF]] recompute obscurance values in the context of radiosity-like secondary diffuse illumination computations; the method requires subdivided environments and could not handle the type of complexity we address here. [START_REF] Sattler | Hardware-accelerated ambient occlusion computation[END_REF]] compute visibility at vertices by sampling the environment and utilizing temporal coherence. The approach uses hardware occlusion queries in a vertexbased approach, which would probably not provide an acceptable quality/speed tradeoff for the very complex scenes of vegetation and trees we target.

Precomputed radiance transfer [START_REF] Sloan | Precomputed radiance transfer for real-time rendering in dynamic, low-frequency lighting environments[END_REF] is also related to ambient visibility, but computes similar quantities at a high precomputation and storage cost. This approach generally results in a more accurate solution than the approaches cited previously.

[ [START_REF] Kautz | Hemispherical rasterization for self-shadowing of dynamic objects[END_REF]] use a hemispherical rasterizer to compute a 2D occlusion mask for visibility to achieve interactive rendering rates. [START_REF] Bunnell | GPU Gems 2[END_REF]] presented an adhoc method for computing ambient occlusion and one-bounce global illumination for dynamic objects. As with the other approaches cited above, these two solutions can handle relatively small dynamic models, but are not adapted to our large outdoor scenes. Finally, [START_REF] Kontkanen | Ambient occlusion fields[END_REF] use a precomputation step to compute occlusion which allows real-time viewing. The precomputation step is in the order of minutes for relatively small models.

Visibility approximation

As mentioned above, we do not necessarily seek a physically accurate solution for ambient occlusion, but only one which is visually plausible. For the very geometrically complex scenes we consider, the resources of even the most powerful rendering systems are already stretched just to push through enough geometry with even the most basic lighting model. We thus require very simple additional computational overhead, and we preclude precomputation since we want to treat dynamic scenes and avoid extra storage.

Most of the difficulty in computing ambient occlusion is due to its global nature -even objects arbitrarily far away can potentially change sky dome visibility from a given point. Our solution is to apply an aggressive approximation to the problem and replace global visibility computation with a purely local expression.

During a typical shading computation, the only geometric information readily available for a given point to be shaded is its position in space x and normal direction n. Ambient visibility V will therefore be written as a function of only these variables: V = V (x, n). Since we cannot compute V for a specific scene, we take a statistical approach, i.e., we consider the value of V averaged over many instances of similar scenes, which we write V .

We develop our approximation on a simple environment consisting of a single large sphere filled with small occluders according to some distribution, which is a reasonable approximation for many types of trees. We can create multiple instances of such an environment, with constant sphere radius and position, and in each of them compute visibility V i (x, n) at a given point x with normal n. This quantity will, of course, be different for each sphere created. However, the average visibility over this ensemble will converge to the value V (x, n) = V i (x, n) we are interested in. Since averages over an ensemble and over distribution are equivalent, one could in principle derive V (x, n) if the shape and all parameters of occluder distribution are known. However, obtaining useful simple closed form expressions is unlikely even for the simplest distributions such as uniform.

In reality, the occluder distribution inside a canopy of a tree is, of course, much more complex than uniform. To obtain a practical solution, we propose to estimate V (x, n) for a large, near-spherical approximation of a tree as follows.

We first enclose the complete tree inside a spherical shell and obtain its position and radius. This is the only precomputed information we use and is just a simple spherical bounding box computation. At runtime, for a visible point x to be shaded, a plane in world space is defined by the point and the normal vector (see Figure 1). The volume of the sphere segment (V cap ) in the direction of the surface normal is then computed as:

V cap = 1 3 πh 2 (3R -h) ( 4
)
where h is the height of the segment and R is the sphere radius. For a uniform occluder distribution, this volume is proportional to the total number of occluders affecting the given point. A point closer to the sphere boundary should be less occluded than a point on the same plane but closer to the sphere center. An additional factor f accounts for such off-center effects:

f = h 2 (h 2 + d 2 ) ( 5
)
where d is off-center distance in the plane. f is the (squared) cosine of the "off-center angle" for the given point. The visibility approximation we use is then:

V = exp(-αρ fV cap ) (6)
where α is a constant which can be adjusted by the user and ρ is occluder density.

For a point at the center of a disk cutting through the sphere ( f = 1), ambient visibility simply gives the average probability (over directions ω) of finding no occluders between the current point x and a point outside the tree. Equation 6 then corresponds to the probability of finding zero occluders in a Poisson random process with intensity proportional to the number of occluders affecting the current point. A more detailed discussion of visibility in random fields can be found in the mathematical literature [START_REF] Zacks | Stochastic Visibility in Random Fields[END_REF][START_REF] Hall | Introduction to the Theory of Coverage Processes[END_REF]]. In the first two rows of Figure 3, we show comparisons of two spheres with uniform distributions of small triangles. The leftmost column is the true ambient occlusion, the middle column is the approximation described above and the righmost column is the absolute difference. As we can see, for the specific case of uniform distributions, the approximation works well. Evidently, as can be seen in the last row, the error for the case of a more general object (an apple tree), is much larger, but the overall appearance trends are still well preserved.

As underlined by this last example, the reasoning above simply provides some intuition behind the formulas we have chosen, which work well in practice, and is not intended as a rigorous derivation. Clearly other forms are possible, but both visual and numerical results show that our approach works well when we are close to our assumptions. For the general case, it delivers visually plausible results at the cost we can afford, despite potentially high numerical error. Not all trees can be approximated well by a spherical shell. For some an ellipsoid is a much better choice (Figure 3, right). In such cases, the same runtime computation is performed but only after Figure 4: Comparison between Reeves and Blau method (left) and our method (right). Reeves and Blau parameters are ad hoc and do not correspond to parameters in our model. We show the best image we were able to produce with the Reeves and Blau method. corresponding scaling along the axes is applied, bringing the ellipsoidal shell into a sphere. This gives a new point position and cutting plane orientation. More involved approximations are briefly discussed later (Section 5).

A comparison to the Reeves and Blau technique [START_REF] Reeves | Approximate and probabilistic algorithms for shading and rendering structured particle systems[END_REF] for a single tree is shown in Figure 4. As we can see, our method has significantly richer visual detail, compared to the somewhat smooth solution provided by their approximation.

Interaction with Objects. The model presented so far computes occlusion only for a single object in isolation. A tree can also affect visibility for objects such as ground or nearby moving objects. The simplest approximation to computing ambient occlusion is to find the solid angle of large occluders (trees in our case).

The solid angle subtended by the sphere of radius r at a point at distance d away from the center of the sphere is approximately

A SA = π(r/d) 2 (7)
The ambient occlusion due to large occluders is then A large occluder can be represented by several spheres of various sizes (see Figure 5). In our experience with outdoor scenes, a single sphere is often enough to convey the effects of large occluders on nearby objects. More accurate inter-object shadowing due to distant illumination can be computed using other methods, see [START_REF] Mei | Rendering with spherical radiance transport maps[END_REF][START_REF] Kontkanen | Ambient occlusion fields[END_REF].

V = 1.0 -(A SA /2π) (8)
Visibility approximation for grass. In the case of a grass field (see Figure 6), integrated visibility is given as follows:

V = P z 2h (1 + cos θ ) (9)
where P z the height of the hit point on the grass blade and h is the total height.

Examples

The visibility approximation from the previous section has been implemented in both a software-based renderer (raytracer) and graphics hardware. We show some examples of our test system, running on an Intel Xeon 2.4GHz PC with a 256MB NVIDIA GeForce 6800 Ultra graphics card. All the images in the paper are rendered using the 8xS antialiasing mode. The density and size of occluders (αρ product) have been chosen by hand for all examples; this is a simple once-per-object operation. In the future, a simple one-time preprocessing step could compute these two parameters automatically and provide the user with an initial guess for further fine-tuning. We used the NVShaderPerf tool from NVidia to analyze the theoretical performance of our fragment shaders. This tool computes the pixel throughput of the fragment pipeline measured in megapixels per second (MP/s). We evaluated each of our three shaders: our approximation, Reeves and Blau, and simple ambient. For a 6800 Ultra card, the resulting throughput numbers are 460 MP/s, 580 MP/s, and 2300 MP/s respectively.

Figure 7 compares results using our ambient occlusion approximation with that obtained with ambient illumination only. A single sphere was used to approximate the shape of the tree. At 512x512 image size, we can render a single 550k triangle tree mesh at 23 frames per second using our approximation. By comparison, ambient runs at 31 fps and the Reeves and Blau approximation at 25 Figure 8: "Ellipsoidal" tree. This tree is not well approximated by a single sphere or multiple shells. The tree on the left uses an ellipsoid while the tree on the right uses a spherical shell approximation.

fps. These numbers correspond well to the expected performance ratio as computed by NVShaderPerf. Figure 3 (bottom row) shows the difference between the approximate ambient occlusion and a ground truth occlusion computed using a standard raytracer (128 directions over a hemisphere) for this scene.

The effectiveness of the ellipsoidal approximation for some trees is illustrated by Figure 8. The difference between using a single spherical shell and a single ellipsoid is prominent. An example of applying the approximation for the interaction of the three with the ground plane is shown in Figure 9. A grass field rendered using the approximation described in Equation ( 9) is shown in Figure 10.

Figure 12 shows two vantage points (image resolution 800x500) of a complex outdoors scene containing grass and a several different types of plants and trees. This scene contains over 22 million vertices and 12 million triangles and was generated by an ecosystem modeling program [START_REF] Deussen | Realistic modeling and rendering of plant ecosystems[END_REF]]. Our current OpenGLbased implementation only performs very basic optimizations. Due to the geometric complexity of this scene, the bottleneck in the graphics pipeline is not in the fragment stage. For this reason, the rendering speed is about 5 frames per second for all three fragment shaders we compare. Performance could be improved significantly by using an LOD scheme such as [START_REF] Deussen | Interactive visualization of complex plant ecosystems[END_REF] or one of many other acceleration techniques. Our approach is independent from, and can be used together with, any such method. In addition, any increase in future graphics hardware performance will be directly reflected on the frames rates cited here.

The middle row of Figure 12 shows that the Reeves and Blau [START_REF] Reeves | Approximate and probabilistic algorithms for shading and rendering structured particle systems[END_REF] approach gives a pleasing, smooth image, which is appropriate as a replacement to an ambient term. However, the visual result of our approximation (top row) is much closer to what one would expect from an ambient occlusion solution, see Figure 11. In particular, there is a noticable gain in the visual quality of the trees.

Conclusion

Our goal at the outset was to provide a simple approximation to ambient occlusion, allowing plausible and compelling skydome-like lighting for complex and dynamic outdoors scenes containing trees and grass at interactive rates.

To achieve this goal, we developed an approximation based on assumptions which hold in a very simple configuration. We have illustrated that our approximation gives quite accurate results for these simple configurations, and that it provides plausible visual results for the more general case, even though it can result in large numerical error.

Our method requires virtually no precomputation, and is sufficiently fast to interactively render scenes with moving geometry. In the images and the accompanying video, we show that we can walk through complex and dynamic outdoors scenes at interactive rates.

Given our self-imposed constraints we clearly needed a very aggressive approximation approach such as that presented here. Even though we illustrate that the heuristic chosen does have some justification in the simple case, for the general case we have no guarantee on error. Our argument is that the resulting visualizations are nonetheless compelling, and sufficient for example in the context of interactive environments such as games, since the user is not sensitive to the type of error which our approximation incurs. In the results presented here, we only provide subjective evidence of this claim.

In future work, an even better approximation of tree shape is posby combining several, usually relatively few, smaller spherical (or ellipsoidal) shells (Figure 3,left). For simplicity, we can consider a point to belong to exactly one such shell (that with the closest center) even if geometrically it lies in several shells simultaneously.

Creating the set of shells and localizing a point to a specific one does require some additional precomputation and runtime processing. In fact, a complete hierarchy of such shells of different sizes on multiple levels can be built with any point clustering method. If the Figure 11: Our ambient occlusion approximation for complex ecosystem scene. performance of the system allows it, visibility can also be computed on several levels and the results combined.

Finally, we would also like to conduct a user study to determine the level of "believability" of such approximations: we would present users with a walkthrough of the exact solution and that of the approximation, and ask them to rate "perceived realism". We are fairly confident that for a non-expert user, the difference will be low.

Figure 12: Two views of an ecosystem rendered with our ambient occlusion (top row), Reeves Blau approximation (center row) and ambient term only (bottom row). Note the improvement in quality in the trees with our approximation.

Figure 1 :

 1 Figure 1: Spherical cap geometry as defined by a point x and normal n. The approximate visibility is determined by spherical cap volume V cap , density of occluders ρ and off-center distance d.

Figure 2 :

 2 Figure 2: Comparisons between true occlusion and our approximation. Left: true image Middle: our approximation Right: absolute difference

Figure 3 :

 3 Figure 3: A tree can be approximated by multiple spherical shells (left) or by ellipsoid (right).

Figure 5 :Figure 6 :

 56 Figure 5: Tree ground interaction. A large occluders affects visibility on the ground. The distance to the occluder and the solid angle it covers are used to compute approximate occlusion.

Figure 7 :

 7 Figure 7: Comparison between true approximated ambient occlusion (left) and ambient only approximation.

Figure 9 :

 9 Figure 9: Interaction between the tree and the ground. Using our approximation (left) and no interaction (right).

Figure 10 :

 10 Figure 10: Grass field rendered using our approximation.

Acknowledgements

The ecosystem scene was generated using XFrog modelling software (www.greenworks.de). The last author's participation was funded in part by the French Ministry of Research ACI MD project SHOW ¨.