Catherine Plaisant
email: plaisant@cs.umd.edu

Jean-Daniel Fekete

Georges Grinstein
email: grinstein@cs.umd.edu

Promoting Insight-Based Evaluation of Visualizations: From Contest to Benchmark Repository

Keywords: Visualization, information, competition, contest, benchmark, repository, measure, metrics Digital Object Indentifier 10

Information Visualization (InfoVis) is now an accepted and growing field but questions remain about the best uses for and the maturity of novel visualizations. Usability studies and controlled experiments are helpful but generalization is difficult. We believe that the systematic development of benchmarks will facilitate the comparison of techniques and help identify their strengths under different conditions. We were involved in the organization and management of three information visualization contests for the 2003, 2004 and 2005 IEEE Information Visualization Symposia, which requested teams to report on insights gained while exploring data. We give a summary of the state of the art of evaluation in information visualization, describe the three contests, summarize their results, discuss outcomes and lessons learned, and conjecture the future of visualization contests. All materials produced by the contests are archived in the Information Visualization Benchmark Repository.

Introduction

Information Visualization is now an accepted and growing field with numerous visualization components used in mainstream applications such as SPSS/SigmaPlot, SAS/GRAPH, and DataDesk, in commercial products such as Spotfire, Inxight, Tableau, HumanIT, and ILOG JViews, and in domain specific standalone applications such as interactive financial visualizations [SMo06] and election data maps [NYT06]. Nevertheless, questions remain about the potential uses of these novel techniques, their maturity and their limitations.

Plaisant reviewed evaluation challenges specific to information visualization and suggested initial actions [START_REF] Plaisant | The Challenge of Information Visualization Evaluation[END_REF] such as refined evaluation methodologies, use of toolkits, dissemination of success stories, and the development of contests (Figure 1), benchmarks and repositories. Empirical user studies are very helpful but take significant time and resources, and are sometimes found of limited use as they are conducted with ad-hoc data and tasks in controlled laboratory situations. Benchmarks facilitate the comparison of different techniques and encourage researchers to work on challenging problems. However to be convincing, the utility of new techniques needs to be demonstrated in a real setting, within a given application domain and a set of users. Contests attempt to create these surrogate situations that are representative of real world situations. They engage a competitive spirit and often produce results that help the community with comparisons of visualization tools applied to the same problem.

Competitions help push the forefront of a field quickly. TREC evaluations (the Text REtrieval Conference) [START_REF] Voorhees | Overview of the sixth Text Retrieval Conference (TREC-6)[END_REF] exemplify the best of these in obtaining the participation of many corporate and academic research groups. For many it is simply the emotional aspect of winning or the excitement of the competition that compels them to participate.

A contest presents a problem that many will attempt to solve. If the problem is challenging and representative of a real world situation, then the contestants' solutions highlight what techniques are possible, and which Final version -May 2007 2 solutions seem better to pursue. Participants can describe the insights they gained while using their tools. Insight can simply be defined as a non-trivial discovery about the data or, as a complex, deep, qualitative, unexpected, and relevant assertion [Nor06]. We believe that the Infovis 2003 contest was the first attempt to include the reporting of insights as an evaluation criterion [Inf03]. Often some solutions provide such good results that other participants are driven to compete in the next year's contests if the problem offered is similar. But for contests to have a long term impact, benchmark datasets, the associated tasks and the submitted materials need to be archived in a repository, safeguarding the baselines against which new techniques and approaches can be compared.

In this paper we briefly review the state of the art and the challenges for evaluation in information visualization. We also describe the first three information visualization contests, summarize their results, and discuss their effect.

Information Visualization Evaluation State of the Art

Information visualization systems can be very complex [Chen00] and require evaluation efforts targeted at different levels. One approach described in the evaluation section [START_REF] Laskowski | Evaluation Methodologies for Visual Analytics[END_REF] of the Visual Analytics research agenda [START_REF]Illuminating the Path: The Research and Development Agenda for Visual Analytics[END_REF] identifies three levels: the component level, the system level, and the work environment level (Figure 2).

The component level includes the individual algorithms, visual representations, interactive techniques and interface designs. Data analysis algorithms can often be evaluated with metrics that can be observed or computed (e.g. speed or accuracy), while other components require empirical user evaluation to determine their benefits.

There have been demonstrations of faster task completion, reduced error rates or increased user satisfaction measured in laboratory settings using specific visualization components. For some techniques some scores can be computed to evaluate the potential quality of simple displays, e.g. [START_REF] Mackinlay | Automating the design of graphical presentations of relational information[END_REF], but controlled experiments remain the workhorse of evaluation.

Figure 2: The three evaluation levels for Visual Analytics (Figure 6.1 in [START_REF]Illuminating the Path: The Research and Development Agenda for Visual Analytics[END_REF])

The system level focuses on interfaces which combine and integrate multiple components and need to be evaluated by comparing them with technology currently used by target users (e.g. [START_REF] Plaisant | SpaceTree: Supporting exploration in large node-link tree: design evolution and empirical evaluation[END_REF]). Those evaluations also usually consist of controlled experiments that take place in the laboratory, using surrogate scenarios and short tasks. A more modern approach is to conduct insight-based evaluations where participants are given open-ended complex tasks and asked to report on the insights gained [Nor06, Sar04]. The discovery process is rarely an isolated, short-term process nor is it unique. Users may need to look at the same data with multiple tools over a longer period of time (days or months). This cannot be captured with controlled laboratory studies.

The third level is the work environment level where evaluation addresses issues influencing adoption. Case studies describe users in natural environment doing real tasks [START_REF] Gonzales | Benefits of information visualization for administrative data analysts[END_REF][START_REF] Trafton | Turning pictures into numbers: extracting and generating information from complex visualizations[END_REF]. These are time consuming and may not be applicable to other domains but they can describe discoveries that take place over a long period. A few research communities have a long history of success at promoting large scale evaluation programs and methodologies for measuring the effectiveness of information technology. For example TREC (the Text REtrieval Conference) [START_REF] Voorhees | Overview of the sixth Text Retrieval Conference (TREC-6)[END_REF][START_REF]Text REtrieval Conference (TREC)[END_REF] started in 1992 and provides datasets and scoring methods to evaluate information retrieval methods. The Message Understand Conference (MUC) series [START_REF] Chinchor | Evaluating message understanding systems: an analysis of the third message understanding conference (MUC-3)[END_REF] was started in 1987 for the purpose of qualitative evaluation of the state of the art in message understanding and has contributed to the development of improved linguistically-based systems. The speech recognition evaluation program [START_REF] Pallett | Measurement in support of research accomplishments[END_REF] has been creating speech corpora and benchmark tests primarily based on word error rates since 1988. The earlier work provides baselines against which researchers could demonstrate the effectiveness of recognition algorithms. Recent work has included the development of broadcast news transcription tasks to support topic detection and speech-to-text research.

Those sponsored evaluation programs are successful in part because clear computed metrics can be established to compare the results of tools with a trusted "ground truth", even if sometimes the ground truth is generated by humans as it is for TREC where human experts decide the "true" relevance of documents.

Other volunteer efforts have also been useful. The KDD Cup is an example of benchmarking where the knowledge discovery community agrees on scores that can be computed on new datasets created every year [START_REF] Gehre | Overview of the 2003 KDD cup[END_REF].

Recently, the first Visual Analytics Science and Technology contest took place [VAS06; Gri06]. The VAST contest used a synthetic data set with embedded ground truth integrated into real text data in a careful manner [START_REF] Whiting | Threat stream data generator: creating the known unknowns for test and evaluation of visual analytics tools[END_REF]. The scenario involved individuals performing some fraudulent activities and the discovery required analyzing and exploring the collection of multimedia documents (mostly text) and identifying the links between the individuals, places, and events. Ground truth was available about the "who, what, when, where and why" of suspicious activities facilitating evaluations as to how well the teams had analyzed the situation and permitting better estimates of the general utility of the tools. A pilot investigation looked at quantitative metrics using for example the number of subplots identified by each team and their relative complexity and subtlety, and found the ranking of the submissions to closely match the subjective assessments of the judges.

In the Human-Computer Interaction community contests are usually design competitions, one memorable exception being the 1997 CHI Browse-off [START_REF] Mullet | On your marks, get set, browse! (the great CHI'97 Browse Off)[END_REF]. It brought together several visualization and browsing technologies for a live comparison, almost a competition, in which the tasks were generic information retrieval ones. In parallel with the development of benchmarks, specific evaluation methods and tools for emerging new technologies are being designed, for collaborative environments [START_REF] Scholtz | A Framework for Evaluating Collaborative Systems in the Real World[END_REF], ubiquitous computing [START_REF] Scholtz | User-Centered Evaluations of Ubicomp Applications[END_REF], and intelligence analysis [START_REF] Cowley | Glassbox: an instrumented infrastructure for supporting humaninteraction with information[END_REF].

Many other communities have evaluation programs. A complete survey of practices and lessons learned is needed and would help foster collaboration and coordination between evaluation efforts.

The Infovis Contests

The Contest Process

All three contests had a similar general organization. The contest was announced at the previous conference and the call for participation made public in February at the same time as the corresponding InfoVis Symposium call. The dataset and tasks were posted shortly thereafter. Participants had four months to prepare with a June deadline and had to submit a set of electronic documents consisting of • a two page summary, • a video illustrating the interactive techniques used, • a detailed webpage describing the tool, how it was used to accomplish the tasks, and what insights were gained, and • information about the team and tool provenance.

Judging was conducted during the summer primarily by the co-chairs and resulted in the selection of multiple first and second place entries. In October a session at the conference was dedicated to the contest, during which the chairs summarized the process and the lessons learned, and several teams presented their results. The

Judging

The criteria for judging included 1) the quality of the data analysis (what interesting insights were found in the data), 2) the appropriateness of the visual representation for the data and the tasks, the usefulness and creativity of the interactivity, and the flexibility of the tool (how generic was the approach), and 3) the quality of the written case study (description of the strengths and weaknesses of the tool used).

Judges reviewed all the submissions, a very time consuming activity. A list of criteria was used, but numerical ratings were not necessarily compiled, (although attempted) because the number of entries was small enough that judges could easily group the entries in three categories and nominate the first and second place candidates. A number of conference calls and meetings allowed the chairs to decide on the number of winning entries and finalize the results.

Selecting the first place was generally easy as some entries stood out, but deciding on the cut-off for rejected entries was very difficult as many entries of lesser quality still had some interesting features we wanted to highlight and helped demonstrate the variety of solutions. We also wanted to encourage and reward participation.

Laskowski [START_REF] Laskowski | Evaluation Methodologies for Visual Analytics[END_REF] provides a thorough review of the challenges of visual analytics evaluation many of which are common to information visualization evaluation.

The contests placed the participants in a fairly realistic situation, giving them a fair but not arbitrarily large amount of time to analyze the data and prepare their answers. Although ideally we would have liked to evaluate the quality of answers computationally this was not possible. The problem was due to the fuzziness and variety of both the questions and answers to the contest. These included visual representations, collections of articles, new algorithms, or new visualization tools, each of whose correctness or evaluation may not be computable. This still forces human evaluation. TREC, for example, still uses human judging for determining the accuracy of the retrieval and such is the case for the IEEE InfoVis Contests we discuss.

Another difficulty for information visualization comes from the impact of the discovery process, an extremely interactive and personal activity. Whereas computational algorithms can be compared through the accuracy of their results, it is still not possible to accurately measure the results or impact of a single or multiple visualizations. We still do not have measures of perceptual information transfer. There is beginning research in measures of interestingness and other metrics related to visualization [START_REF] Keim | Test datasets for evaluating data visualization techniques[END_REF][START_REF] Grinstein | Benchmark Development for the Evaluation of Visualization for Data Mining[END_REF], but these are in their infancy and too simple to be applied to the current contests.

So we either propose simple tasks which yield precise results or specify more complex exploratory tasks and thus have much less predictable results. The latter is more interesting but makes the evaluation process difficult to plan for and forces real time evaluation criteria which sometimes end up reviewer dependent. One additional argument for simple tasks even if they are unrealistic is that a system which fails to achieve simple tasks would be a very limiting system and is likely not to support more complex or exploratory tasks.

Our approach was to balance task simplicity and complexity to obtain a satisfying tradeoff.

We now review the 2003, 2004 and 2005 contest one by one in detail describing the data, tasks, judging, results and lessons learned.

The 1st contest -Infovis 2003

The first contest took place in 2003 [Inf03]. It focused on the analysis of tree structured data and in particular looking at the differences between similar trees.

The 2003 Data

There are hundreds of types of trees with varying characteristics. In an effort to represent this diversity in an accessible manner, contestants were provided three very different application examples with datasets in a simple XML format. These were Phylogenies -Small binary trees (60 leaf nodes) with a link length attribute. No node attributes except their names.

File system and usage logs -Large trees (about 70,000 leaf nodes) with many attributes, both numerical and nominal. Changes between the two trees could be topological or attribute value changes. Data for four time periods was provided. Classifications -Very large trees (about 200,000 leaf nodes) with large fan-outs. Three node attributes, all nominal. Labeling, search and presenting results in context is important. We allowed teams to work on a

Final version -May 2007 5 subset of the dataset (the "mammal" subtree) if they could not handle that many nodes.

The 2003 tasks

We provided general tasks (about 40 tasks in 11 categories) and tasks specific to the selected datasets. General tasks were low level tasks commonly encountered while analyzing any tree data: topological tasks (e.g., which branch has the largest fan-out?), attribute-based tasks (e.g., find nodes with high values of X), or comparison tasks (e.g., did any node or subtree "move"?).

The tasks specific to each dataset included more broad goal-setting tasks (e.g., for the phylogenies, what mapping between the two trees topologies could indicate co-evolution and, maybe, the points where the two proteins were not co-evolving?)

We made it clear that it was acceptable to submit partial results. For example, one could work only on some of the tasks. We also clarified that we were not looking for a detailed result list (e.g., a list of deleted nodes for the task "what nodes where deleted") but an illustration or demonstration of how the visualization helped find these results. General background information was provided about the data and tasks. This was particularly important for the Phylogenetic data.

The 2003 Submissions and Judges

Teams had about five months to prepare. We had several judges in standby mode in case we received a large number of submissions. We received eight entries, a small number, but satisfactory for a first contest. The judging was completed by the two chairs and two additional judges, all knowledgeable in information visualization and human-computer interaction, with one of the judges also knowledgeable in biology, useful because of that specific dataset.

The 2003 Results

The first main finding was that the tasks and datasets were too complex for such a contest. Each tool addressed only a subset of the tasks and only for a subset of the datasets. The phylogeny chosen required domain expertise hence was "real", and even though it consisted of a small binary tree, it was not used, probably because the tasks were complex and required biological knowledge (e.g., perhaps working with a biologist).

The second main finding was that it was difficult to compare systems even with specific datasets and tasks.

We had hoped to focus the attention of submitters on tasks and results (insights), but the majority of the materials received focused on descriptions of system features. Little information was provided on how users could accomplish the tasks and what the results meant, making it very difficult for the judges to compare results. The systems presented were extremely diverse, each using different approaches to visualize the data.

Selecting the three first-place entries was straightforward.

Only the first place submissions demonstrated the benefits of their tools by reporting on the insights gathered, and both the descriptions of the tool and processes were clear making it easy to understand how the tool had been used (see Figure 3). 4) submitted the most convincing description of how the tasks could be conducted and the results interpreted (see also [START_REF] Munzner | TreeJuxtaposer: Scalable tree comparison using Focus+Context with guaranteed visibility[END_REF]).

Zoomology [START_REF] Hong | Zoomology: Comparing Two Large Hierarchical Trees[END_REF] (Figure 5) demonstrated how a custom design for a specific single dataset could lead to a useful tool that addressed many of the tasks satisfactorily. InfoZoom [Spe00] (Figure 6) was the most surprising entry. This tool had been designed for manipulating tables and not trees. However the authors impressed the judges by showing that they could perform most of the tasks, find errors and provide insights in the data. The three second-place entries showed promise but provided less information to the judges on how the tasks were conducted and the meaning of the results. EVAT [START_REF] Auber | EVAT -Environment for Visualization and Analysis of Trees[END_REF] (Figure 7) demonstrated that powerful analytical tools complementing the visualization could assist users in accomplishing their tasks. TaxoNote [START_REF] Morse | Comparison of Multiple Taxonomic Hierarchies Using TaxoNote[END_REF] (Figure 8) demonstrated that labeling is an important issue making textual displays attractive. The Indiana University submission [START_REF] Sheth | Treemap, Radial Tree, and 3D Tree Visualizations[END_REF] (Figure 9) illustrated the benefits of toolkits (e.g. [Bor06; Fek04] by quickly preparing an entry combining several tools, each accomplishing different tasks. All entries were given a chance to revise their materials after the contest. Participants provided a structured form with screenshots and detailed explanations for each task.

The InfoVis 2004 Contest

The second competition coincided with the 10 year anniversary of the InfoVis Symposium. As the visualization of the history of a research field is a problem interesting in itself, it was natural for it to become the core part of the contest. The key advantage of the topic was that it was familiar to all participants. The disadvantage was that the selected corpora were not readily available in a usable form.

The 2004 data

The set of all publications on a topic is too large a universal set of discourse for a competition. We first Final version -May 2007 7 argued about which conferences or journals to include and resolved to limit the dataset to all the IEEE InfoVis Symposium papers and all of the articles used as reference in those papers. Metadata is rich for IEEE and ACM publications and unique keys available.

Producing a clean file (metadata for the collection of documents) was a much bigger challenge than we had imagined. We first made an assumption that both the articles and the most important authors in information visualization would be referred by most of the articles published within the InfoVis symposium. Our look at the references initiating from articles published within InfoVis seemed to us at the same time focused on the field and complete. It would be unlikely that an important publication in information visualization would seldom be referenced by other articles.

This was partially correct but text metadata still yielded numerous ambiguities. IEEE manages the InfoVis articles which are less curated than those of the ACM. Much text metadata was non-unique (e.g., many-to-one names such as Smith, Smyth, Smithe, …). Reference titles were too inaccurate and in many cases erroneous as text handled by the ACM Digital Library uses strings and numerical computations with such string are still imprecise (long string comparisons). Much curation on our end was necessary as references were noisy, sometimes missing, and even sometimes pointing to non-existing URLs.

We thus embarked on cleaning the data. This was a complicated process, with multiple passes, and manually intensive, even with automatic reference extraction as we found no reasonable automatic system to suitably resolve the problems. We manually extracted the articles from eight years of pdf files from the symposia available in the digital library. We then semiautomatically retrieved the articles referenced in those papers again from the digital library. We extracted those which existed when found and manually cleaned and unified the publications not included in ACM library.

The result was a file containing 614 descriptions of articles published between 1974 and 2004 by 1,036 authors, referencing 8,502 publications. It took well over 1,000 hours for us to construct that file, with over 30 people involved.

The 2004 tasks

We proposed 4 high level tasks with a great deal of flexibility for a variety of solutions:

1. Create a static visualization showing an overview of the 10 years of InfoVis 2. Characterize the research areas and their evolution 3. Explore the people in InfoVis: where does a particular author/researcher fit within the research areas defined in task 2? 4. Explore the people in InfoVis: what if any, are the relationships between two or more researchers?

We suggested particular names for task 3 to facilitate comparisons between submissions, and participants used them, along with other names.

The 2004 submissions and judges

The participants were required to submit:

• a two page summary,

• a video illustrating the interactive techniques used, • a structured web form providing details as to how the tasks were accomplished and what discovery or insights were identified.

There were 18 submissions from 6 countries (USA, Canada, Germany, France, Australia, and Netherlands) with 7 academic participants. The judging was done by the three contest chairs (the authors) with the help of one outside reviewer. Twelve teams were selected to have their results published in the InfoVis repository. Four received a first place prize and gave a short talk at the conference. Six teams received a second place prize and presented a poster.

The 2004 results

Quality improved dramatically between 2003 and 2004. The good news was that most teams had provided a lot more insights than we had seen in the first contest. Still, some teams had tools that seem promising "on paper" but reported very few insights (in consequence they did not do very well in the contest.) On the other hand some teams presented tools that seemed of doubtful utility to the reviewers at first but were able to report useful insights, therefore faring better than we had expected in the results. Of course the strongest teams performed well with all requests: had promising visualizations, many insights reported, and convincing explanations of how the insights were obtained using the tools.

None of the twelve selected teams answered all the questions. A few of the participants had extensive experience with text analysis and that was visible in their results. Other had background knowledge of the InfoVis community and could provide better hypotheses about what they were seeing. One tool was developed entirely from scratch for the contest but most teams showed interesting new uses of existing techniques. Node-link

Final version -May 2007 8 diagrams were a very commonly used representation for many of the tasks, with some notable exceptions.

This second contest had a single dataset and simpler tasks so we anticipated reviewing and comparing results would be much easier. Not so. Again, we had hoped that we would be able to evaluate the quality of answers computationally but this was not possible. The problem again was the fuzziness of the answers and the lack of "ground truth" or even consensus on what the best answer might look like. Teams' answers took various forms from a collection of articles or names to a new algorithm to a new visualization, all of whose correctness was not computable. Only human evaluation was appropriate to judge the validity of the answers. In information retrieval, TREC for example does use human judging and pools results from all participants to determine the relevance of documents (i.e. the answers) from which metrics can be computed for a team's set of results. Short of spending time with the team throughout the discovery process (an extremely interactive and personal activity) we could only base our judgment on the materials provided (the video and forms).

There were three first place entries and one student first place:

The entry from Indiana University [START_REF] Ke | Indiana University Contest Submission[END_REF] had many insights gathered from a variety of mostly low tech displays such as barcharts, displaying the output of separate analysis tools. The best display showed a simplified view of the co-authorship network clearly highlighting the main players in the field of InfoVis (as represented by the dataset) (Figure 10). It has since then been used by many as an overview of the community. The entry from the Pacific Northwest National Laboratory [START_REF] Pak | Poster Compendium of IEEE Information Visualization[END_REF] nicely reflected the extensive experience of the team and the rich set of analysis features provided by their tools (Figure 11).

The entry from Microsoft and the University of Maryland [START_REF] Bongshin | Poster Compendium of IEEE Information Visualization[END_REF] was interesting because it took a completely different approach to the problem, departing from node-link diagrams, and using multiple tightly coupled ordered lists to represent the interconnections between authors, papers and references (Figure 12).

The student first prize went to a team from the University of Sydney [START_REF] Ahmed | Poster Compendium of IEEE Information Visualization[END_REF]. It surprised us with its whimsical 3D animation and provided sufficient insights to convince us that the technique had merit (Figure 13).

Second place prizes (see Figure 14 to 20) went to the Université de Bordeaux I with the University of British Columbia [START_REF] Delest | Poster Compendium of IEEE Information Visualization[END_REF], the Technische Universiteit Eindhoven [START_REF] Frank Van Ham | Technische Universiteit Eindhoven Contest Submission[END_REF], Georgia Institute of Technology [START_REF] Hsu Tzu-Wei | Poster Compendium of IEEE Information Visualization[END_REF] and [START_REF] Tyman | InfoVisExplorer[END_REF], the University of Konstanz [START_REF] Keim | University of Konstanz Contest Submission[END_REF], the two teams from Drexel University [START_REF] Xia | White, Associative Information Visualizer[END_REF] and [Chen04], and the University of California, Davis [START_REF] Tee Teoh | One-For-All -University of California, Davis Contest Submission[END_REF]. Each has some interesting technical component and some interesting insights.

We were satisfied that teams reported useful insights but we were still surprised by how few were reported, and that even fewer were really surprising insights. Insights about the whole structure were rare and only came from teams who had experience looking at other domains (e.g., the fact that InfoVis is a small world, tightly connected, was mentioned only by 2 teams.) One team noticed that the most referenced papers were published at CHI, not at InfoVis. Only three teams noticed the existence of references to future papers, a problem resulting from automatically processing references and confusing multiple versions with similar titles such as a video and a paper. Only one insight highlighted something surprisingly missing, namely that there were no papers in the dataset from several of the other competing InfoVis conferences, despite the fact that they had been held for several years.

Teams interpreted the tasks and used the data in surprisingly very different ways. A task such as "describe the relationships between authors" was interpreted in at least the following nine different ways as report on co-authorship; or co-citation; or people working on similar topics; or having a similar number of co-authors; or being a part of big groups or teams; or having a similar number of publications; or a similar number of references; or working in the same institution; or working in the same clique or empire.

Teams also used the data differently. Some created displays showing either only the IEEE Infovis symposium papers or all papers including the references. Some combined both authors and topics and some used separate displays. In one case we suspected that a team used only the papers' first authors but were unable to determine that precisely. One team only used references from InfoVis papers references but not references to papers from other venues.

The presented data was generally pruned dramatically to work with the tools or to create more useful or possibly appealing displays. Few attempted to show complete views. Some teams had a "celebrity" approach ignoring everything but the star papers or authors based on some unique criteria (e.g., numbers of citations). Some clustered first then pruned later with no clear explanation of what had been pruned.

IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication.

Final version -May 2007 9

Reviewing the displays seemed easy at first, but it quickly became impossible to remember what data we had just been looking at, let alone compare different results even when it would have been possible. Many displays had no or very poor captions and none had any summary of the process that generated the display. Each team probably had a clear model of the scope of the data and how it was filtered, aggregated and interpreted, but the displays did not reflect that.

Few teams even attempted to answer the first question, to create a static visualization showing an overview of the 10 years of InfoVis. Teams merely reused one of their screen shots from other tasks so we felt only one aspect of the data had been portrayed and not the entire 10 years of InfoVis. Teams reported very different topics and different numbers of topics (from 5 to 12) and some created topics on the fly, refining the topics iteratively. Sometimes a seemingly narrow topic would take a prominent place: "parallel coordinates" was a major topic in one case while in another system "taxonomy" was a major topic. Reviewing all the submissions gave us an impression of randomness in the choice or labeling of the topics. One of the student teams used their professor's notes to extract topics. It was innovative but again, affected our ability to make comparisons. Most visualizations limited the total number of topics which limited the insights to be related to those topics. But topic extractions were not the focus of the contest so we did not judge the quality of the topics. Nevertheless this made it more difficult to compare insights. Some tools (e. g., In-spire [START_REF] Pak | Poster Compendium of IEEE Information Visualization[END_REF] and an entry from Rutgers University [START_REF] Xia | White, Associative Information Visualizer[END_REF]) allowed users to refine the topics quickly and iteratively by entering a seed term or removing words. Some teams chose to remove common words (e.g., Information Visualization) while others kept them as labels for major topics which was not very useful. Sadly, evaluation only appeared once as a topic, and only after iterative refinement.

Overall, labeling remained a very big problem. Very rarely could we actually guess paper titles when looking at a display. Better dynamic layout techniques for labels were clearly needed. Labels for papers usually consisted of the first few words or even just the first author making it difficult to remember if we were looking at author relationships, or papers, or even topic relationships, e.g. a large node labeled "Johnson" could represent the often-referenced Treemap paper. Part of the problem was simply that teams most often used exploratory tools for discovery but use the same displays for presentation.

Some tools used a single window [START_REF] Tee Teoh | One-For-All -University of California, Davis Contest Submission[END_REF], but most used multiple windows, showing either variants [START_REF] Frank Van Ham | Technische Universiteit Eindhoven Contest Submission[END_REF] or very different displays for different tasks [Chen04], [START_REF] Keim | University of Konstanz Contest Submission[END_REF]. The PaperLens submission [START_REF] Bongshin | Poster Compendium of IEEE Information Visualization[END_REF] illustrated the importance of coordinating views. Only two teams dealt with missing data and uncertainty, others ignored the problem entirely. Visual metaphors seemed to have had an effect on the words teams used to describe their findings, e.g. one team [START_REF] Ahmed | Poster Compendium of IEEE Information Visualization[END_REF] talked about empires when looking at towers in 3D, while others talked about cliques while looking at clusters on node link diagrams. Unfortunately, we also saw examples of "nice pictures" that didn't seem to lead to any insight.

The 2004 contest session at the workshop was very well attended and we received extremely positive feedback.

Attendees reported an appreciation for the wide diversity of solutions and contrasting different techniques. We conjecture that the topic we had selected also made the contest more appropriate to the audience as well as more accessible.

The InfoVis 2005 contest

In the third competition [START_REF] Grinstein | IEEE InfoVis 2005 Contest, Technology Data in the US[END_REF] the chairs aimed for the evaluation of more complete visualization systems and a different type of data. The data set was larger and the questions more targeted. The goal was to identify how well visualization or visual analytics systems or even specific tools could perform with a large but easily understood data set. The chairs missed a key point in that the problem was probably better phrased as a Geographic or Geospatial Information (GIS) challenge rather than simply an information visualization one. The chairs also released the data set only for the competition. The owner of the data set did not permit an open release, something that the chairs tried to avoid and hopefully will avoid in the future.

The 2005 data

This large, information rich, and real data set consisted of information on 87,659 technology companies in the US, and included the year founded, zip code, yearly sales, yearly employment information, along with industry and product information using the North American Industry Classification System [Nai01]. This was a large data set with geographic interpretation, one which pushed the limits of many systems. The data was cleaned by graduate students at the University of Massachusetts at Lowell. Missing data was eliminated as best as possible.

The 2005 Tasks

The three questions related to the characterization of correlations or other patterns amongst variables in the data were 1. Characterize correlations or other patterns among two or more variables in the data. 2. Characterize clusters of products, industries, sales, regions, and/or companies. 3. Characterize unusual products, sales, regions, or companies.

One additional question was more general and openended 4. Characterize any other trend, pattern, or structure that may be of interest.

The chairs felt that these precise questions would make evaluation simpler. And again this was not correct as all questions were too open-ended in their interpretation and comparing the discovery of different correlations was difficult.

The 2005 Submissions and judges

Participants were required to submit materials using the same format as in 2004. There were only 10 participants. This was a surprise but the short time from available data to submission deadline and the size of the data were probably the most important factors. We had no submissions from student teams possibly because we released the first version of the data set at the end of February during which most university information visualization classes already are well under way. The judging was done by the four contest chairs using some numerical ratings but again the selection process was fairly straightforward.

The 2005 Results

The chairs managed the review process and evaluated the entries in a similar manner as the previous year, but used specific measures for insight, presentation, interaction, creativity, flexibility, and novelty. There were two first and two second place awards. Teams led by the Iowa State University [START_REF] Hofmann | Boom and Bust of Technology Companies at the Turn of the 21st Century[END_REF] and Penn State

Final version -May 2007

12

University [START_REF] Chen | Space-Time-Attribute Analysis and Visualization of US Company Data[END_REF] took first place having answered all questions, while the Universität Karlsruhe [START_REF] Hoser | CPS-Company Positioning System: Visualizing the Economic Environment[END_REF] and Augsburg University [START_REF] Annerose | Antony Unwin Analyzing Company Data with Interactive Statistical Graphics[END_REF] provided strong answers and received second place prizes.

The first place winners took two different approaches.

The team from Iowa State University provided a classic information visualization approach and highlighted the important role of the analyst [Hei05]. Their system did not have high end visualizations but their analyses led to answers for all questions (Figure 21). Their focus on "boom and bust" was right on target and very elegant. The Penn State University and University of South Carolina team looked at the problem from the point of view of a GIS time-based system and used many of their map visualization tools to explore cross-state patterns (Figure 22) [Jin05]. Their system was the most novel with superb interaction. This of course highlighted the duality nature of the data set. One could look at it strictly from the information visualization point of view or from the GIS point of view and thereby harness a great deal of knowledge and techniques from the mapping and geospatial fields.

The second place winners had strong answers. The Company Positioning System from the Universität Karlsruhe was visually stimulating, and had high scores on interaction and novelty (Figure 23) [START_REF] Hoser | CPS-Company Positioning System: Visualizing the Economic Environment[END_REF] while the team from Augsburg University used interactive statistical graphics to derive and present their answers (Figure 24) [START_REF] Annerose | Antony Unwin Analyzing Company Data with Interactive Statistical Graphics[END_REF]. Their interactive solutions were insightful. All in all the four winners covered a broad spectrum of techniques for information visualization solutions. In all cases we found that statistical and computational methods played a key role. The data was just too large for simple human consumption thereby putting visualization in a collaborating role with analysis. Much processing of the data took place and all contestants used coordinated views to answer the questions.

Final version -May 2007 13 4.6 Lessons Learned from the Three Contests

About the evaluation process

The contests illustrated the difficulty researchers have at giving evidence that their tools can effectively perform the tasks. Demonstrating the power of a tool can be difficult. Researchers are trained to describe their tools' novel features more than illustrating them with convincing examples using real data.

In 2003 participants barely reported any insight at all. Everyone was focused on the description of their tools. By 2004 more participants (not all) were able to provide insights. In 2005 insights were more common. Providing good examples of insight reports seemed to help the participants.

Half of the participants were students who built their own tools. These tools were not as polished as industrial products or well developed research systems. Providing benchmarks that fit student project's sizes seems important to the success of future contests. In 2003, we provided large data sets with some meaningful subsets. In 2004 the data set was not very large. However, in 2005, the dataset was large and there was no subset provided so the number of participating students dropped.

The evaluation process is time consuming and looked at by some of the chairs as a daunting task. Ideally one could compute metrics and add these up assuming independence to get a summary score, but since the questions are open-ended and there is no known "ground truth" human evaluation is still required. Even though there were a limited a number of specific tasks, these can be interpreted differently and of course participants did interpret them in many ways. We always felt as if we had to compare non-comparable processes and results.

Evaluating the results remained a subjective activity. After seeing the submissions, the 2003 and 2004 contest reviewers decided to classify the teams into three categories based on increasing insight: no evidence of insight gathered using the tool, some insight, and much insight, i.e. worthy of a first place. On the other hand the 2005 contest used more than 6 categories. This helped with discussions but required a great deal more time reviewing the materials, and it is not clear that it resulted in better selections. This is an issue, as we do not have a good way to evaluate insight without ground truth. The recently completed visual analytics contest associated with the IEEE VAST 2006 symposium [VAS06; Gri06] provided more evidence for the benefits of having ground truth. It introduced a promising approach for the development of automated tools and the collection of metrics.

Another challenge in judging is the required repetitive, often mostly visual evaluations of similar entries. It can be very hard to remember "who did what" or "who had this insight". An insight implies novelty of the finding, so reviewers might more positively weigh a reported insight the first time they encounter it and undervalue it later on when reported by another team.

Having a shared environment to manage the tracking and comparison of the reported insights would be every helpful.

Videos were extremely important. Without them it would have been impossible to understand how most tools worked and what process was used to answer the questions.

With videos, interactions become understandable. Verbal comments on the videos were indispensable in explaining what the participants were doing. This is quite different than simply reviewing a paper. On the other hand video requires the reviewers to remember key points as it is quite difficult to scan a video quickly to refresh one's memory. Dealing with videos was also very time consuming. For the first two contests we were flexible about the format of the videos submitted but this created problems such as finding format converters or hunting for missing codecs. For the 2005 contest we required a single format which simplified the process. Requesting multiple short videos may also prove helpful.

Data preparation

With the InfoVis 2003 contest we attempted to provide real data and tasks while trying to narrow the problem to one data type (trees) and three representative tree types. The contest taught us that the problem was still too large for a contest and that the vague nature of the tasks made it impossible to compare answers effectively. In contrast the 2004 contest had only one dataset, much fewer tasks and a more structured reporting format. Nevertheless, the open-ended nature of realistic tasks and the diversity of interpretations and approaches still made judging the submitted entries a challenge.

We felt that the time to generate a reasonably clean data set was too large, around 1000 staff-hours each year with a similar large figure for the VAST 2006 and 2007 contest data sets. This is a serious issue for the development of benchmarks. Domain experts should be solicited for cleanup and experimentation on various task solutions should be attempted before the data is released. We hope that industry groups or government agencies wishing to see more research conducted on specific data of interest to them will take on the burden of developing the benchmarks datasets or support groups to do so. Most participants reported working very hard to prepare their submission. Many acknowledged that it pushed them to improve and test their tools. Students were encouraged to work on the contest as class projects sometimes continued beyond the semester. Some wanted to test their PhD research. Small companies reported appreciating the exposure.

In all three contests we gave small prizes from various sponsors to the first and second place teams. All appreciated them especially the students who received gaming systems. We also presented all winners with certificates. Participants appreciated being able to mention the award and the resulting publication (even if a minor one) in their resumes. On the other hand, in our polling (see below) some tenure-track faculty reported being interested in the contest but preferred focusing on writing full papers instead. Providing better incentives should help attract more participants.

We felt that we should have chosen the data set and plan for an earlier release date. Given that we ran into errors and noise in almost all the data sets, having more time might help clean the data and prepare better tasks. Pushing the deadline further into the late summer would allow summer interns to work on the contest, but would reduces the reviewing period dramatically, which doesn't seem feasible for a conference in October.

Many people downloaded the dataset without submitting results and we collected names and emails (for 2004 and 2005). The chairs performed an informal survey of those that had downloaded the 2005 contest to see why there were so few participants. The participants stated that there were no problems with the data set or questions, that the data set was a great data set to show system and tool capabilities, and that all had enjoyed the process and would do it again. Most expressed that they wanted a better organized website, automated email on data or news updates, and would have preferred the data in a database format. Some expressed strong interest in splitting entries into commercial and academic categories. Four of the non-participants stated that the requirement to attend the conference hindered their participation and most expressed that they were too busy in their company to tackle such a project. Several expressed a desire for mini-questions such as "find a more elegant way to look at …" There was one recurring theme which all participants and non-participants expressed and that was the need for more time. That was the reason the 2006 contest data was made available at the 2005 conference, but again the participation did not increase substantially so other factors may be more important, such as incentives, or personal interest in the data and problem proposed.

Running a successful event at the conference

For all three contests we had a full session at the conference where we summarized the results and had some authors present their submissions. In 2003 only the first place authors presented and we summarized the second place submissions. Many commented that it would be better to have shorter presentations but allow more presenters to speak. The following year we arranged for all first and second place winners to present with the second place ones having only 2 minutes. This format was very well received. We specified tasks that presenters should focus on so the attendees could better compare the different entries, at the cost of not seeing every feature of the tools. We found that handing out the awards rapidly and keeping photos to a minimum (a group picture at the end of the session) was preferable. This left more time for the presentations and still gave a festive atmosphere to the event. All winners were also given a chance to have a poster displayed during the normal poster session.

Impact and repository

A contest is only a first step. The revised materials provided by the authors and the datasets have to be made available after the event to be actively used. We are keeping the contest pages active and have made the submissions available in the InfoVis repository hosted at the University of Maryland [Bmr06]. On that website we encourage researchers and application developers to continue using the datasets and tasks. We hope that others will add the results of their analysis to the repository thereby enriching it and providing a more comprehensive review of visualization techniques available. Appropriate advertising and promotion of the repository should help. More importantly we believe that an infrastructure is needed to support and facilitate the use of those datasets (e.g. by providing multiple views and versions for different uses) by monitoring the usage of the repository (downloads, visits) and gathering information regarding its impact (e.g. follow up to collect researchers' reports on their use of the datasets, or collections of the citation). In 2003 we did not monitor any activity; in 2004 we know that the dataset was downloaded over 350 times by the end of 2006, and we found 16 papers that mentioned using the 2004 contest datasets (e.g. [Bor06b, [START_REF] Faisal | Subjective Information Visualisations[END_REF][START_REF] Minghim | Content-based Text Mapping using Multi-dimensional Projections for Exploration of Document Collections[END_REF]), but a more thorough follow-up would be needed to evaluate the impact of the contest. The 2005 contest is now making its appearance in publications ([Unw06], [START_REF] Chen | Visual Inquiry Toolkit An Integrated Approach for Exploring and Interpreting Space-Time, Multivariate Patterns[END_REF], [START_REF] Chen | Supporting the Process of Exploring and Interpreting Space-Time, Multivariate Patterns: The Visual Inquiry Toolkit[END_REF], [START_REF] Guo | A Visual Inquiry System for Space-Time and Multivariate Patterns (VIS-STAMP)[END_REF]). Running an information visualization contest is fairly taxing and volunteers who Final version -May 2007 15 run the contest tend to run out of steam by the end of the contest, making reporting of lessons learned and followup activities difficult. We believe that modest support for a long term coordinated evaluation program and infrastructure would greatly increase the impact of the information visualization contest and the benchmark repository.

Recommendations

We now propose a set of practical recommendations for tools researchers, contest organizers, and visualization users.

For potential contest participants

Find partners if you cannot complete the task alone Suggest next years' topic if this year doesn't fit your research or directions Encourage your sponsors to prepare data sets for contests Ask questions (you rarely ever do) Participate even after the contest by adding entries to the repository For researchers Exercise your tools with the benchmarks datasets and tasks Participate in contests (those who participate report many benefits) Leverage existing contests to test your new metrics or novel evaluation methods For organizers of contests Provide incentives for contestants that go beyond gifts (e.g. negotiate full papers in journals or plan early for the inclusion of the summaries in a digital library). Announce those incentives in the call. This may be the most driving force for academics, students and faculty alike. Facilitate student participation (choose the schedule carefully, advertise in classes, include smaller problems, plan for travel grants or student volunteer positions) Provide templates and examples for participants to report their findings in a structured fashion Set up a registration process to monitor downloads and plan to monitor and encourage usage after the contest. Plan to write reports in a timely fashion. Use datasets with established ground truth Balance the diversification of problems chosen with some continuity from year to year to allow participation to build up Although a contest is an artificial testing situation the information visualization contests encouraged participants to thoroughly exercise their systems over a long period of time, mimicking a fairly realistic analysis process.

Participants were asked to report on the insights gained from exploring the data. The impact of contests is most obvious for those who participate and those who can compare their results at the conference but the datasets, tasks and results remain available after the contests thereby extending their impact. Contests can be used by developers to exercise their tools and identify missing or weak features, and to compare their tools' effectiveness with those of others. Evaluators can use contests to evaluate, explore and enrich their testing procedures with complex tasks. Developers and evaluators then will have baseline results with which to compare their own results. We hope that these data can also be used in controlled experiments, and that the other more specific lists of tasks used in those experiments will be added to the repository for further reuse.

Benchmarks are difficult to create, promote and use. Our belief is that we are developing solid and evolving benchmarks and are beginning to understand how to better evaluate submissions. Good benchmarks must be real (witness the success of TREC and CAMDA, and the excitement generated by the VAST contest) to both draw the audiences and participants and to strongly push the technology curve. Good benchmark tasks must be openended to provide for the flexibility in solutions. We know that this makes the results more difficult to measure analytically but this is realistic. We need to accept that more human evaluation will be required in the future and evolve a collection of volunteer judges. These contests continue to demonstrate the challenges of benchmark design and especially of system and tool evaluation.

Final version -May 2007 16 By making the results of analyses available to the community in a benchmark repository we provide a set of comparison baselines for developers. Even though teams did interpret our tasks in many different ways, making comparison difficult, we feel strongly comparison with the same data set and tasks are useful and important. Plus the different number of interpretations could be reduced by more explanation.

The integration of analysis is now more necessary as data sets are more complex, large, and coming from diverse sources. The identification of anomalous patterns of data from phone calls, from bank transactions, and from news articles requires new techniques and strong analytical tools. We believe that such data sets and competitions will continue to encourage the community to work on difficult problems while building baselines of comparable tasks and datasets.

Figure 1 :

 1 Figure 1: A collage of sample screens from the InfoVis 2004 contest submissions which illustrates the diversity of visualization methods used to address a task and highlights the difficulty of comparisons.

Final

  their systems during the poster session. Finally two page summaries were published in the adjunct proceedings and all materials archived in the Information Visualization Benchmark Repository [Bmr06].

Figure 3 :

 3 Figure 3: General comparison of 1 st and 2 nd place entriesTreeJuxtaposer[START_REF] Treejuxtaposer | Poster Compendium of IEEE Information Visualization[END_REF] (Figure4) submitted the most convincing description of how the tasks could be conducted and the results interpreted (see also[START_REF] Munzner | TreeJuxtaposer: Scalable tree comparison using Focus+Context with guaranteed visibility[END_REF]).Zoomology[START_REF] Hong | Zoomology: Comparing Two Large Hierarchical Trees[END_REF] (Figure5) demonstrated how a custom design for a specific single dataset could lead to a useful tool that addressed many of the tasks satisfactorily. InfoZoom [Spe00] (Figure6) was the most surprising entry. This tool had been designed for manipulating tables and not trees. However the authors impressed the judges by showing that they could perform most of the tasks, find errors and provide insights in the data. The three second-place entries showed promise but provided less information to the judges on how the tasks were conducted and the meaning of the results. EVAT[START_REF] Auber | EVAT -Environment for Visualization and Analysis of Trees[END_REF] (Figure7) demonstrated that powerful analytical tools complementing the visualization could assist users in accomplishing their tasks. TaxoNote[START_REF] Morse | Comparison of Multiple Taxonomic Hierarchies Using TaxoNote[END_REF] (Figure8) demonstrated that labeling is an important issue making textual displays attractive. The Indiana University submission[START_REF] Sheth | Treemap, Radial Tree, and 3D Tree Visualizations[END_REF] (Figure9) illustrated the benefits of toolkits (e.g. [Bor06; Fek04] by quickly preparing an entry combining several tools, each accomplishing different tasks.

  Figure 4: TreeJuxtaposer [Mun03a]

Figure 7 :

 7 Figure 7: EVAT [Aub03]

Figure 10 :Figure 12 Figure 13 :Figure 14 :Figure 16 :Figure 19 :Figure 20 :

 10121314161920 Figure 10: Node-Link diagram, from Indiana University [Wei04]

Figure 21 :

 21 Figure 21: Iowa State University [Hof05]

Final

  of participants Participating in the contest takes time and motivation.

Acknowledgements

For all the contests, we thank the organizers of the IEEE Visualization InfoVis Symposium, in particular John Dill, Tamara Munzner and Stephen Spencer, members of the InfoVis community for their intellectual stimulations, and the participants without whom there would be no contest. We would also like to thank the numerous students for their help in extracting the metadata, cleansing the data set, and producing a richly usable data set: Caroline Appert (Université Paris-Sud, France) and Urska Cvek, Alexander Gee, Howie Goodell, Vivek Gupta, Christine Lawrence, Hongli Li, Mary Beth Smrtic, Min Yu and Jianping Zhou (University of Massachusetts at Lowell). After the first release of the datasets many others offered their help, including Jeff Klingner from Stanford, Kevin Stamper, Tzu-Wei Hsu, Dave McColgin, Chris Plaue, Jason Day, Bob Amar, Justin Godfrey and Lee Inman Farabaugh, from Georgia Tech, Niklas Elmqvist from Chalmers, Sweden, Jung-Rung Han, Chia-Ning Chiang, and Maylis Delest from the Université de Bordeaux. We want to thank Cynthia Parr, a biologist at the University of Maryland and Elie Dassa from the Institut Pasteur in Paris for helping us create the 2003 datasets. We also thank Anita Komlodi from UMBC and Cyndy Parr for participating in the review process. We thank ACM and IEEE and in particular Mark Mandelbaum and Bernard Rous for helping make the 2004 data available and working with us to prepare the dataset, Shabnam Tafreshi for help with the website, and finally but not least Paolo Buono from the University of Bari, Italy, for participating in the review process. We thank Michael Best (University of Massachusetts at Lowell) for working with us on releasing the technology company data for the 2005 contest. We thank the sponsors who provided prizes, The Hive Group, ILOG and Stephen North in 2003, Kitware, Nvidia and Microsoft research in 2004. We also thank Sharon Laskowski for working with Catherine Plaisant on the evaluation section of the NVAC research agenda [Tho05] which helped refined some of the sections of this paper and lead to Figure 2.


H Gailhanou

P Blanc

A Lassin

P Vieillard

R Denoyel

E Bloch

S Gaboreau

E Giffaut

Thermodynamics of hydration of MX80-Na: an experimental study of the hydration energies

1. BRGM, 3 Avenue Claude Guillemin, BP 6009, F-45060 Orléans Cedex 2 (h.gailhanou@brgm.fr) 2. CNRS-IC2MP-UMR7285, HydrASA, 5 Avenue Albert Turpain, F-86022 Poitiers Cedex (philippe.vieillard@univ-poitiers.fr) 3. MADIREL, Aix-Marseille Université, CNRS-UMR 7246, F-13397 Marseille Cedex 20 (emily.bloch@univprovence.fr) 4. ANDRA -Parc de la Croix Blanche, 1-7 rue Jean Monnet, 92298 Châtenay-Malabry Cedex 2, France Hydration properties of swelling clay minerals may be very variable depending on the chemical composition of the clay, on the nature of the interlayer cations and on the interlayer charge [START_REF] Berend | Mechanism of adsorption and desorption of water vapor by homoionic montmorillonites : 2. The Li+, Na+, K+, Rb+ and Cs+exchanged forms[END_REF][START_REF] Vieillard | Hydration thermodynamics of the SWy-1 montmorillonite saturated with alkali and alkaline-earth cations: a predictive model[END_REF]. The Wyoming smectite has been largely studied, notably for assessing its hydration behavior as a function of the interlayer cations, in connection with its structural characteristics [START_REF] Ferrage | Investigation of smectite hydration properties by modeling of X-ray diffraction profi les. Part 1. Montmorillonite hydration properties[END_REF]Salles et al., 2007). In the present work, carried out as part of a collaborative Andra/BRGM/HydrASA research program for ThermoChimie project, we propose an original experimental study, based on adsorption and desorption isotherms performed on MX80 clay samples. The goal is to determine energetic contributions to the reactions of hydration, which have been revealed to be non-negligible with respect to the stability of the clay minerals (Gailhanou et al., submitted). In particular, the present work addresses the problems of the hysteresis loop between adsorption and desorption isotherms and of the irreversibility of hydration reactions. This is directly related to the application of classical thermodynamics to the hydration reactions of clay minerals.

In a first stage, an experimental study is dedicated to better understand the origin of the hysteresis loop which is systematically observed for the adsorption-desorption isotherms at 25°C. The development of the hysteresis loop has been studied by considering several kinetically related parameters: stabilization periods, temperatures (from 25°C to 60°C) and hydration steps (Figure 1). No sensible change was observed in the hysteresis loop. Therefore, the amount of adsorbed water depends on the followed reaction pathway (adsorption or desorption). The variations in microstructures and in the distribution of hydration layers (0/1/2 water layers; [START_REF] Ferrage | Investigation of smectite hydration properties by modeling of X-ray diffraction profi les. Part 1. Montmorillonite hydration properties[END_REF] as a function of relative humidity (RH) could provide a possible explanation for this phenomenon. Amount of adsorbed water (mg/g of sample)

Relative Humidity (%)

Figure 1. Adsorption/desorption isotherms at 25°C on Na-saturated MX-80. Influence of the stabilization period between two injections of water, on the hysteresis loop.

Apparent Gibbs free energies of hydration have been calculated from both the adsorption and desorption isotherms [START_REF] Tardy | A method of estimating the Gibbs free energies of formation of hydrated and dehydrated clay minerals[END_REF]. Figure 2 presents the Gibbs free energies of formation of adsorbed water ∆G°f ,H2Oads , obtained by considering the reaction H 2 O (l) = H 2 O (ads) , calculated from the adsorption and the desorption isotherms. The calculation method has been optimized at very low RH in order to improve the accuracy of the results. The integration of P/P 0 values has been performed on both the adsorption and desorption branches following the corresponding reaction pathway. The plots of ∆G°f ,H2Oads from the adsorption and desorption isotherms diverge from each other at low RH, because of the irreversibility of the hydration reaction. However, surprisingly, they are very similar for RH> 55%. This implies that, for this RH range, the Gibbs free energy of hydration is independent of the reaction pathway, and consequently, may be used for classical thermodynamic calculations. However, comparison between the data obtained from calorimetric measurements at two highly hydrated states (squares in Fig. 2; obtained from solution calorimetry for enthalpy and low temperature adiabatic calorimetry for entropy, Gailhanou et al., submitted) and the aforementioned values at the corresponding RH, shows a non-negligible discrepancy, despite a rather large uncertainty associated with the calorimetric values (Figure 2). This discrepancy is not yet explained, but could originate partly from the contribution of energies of dilution in the calorimetric measurements.

Figure 2 .

 2 Figure 2. Apparent Gibbs free energies of formation of water adsorbed in Na-saturated MX-80, at 25°C, calculated from adsorption (open circles) and desorption (black circles) isotherms and from calorimetric measurements (squares).


P Blanc
email: p.blanc@brgm.fr

P Vieillard

H Gailhanou

S Gaboreau

A Lassin

E C Gaucher

C Fialips

E Giffaut

EXTENDING THE PREDICTION OF THE THERMODYNAMIC PROPERTIES OF CLAY MINERALS TO THE TRAPPING OF TRACE ELEMENTS

Introduction

The thermodynamic properties of clay minerals, which control the stability of these minerals in solution, are still a matter of debate in spite of recent advances (Gailhanou et al., submitted). This is especially the case for the minerals that may structurally include trace elements and potential radionuclides such like Ni, Cd, Co, Cr, Mn, Pb, … The usual methods developed in order to predict thermodynamic properties are parameterised using a given set of minerals. For clay minerals, the latter are mainly composed by Si, Al, Fe and Mg, apart from the alkalis elements [START_REF] Chermak | Estimating the thermodynamic properties (∆G°f and ∆H°f) of silicate minerals at 298K from the sum of polyhedral contributions[END_REF], which means that predictions are limited to minerals whose layers are composed by Si, Al, Fe and Mg. At the vicinity of H&ILW disposal cells, the possible interactions between clay rock or engineered barrier and waste degradation products can result in the appearance of clay minerals that may structurally include radionuclides within an irreversible trapping process. This work aims at proposing a method for predicting the thermodynamic properties of such minerals. [START_REF] Vieillard | Prediction of enthalpy of formation based on refined crystal structures of multisite compounds.1. Theories and examples[END_REF] has developed a methodology of estimation based on the difference of electronegativity by considering three scales of values of the parameter ∆ H O = (M z+ clay ) in the three sites of phyllosilicates. We have considered the work of [START_REF] Vieillard | Prediction of enthalpy of formation based on refined crystal structures of multisite compounds.1. Theories and examples[END_REF] that originally applies to the estimate of ∆H 0 f and extended it to the estimate of Cp(T) , S 0 and V. Some popular estimate methods [START_REF] Chermak | Estimating the thermodynamic properties (∆G°f and ∆H°f) of silicate minerals at 298K from the sum of polyhedral contributions[END_REF] are based on the hypothesis that the thermodynamic property of a mineral can be obtained by combining the properties of its components. An improvement of this principle had consisted in decomposing minerals into their polyhedral components [START_REF] Chermak | Estimating the thermodynamic properties (∆G°f and ∆H°f) of silicate minerals at 298K from the sum of polyhedral contributions[END_REF]. Now, we can write the fictive solution equilibrium with a basic polyhedral component M x O y as:

Theoretical principle and selection of calibration phases

y x z aq O M yO xM = + = +
and assumming the entropy of this fictive reaction is zero, we can define a ∆ S O = parameter as:

( ) ( ) ( ) ( ) y M S x O M S O S M O z aq y x z clay S + = + = ⋅ - = = ∆ 0 0 0 .
The value for the oxide analog of the polyhedral unit is obtained by implementing S 0 of the oxide in the S 0 (M x O y ) term. We have also defined, from the same reasoning, similar parameters for heat capacity and volume of the basic polyhedral components:

( ) ( ) ( ) y M Cp x O M Cp M O z aq y x z clay Cp + + = ⋅ - = ∆ ; ( ) ( ) ( ) y M Cp x O M V M O z aq y x z clay Cp V + + = ⋅ - = ∆ , .

Results and discussion

On Figure 1, we have displayed, for entropy, the correlation obtained between calculated values of ∆ S O = for the polyhedral unit and for the oxide analog. A straight line and a secondorder function are obtained, for the interlayer and octahedral cations, respectively, with a good correlation coefficient. The implementation of the derived semi-empirical, first or second order relations allows to estimate the thermodynamic properties of a clay mineral, MX80 (Na 0.409 K 0.024 Ca 0.009 (Si 3.738 Al 0.262 )(Al 1.598 Mg 0.214 Fe 3+ 0.173 Fe 2+ 0.035 )O 10 (OH)2) in the present case, loaded by 6 radionuclides and to compare the values with the results obtained by Gailhanou et al. (submitted).The results can be expressed in terms of the concentrations for the elements Ni, Cd, Co, Cr, Mn and Pb and in terms of energetic potential with respect to the measurements performed by Gailhanou et al. (submitted).

Fig. 1 -

 1 Fig. 1 -Development of predictive capacity for entropy estimates