Daniel Hienert
email: daniel.hienert@gesis.org

Dennis Wegener
email: dennis.wegener@gesis.org

Siegfried Schomisch
email: siegfried.schomisch@gesis.org

Making Sense of Open Data Statistics with Information from Wikipedia

de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Nowadays, a mass of open data statistics is available from providers like Eurostat, the United Nations or The World Bank. Eurostat for example offers about 4,500 different statistics with a wide topical range from General and Regional Statistics, Economics and Finance, Population and Social Conditions, Industry, Trade and Services, Agriculture, Forestry and Fisheries, International Trade, Transportation, Environment and Energy or Science and Technology.

Visualization software on the Web, such as Gapminder [START_REF] Rosling | Visual technology unveils the beauty of statistics and swaps policy from dissemination to access[END_REF], makes use of these statistics and presents them in interactive graphics like diagrams or maps. Users, from laymen to politicians, can explore the data and find trends and correlations in order to "easily improve their understanding about the complex society" 1 .

However, statistics -even interactively visualized -are not self-explanatory. They reflect trends for a statistical indicator (what) in a certain time period (when) and for various countries (where). But, these trends often base on or are related to certain events in the topic of the statistic. From a user's perspective, one can ask why there is a certain trend. Which influential factors lead to the trend shown in the statistic or may be related? Additional information, topically related to statistics, can help users to interpret them, get background information and hints where to continue in the search process.

We have created a prototype user interface that gives access to thousands of open data statistics from The World Bank, Eurostat and Gapminder. Users can interactively explore these statistics in different diagrams and map views. Additionally, for the Gapminder statistics, we aim at finding topically related historical events to present them in a timeline.

However, there is not much meta-information about the statistic except the title or sometimes a category. Using only keywords from the title leads to very few results when querying related information like historical events. Therefore, we use a query expansion method based on the Wikipedia and the DBpedia corpus to expand a search query with additional related terms based on the statistic title. Statistical visualizations and a timeline with the related events found are then interactively connected in a user interface to let the user explore relationships and explanations for trends in the statistic. Groups of events found with query expansion can be hidden or revealed with facets. In a user test we evaluate if users are able to use the prototype for the exploration of statistical data and related information.

Section 2 gives an overview of related work. Section 3 presents our approach for the combination of open data statistics and topically related historical events. Section 4 presents a user study to examine the exploratory search process. We discuss the results and conclude in Section 5.

Related Work

A lot of different data providers give access to open data statistics with focus on the world development, e.g. the United Nations, Eurostat or The World Bank. Most datasets contain information on a statistical indicator (often with several dimensions) for countries over time. Gapminder for example aggregates data from several providers and offers about 500 different indicators for about 200 countries and with a temporal coverage beginning from 1800 AD till today. Some of these data providers have recently also developed web-based visualization tools for their statistical data. Most of these web applications use maps and line/bar charts to show indicator development over time and for different countries. Gapminder uses a different approach and shows two related indicators in a scatterplot. Similar to statistical software for scientists, such as SPSS, STATA or R, one can see correlations between these indicators. Gapminder additionally uses an animation to show the development of correlation over time for all countries of the world.

For the semantic modeling of events in RDF a number of ontologies is available, e.g. EVENT 2 , LODE [START_REF] Shaw | LODE: Linking Open Descriptions of Events[END_REF], SEM [START_REF] Hage | Design and use of the Simple Event Model (SEM)[END_REF], EventsML3 and F [START_REF] Scherp | F-a model of events based on the foundational ontology dolce+DnS ultralight[END_REF]. A comparison of such ontologies can be found in [START_REF] Shaw | LODE: Linking Open Descriptions of Events[END_REF]. Furthermore, there exist ontologies and systems for the annotation of events in the timeline. Gao & Hunter [START_REF] Gao | Publishing, Linking and Annotating Events via Interactive Timelines: an Earth Sciences Case Study[END_REF] make use of ontologies not only for the semantic markup of events, but also for the modeling of timelines, relationships between events and for the annotation of events of different timelines. Therefore, not only events can be referenced in the Linked Data Cloud, but also relations and annotations. In a web-based interface, users can search for geological events with a high impact such as earthquakes, tsunamis and volcanic eruptions, can analyze relationships and make connections between them with annotations.

The simple representation of events in timelines can be implemented with various web-based tools such as SIMILE 4 widgets. The use of timelines can cause user interaction difficulties, for example, if too many events are visualized and the user loses the navigational overview [START_REF] Kurihara | Flexible timeline user interface using constraints[END_REF]. Kumar et al. [15] propose a theoretical model for timelines on the storage and presentation layer with levels like content, operations, browsing and editing. An example implementation of this model allows the creation, visualization and browsing of bibliographic metadata. LifeLines [START_REF] Plaisant | LifeLines: visualizing personal histories[END_REF] utilizes the arrangement of several timelines in one view. Facets of personal records, either flat or hierarchic, can individually be selected by buttons or trees to give an overview or discover relationships. Other complex timeline visualizations like Semtime [START_REF] Jensen | Visualizing Complex Semantic Timelines[END_REF] or Sematime [START_REF] Stab | Sematime -timeline visualization of time-dependent relations and semantics[END_REF] display in a similar fashion several stacked timelines with additionally depicted time-dependent semantic relations between events. Advanced interaction techniques include hierarchical filtering and navigation like zooming or expanding sub-timelines. Sense.us [START_REF] Heer | Voyagers and voyeurs: supporting asynchronous collaborative information visualization[END_REF] is a web-based visualization system with an emphasis on the social and collaborative aspect. Users can share visualizations, e.g. a chart of US census data, and can discuss and comment on different states of the visualization. Bookmarking ability of different states and graphical annotation tools allow annotating and discussing certain data points or trends in statistical visualizations in order to make sense of the pure data. ChronoViz [START_REF] Fouse | ChronoViz: a system for supporting navigation of time-coded data[END_REF] is a system that uses the timeline metaphor for the simultaneous display of multiple data sources such as video, audio, logs, sensor data and transcriptions. Also here, users can add annotations with a digital pen and use them as anchor links. Aigner et al. [START_REF] Aigner | Visualizing timeoriented data-A systematic view[END_REF] give a systematic overview of visualizing time-oriented data based on categorization criteria on the time, data and representation level. They emphasize open problems and future work like multiple views of time-oriented data and their coordination. In this sense, some case studies exist which use Multiple Coordinated Views [START_REF] Baldonado | Guidelines for using multiple views in information visualization[END_REF] of temporal and other visualizations in domains like climate data [START_REF] Shimabukuro | Coordinated Views to Assist Exploration of Spatio-Temporal Data: A Case Study[END_REF] or medicine [START_REF] Aigner | Supporting Protocol-Based Care in Medicine via Multiple Coordinated Views[END_REF]. Timelines are linked to other views, so that selecting data in a timeline highlights data in other views or vice versa.

The search for relationships between pieces of information in different representations can be described by the model of Exploratory Search [START_REF] Marchionini | Exploratory search: from finding to understanding[END_REF]. Multiple iterations with cognitive processing and interpretation of objects over various media like graphs, maps, text and video is needed. The user has to spend time "scanning/viewing, comparing and making qualitative judgments" for the result of "knowledge acquisition, comprehension of concepts or skills, interpretation of ideas, and comparisons or aggregation of data and concepts" [START_REF] Marchionini | Exploratory search: from finding to understanding[END_REF]. Also, timelines can be part of an exploratory search process, for example, to use the time component to explore research articles in a timeline view [START_REF] Alonso | Exploratory Search Using Timelines[END_REF].

Query expansion is used to expand a search query with additional terms like synonyms, related terms, or methods like stemming or spelling correction. For very short queries this can increase the overall recall of a search, and more relevant information objects can be found [START_REF] Voorhees | Query expansion using lexical-semantic relations[END_REF]. Query expansion based on thesauri, ontologies, co-occurrence analysis or other knowledge sources have been utilized in digital libraries of different domains like e.g. the social sciences [START_REF] Mayr | Cross-concordances: terminology mapping and its effectiveness for information retrieval[END_REF] or the medical domain [START_REF] Holzinger | Quality-based knowledge discovery from medical text on the Web Example of computational methods in Web intelligence. Advanced Techniques in Web Intelligence -Quality-based Information Retrieval[END_REF]. Also, the Wikipedia corpus has been used as a database for this purpose [START_REF] Medelyan | Mining meaning from Wikipedia[END_REF].

System Prototype

We have created a research prototype 5 for the visualization of open data statistics and related events. In the following, we will describe (1) the visualization of statistical data in several views, (2) the retrieval and visualization of related historical events.

Visualization of Statistical Data

The web application is implemented in PHP and the visualization and interaction component is realized with HTML5, the canvas element and JavaScript (based on a previous prototype presented in [START_REF] Hienert | Web-Based Multi-View Visualizations for Aggregated Statistics[END_REF]). We have integrated statistical data from Eurostat, The World Bank and Gapminder. The Eurostat dataset contains 4,545 indicators, The World Bank 3,566 and Gapminder 498.

Because the prototype already contains about 8,500 statistics from different providers, it is difficult to provide an overall hierarchical categorization. Therefore, the web application provides a query interface for searching statistics by title. The user can enter keywords and an autocomplete function initially suggests matching statistics. By clicking on the link the indicator is chosen and instantly visualized.

Similar to existing solutions, the statistical data is displayed in various graphical views: a map, a bar chart and a line chart (see Figure 1). On a Google map, indicator values for all different countries and a given year are visualized as circles of different sizes and colors at the country location. The values of the indicators are visually encoded in two dimensions: (1) the higher the indicator value, the greater the circle radius, (2) the higher the indicator value, the more the color shade goes into the red spectrum, low indicators values are encoded with bluish tones. This way, the user can instantly see where high values are clustering. For example, for the indicator Fertility, one can see at first glance that in 2011 still more children per woman were born in Africa than in Europe. The map has standard interaction facilities like zooming and panning, so the user can browse to the region of interest.

The bar chart shows indicator values for each country and a given year in descending order. Countries with high values are at the top, countries with low values are at the bottom of the list. As 200 countries are listed, the user can scroll the list. The width of the horizontal bars allows seeing differences between countries instantly, so the users can compare if there are only small or big differences between two countries in the list. Similar to the map, the bars are color-coded, so one can see instantly if countries have higher or lower values than the Median. Next to the bar chart, there is a line chart that shows the distribution of indicator values over time. Users can select a country from the chart legend or from the overall filters which results in the line showing up in the graph. Hovering with the mouse over a data point shows time, country and value at the top of the chart. Lines are color-coded similarly in the legend and in the graph.

At the top of the page, there exists a control panel for filtering all views simultaneously by one or several countries and a year. From a list, users can select individual countries which are color-coded according to countries in the line chart. In addition, there exists a slider for selecting the year. This way, the selected year for the map and bar chart view can be chosen and the user can browse through the time. With a play button all views can be animated to show data for the actual year.

Retrieval of Related Historical Events

A goal in this paper is to retrieve and display historical events that are related to a statistical indicator and therefore, link the following two datasets:

1. The Gapminder dataset includes 498 different statistical indicators. This dataset is maintained regularly, topically well chosen, includes many data sources and covers all world countries with a wide temporal coverage from 1800 till today. 2. The historical events dataset is based on a database that holds a large collection of historical events extracted from Wikipedia [START_REF] Hienert | Extraction of Historical Events from Wikipedia[END_REF] and can be queried via an API 6 . It has been shown that machine learning with features from DBpedia is a feasible way to achieve an automatic classification of the extracted events into categories [START_REF] Hienert | Automatic Classification and Relationship Extraction for Multi-Lingual and Multi-Granular Events from Wikipedia[END_REF]. The outcome of the effort was a dataset with about 190,000 historical events covering different languages and including category information. The data subset used in this paper contains 37,859 yearly English events from 300 BC to 2013. An important requirement is that both datasets have a focus on years as a temporal reference unit. Statistical indicators are resolved for years; historical events are chosen by the Wikipedia community as an important fact for a year's history.

The statistic title or its category only provide weak information for the retrieval of related events. For example, querying the historical database for the statistic "Children per woman (total fertility)" with the keyword "fertility" leads to 0 results. The query has to be expanded with strongly related concepts to find more events and to get a higher recall. For "fertility", concepts like "contraception", "birth control", or "syphilis" must be found to retrieve historical events that have an explanatory character for trends in the statistic. The two data sources are connected in the following way (see Figure 2):

Step 1: For each of the Gapminder statistics, the title is preprocessed by removing units and stop words/characters.

Step 2: Based on the preprocessed title, the Wikipedia API is queried for an article page with a matching concept. We inspected the top 10 search results and manually selected the semantically best fitting page for each statistic. As a result, we got a mapping to a chosen Wikipedia article for each statistical indicator. All in all, this includes 144 distinct articles, as some of the statistics belong to the same Wikipedia concept (e.g. to the Wikipedia article "Development aid"). Because statistic titles from Gapminder are really short, we decided to manually select the Wikipedia article to guarantee a well-chosen concept with balanced properties of abstractness level, size, number of links etc.

Step 3: As a next step, we want to find related concepts for the mapped concepts. Therefore, we have implemented a web service 7 that returns related concepts for an input term. The service queries in/outlinks from Wikipedia via the Web API as well as broader/narrower terms and category information from DBpedia [START_REF] Auer | DBpedia: A Nucleus for a Web of Open Data[END_REF] via SPARQL endpoint. For each concept, the semantic relatedness (SR) to the original term is computed. For doing so, we use the Normalized Google Distance (NGD) formula [START_REF] Cilibrasi | The Google Similarity Distance[END_REF], but instead of taking hit counts from a search engine we use hit counts from the Wikipedia full text search. In a separate evaluation with evaluation datasets included in the sim-eval framework [START_REF] Panchenko | A study of hybrid similarity measures for semantic relation extraction[END_REF] we found that this approach achieved a Spearman correlation up to 0.729 for human judged datasets and P(20) up to 0.934 for semantic relation datasets. We use all concepts with a SR > 0.3 as keywords for the following step, which filters very broad or non-related concepts. This way, we could compute 77,202 related concepts for 498 statistics with on average 155 keywords per statistic.

Step 4: In the last step, we query events based on the concepts found in the previous step. In detail, we query the Historical Events-API for all events that include at least one of these keywords in the event description and lies within the time interval of the statistic. For all 498 statistics, querying only the Wikipedia equivalent for a statistic returns in sum 8,450 events, which means on average about 17 events per statistic. Using the presented approach for query expansion, for all statistics in sum 137,921 (not distinct) events are returned, which means on average about 279 per statistic. Based on the returned events a timeline is build. To illustrate the different steps, we present two examples related to the topics fertility and earthquakes.

Example "Fertility":

The title of the statistic from Gapminder in the first example is "Children per woman (total fertility)". The title was mapped to the Wikipedia concept "Fertility". Querying related concepts returns 61 keywords with an SR value higher than 0.3 (compare Table 1). These keywords were then used to query the Historical Events-API which results in 22 matching events.

Example "Earthquake": The title of the statistical indicator in the second example is "Earthquake -affected annual number". For this statistic, the Wikipedia article "Earthquake" was selected. 132 related concepts were returned with an SR value higher than 0.3. For these keywords the Historical Events-API returned 1,008 events.

Visualizing Related Historical Events

Below the area that visualizes statistical data, a scrollable timeline shows queried historical events in a timeline (see Figure 1, at the bottom). The individual events consist of a date, keywords, a description and, if available, a thumbnail. Links in the event description allow browsing to Wikipedia articles and to read details there.

Events for a certain keyword can be filtered with a select box above the timeline. The line chart that shows the indicator distribution over time for certain countries and the timeline are linked by a brushing-and-linking mechanism. When a user brushes over a data point in the line chart, the timeline automatically scrolls to events from the same year and highlights them with a yellow background or vice versa.

User Study

We have conducted a user study to examine the following research questions:

• Can participants use the interface intuitively? • How do users normally search for additional information of trends in a statistical indicator? • Can our approach of showing related historical events in a timeline help users to find interesting background information and starting points for further exploration? • Which advantages and limitations does our approach have?

Method & Participants

The participants were asked to carry out a set of tasks in the prototype and to fill out a questionnaire after each task with their actions performed, results found, time required, experienced difficulty level and comments. First, the users could familiarize themselves with the environment for 5 minutes. After that, they had to complete three different tasks. Finally, the users were asked to evaluate the pros and cons of our approach and to assess the overall scenario. The group of participants included eight male researchers and one female researcher, aged 27 to 40 (mean: 31 years). All had a graduate degree in computer science like Master or similar. The participants were asked to rate their experience in dealing with web-based search on a five-point-scale.

The rating was 1.56 ("very good") with a standard deviation of 0.73.

Task & Questions

The participants had to handle the following tasks and answer questions in the questionnaire: For task 1, the users had to write down the response in a text field, answer how long the process took, assess as how difficult the task was perceived on a five-pointscale (2=very easy, 1=easy, 0=normal, -1=difficult, -2=very difficult) and were asked to give comments and suggestions. For task 2, the users had to record the search steps, the total time, the relevant information sources and a confidence score for the search result on a five-point-scale (1=very unsure, 2=unsure, 3=normal, 4=sure, 5=very sure). Furthermore, similar to task 1, they had to assess the difficulty and could give comments and suggestions. For this task, the timeline has not been visible in the UI to prevent any influences on the user. For task 3, the participants could then refresh the user interface and activate the timeline with a GET-Parameter in the URL.

Fertility trends across countries

Causes for the decrease in fertility after 1920 and after 1960 in the US and in the UK

Usage of the timeline

Analog to task 2, please find potential causes or contexts for the decrease in fertility rates in these countries from 1920 to 1940 and from 1960 based on the historical events displayed in the timeline and describe them briefly.

For task 3, users had to write down the response in a text field, enter a confidence score as in task 2 and to assess the difficulty and could give comments and suggestions as in tasks 1 and 2.

Comparison and evaluation of the two methods from tasks 2 and 3 and overall results

Please evaluate the overall scenario:

• What are the pros and cons of both search methods for the application scenario?

• Which search strategy would you favor for the given application scenario and why would you do so? • Was the integrated user interface including the graphical view of information of fertility statistics and historical events in a timeline helpful for answering the questions on the decrease of fertility after 1921/1960 in both countries?

The overall scenario could be rated with a five-point-scale (2=very helpful, 1=helpful, 0=normal, -1=not helpful, -2=not helpful at all) and we left room for general comments, suggestions and criticisms.

Results

After the participants had made themselves familiar with the user interface they could all (n=9) solve task 1 successfully. The average time exposure for sub task (a) resulted in 74 seconds, while answers for the country specific values (b) and (c) took on average 16 seconds (compare Table 2). Participants rated the difficulty level of the task on average with "easy" (1.00). Task 1 showed that the user interface could be easily adopted by users for filtering and read-off processes without further explanation. Nevertheless, some points of criticism and improvement were given in terms of better scrolling functionality in the country list, more precise handling of the sliders, better highlighting of the selected countries, more distinguishable colors and zooming functionality in the line chart. For task 2, the participants recorded 30 query steps in the questionnaire to find possible reasons for the decrease of fertility in the US/UK between 1920 and 1940 (a) and from 1960 (b). The majority of users used Google (23 times), followed by Wikipedia (6 times). Solving task 2 took 122s/118s per identified reason. Users stated Wikipedia (10 times) and other websites (18 times) as sources of information. On average, the participants had a normal confidence in the information found on the Web (3.19/3.29). The average difficulty level of task 2 was evaluated with "difficult" for (a) and "normal" for (b).

In task 3 the participants were faced with the same questions as in task 2, but now could enable and use the timeline in the user interface to find possible reasons. For the decrease in fertility from 1920 to 1940 in the USA/UK, participants named, e.g., nine times the event "1921 -Marie Stopes opens the first birth control clinic in London, England.", five times World War I and II (from several events) or two times "1930/07/05 -The Seventh Lambeth Conference of Anglican Christian bishops opens. This conference approved the use of artificial birth control in limited circumstances, marking a controversial turning point in Christian views on contraception." One user stated that a decrease in fertility is normal for industrial countries. For task a) the participants were confident for possible reasons ("sure", 3.67).

For the decreasing trend from 1960 the users stated, e.g., eight times "1960/05/09 -The U.S. Food and Drug Administration announces that it will approve birth control as an additional indication for Searle's Enovid, making it the world's first approved oral contraceptive pill.", two times "1958/12/31 -Based on birth rates (per 1,000 population), the post-war baby boom ends in the United States as an 11-year decline in the birth rate begins (the longest on record in that country)." or one time Vietnam War (from one event). For task b) the participants were confident in having found possible reasons (4.17, "sure"). The task needed 58 seconds per reason and was perceived as "normal" (0.39) difficult on average. For task 4, users could first compare "search on the Web" (task 2) and "search in the prototype" (task 3). The web search was advantageous to the users in terms of using familiar systems (Google, Wikipedia etc.), having access to all kinds of documents, sources and contexts, the possibility to compare the resulting information and the control of the search process. The disadvantages reported were the heavyhanded search methods to find the relevant documents or sources, the time consuming process, additional search steps, no single answers for complex questions, a lot of irrelevant information and the problem of how to translate the information need into a search query when no starting point is given.

For the search in the prototype, the participants stressed as pros that they have found the results faster and could see them directly. Furthermore, they assessed positively the single point of search as well as the integrated search environment and the synchronization between line chart and timeline. In contrast, they qualified negatively that the search quality in this system depends on the used data sources and that only Wikipedia documents were offered but no scientific studies or other information.

Seven participants would initially prefer the prototype from task 3 to get an overview over a topic and to get results quickly. Some participants stated that this information was enough; most participants explained that they would use the prototype for a quick overview and then use the web in order to get reliable and quotable data or for verification of the results of task 3.

The participants evaluated the search in the prototype tendentially as helpful, which became apparent in the average value of 0.50 ("helpful"). This was also stressed, although with constraints, in the textual evaluation. One participant thought the prototype to be convenient only for easy questions but not for complex issues in the social sciences, other users wished more data sources.

Discussion & Conclusion

The conducted user study gives first hints that the combination of numerical statistical data in different views and related information like events in a timeline in a single user interface can be fruitful. Participants were able to use the interface intuitively without further instructions. The results of task 2 show that participants search for related information of trends in statistics with search engines like Google and Wikipedia with a combination of what, when, and where keywords. It can be seen from the confidence values that the users were confident about having found related information. On the Web, they have access to all kinds of documents, but it was a time-consuming process and a lot of search steps were needed. Our prototype was preferred for a fast overview as a single point of search and a starting point, but, of course, did not include all sources a web search engine provides.

For the user test, we chose the very complex topic "fertility" in contrast to less complex topics like e.g. "earthquakes" (in a sense of how many other diverse complex aspects may have an influence on the indicator). Influential factors are difficult to determine for a normal web search user, because related concepts and keywords are not at hand. Table 1 gives an overview: While concepts for "fertility" are very diverse, a lot of concepts for "earthquake" include the term "earthquake" with combinations of different locations and dates. This makes it easy also for normal web search, where the query instantly leads to the matching Wikipedia article (e.g. for "2010 Haiti earthquake"). In contrast, querying for all aspects of fertility seems to be a harder process. If one is lucky, search queries like "fertility 1920s United Kingdom" (as performed by participants of our study) lead to documents were information and explanations for one location and one time period are included. However, this documents most times only provide very broad explanations for a concept and no further detailed information or links. For example, for fertility users found a document that contains information on the development of fertility in the UK for the last century. Here, influential factors like both world wars, economic depression (late 1920s) and influenza outbreak (after World War I) are described. However, the document did not include any further information or links. Instead, users have to copy and paste keywords into a search engine. Our approach tries to take off the burden of searching relevant keywords and querying related information. In this sense, we computed related concepts and used these to query the data source.

In the presented use case, we concentrated on the combination of statistics with historical events. Since these events are carefully chosen according to their importance on a world history view by Wikipedia users, they offer a good compromise between importance, abstractness, count and temporal coverage (compare [START_REF] Hienert | Extraction of Historical Events from Wikipedia[END_REF]). Of course, other similar data sources could be included such as news articles from the Guardian Open Platform8 or the New York Times API 9 . However, we found that these sources provide the majority of articles for the last three decades and the query with keywords returns a mass of only rather important articles. Here, further aggregation steps have to be applied. But also other related information types can be queried with the computed concepts like Wikipedia articles, web sites, studies and surveys from which the statistics are generated, videos, images etc. Another important aspect is that the search on the Web and the search in the interface are not complementary as proposed in the user study, but users can use links in the timeline to continue with their web search.

At the end, it is always a trade-off between presenting too much or too little information, and carefully choosing important information based on only unsubstantial query information like statistic title (and related concepts), country and time.

Fig. 1 .

 1 Fig. 1. Prototype with statistical visualizations and a timeline with related events for the statistic "Children per woman (Total fertility)" from Gapminder. Hovering the mouse over the data point United Kingdom -1921 scrolls to and highlights events for the same year in the timeline.

Fig. 2 .:

 2 Fig.2. Process steps to retrieve related events for a statistic.

  Briefly describe the development of fertility rates in the United States of America and in the United Kingdom from 1921 to 1940 as shown in the line graph (a). What are the values for each country in the years 1926 (b) and 1937 (c)?

  (a) Find possible reasons for the decrease of fertility in the United States of America and in the United Kingdom from 1920 to 1940. (b) Find possible reasons for the decrease of fertility in the United States of America and in the United Kingdom in 1960. Try to find information outside the given prototype using other sources, e.g. Google. Do not spend more than 5 minutes for each subtask (a) and (b).

Table 1 .

 1 Top 20 Wikipedia concepts including values for semantic relatedness for "fertility" and "earthquake".

	Links for Fertility	SR	Links for Earthquake	SR
	Fertility rate	0.714 Tsunami	0.693
	Total fertility rate	0.657 Seismic	0.651
	Infertility	0.638 2011 Tohoku earthquake and tsunami	0.584
	Sperm	0.622 Seismology	0.581
	Crude birth rate	0.590 Epicenter	0.572
	Contraception	0.588 Aftershock	0.566
	Fertile	0.577 Disaster	0.554
	Fertilization	0.576 2004 Indian Ocean earthquake	0.549
	Gestation	0.572 2010 Haiti earthquake	0.543
	Uterus	0.571 1906 San Francisco earthquake	0.529
	Fertility clinic	0.561 Northridge earthquake	0.523
	Ovary	0.559 Volcano	0.521
	Menstrual cycle	0.558 Megathrust earthquake	0.518
	Menopause	0.552 Subduction	0.512
	Pregnancy	0.544 Seismologist	0.510
	Endocrinology	0.535 Natural disaster	0.507
	Fetus	0.535 Landslide	0.502
	Fecundity	0.533 Foreshock	0.496
	IUD	0.513 Mercalli intensity earthquake	0.491
	Sperm count	0.512 1755 Lisbon earthquake	0.489

Table 2 .

 2 Summarized Results

	Task 1: Fertility trends across countries
			Successful responses	Time	Difficulty
					needed	level
	a) Trend from 1921 to 1941	9/9		74s	"easy"
	b)/c) Values for 1926 and 1937	9/9		16s	(1.00)
	Task 2: Causes for the decrease in fertility after 1920 and after 1960 in the US and in the
	UK
		Query steps	Relevant	Confidence Time	Difficulty
			sources		needed per	level
					reason
	a) From 1920 to	12x Google;	12x	"normal"	122s	"difficult"
	1940	3x	different	(3.19)		(-0.50)
		Wikipedia;	sources; 4x
		1x general	Wikipedia
		knowledge
	b) From 1960	11x Google;	8x different	"normal"	118s	"normal"
		3x	sources; 6x	(3.29)		(-0.25)
		Wikipedia	Wikipedia
	Task 3: Usage of the timeline
		Potential reasons	Confidence Time	Difficulty
					needed per	level
					reason
	a) From 1920 to	9x: Marie Stopes opended	"sure"	58s	"normal"
	1940	first birth control clinic in	(3.67)		(0.39)
		UK in 1921; 5x: World War
		I/II; 2x 7th Lambeth
		Conference; 1x: normal
		development for
		industrialized countries
	b) From 1960	8x: FDA -Approval and	"sure"
		usage of oral contraceptive	(4.17)
		pill; 2x: baby boom ends;
		1x: Vietnam war;

http://www.gapminder.org/faq_frequently_asked_questions/

http://motools.sourceforge.net/event/event.html

http://www.iptc.org/site/News_Exchange_Formats/EventsML-G2/

http://www.simile-widgets.org/timeline/

http://opendatastatistics.org

http://www.vizgr.org/historical-events/

http://www.guardian.co.uk/open-platform

http://developer.nytimes.com/


Yan Kou

Edward Y Chen

Neil R Clark

Qiaonan Duan

Christopher M Tan

Avi Ma'ayan

ChEA2: Gene-Set Libraries from ChIP-X Experiments to Decode the Transcription Regulome

Keywords: ChIP-seq, ChIP-chip, Microarrays, Systems Biology, ENCODE, Enrichment Analysis, Transcriptional Networks, Data Integration, Data Visualization, JavaScript D3

ChIP-seq experiments provide a plethora of data regarding transcription regulation in mammalian cells. Integrating spurious ChIP-seq studies into a computable resource is potentially useful for further knowledge extraction from such data. We continually collect and expand a database where we convert results from ChIP-seq experiments into gene-set libraries. The manual portion of this database currently contains 200 transcription factors from 221 publications for a total of 458,471 transcription-factor/target interactions. In addition, we automatically compiled data from the ENCODE project which includes 920 experiments applied to 44 cell-lines profiling 160 transcription factors for a total of ~1.4 million transcription-factor/target-gene interactions. Moreover, we processed data from the NIH Epigenomics Roadmap project for 27 different types of histone marks in 64 different human cell-lines. All together the data was processed into three simple gene-set libraries where the set label is either a mammalian transcription factor or a histone modification mark in a particular cell line, organism and experiment. Such gene-set libraries are useful for elucidating the experimentally determined transcriptional networks regulating lists of genes of interest using gene-set enrichment analyses. Furthermore, from these three gene-set libraries, we constructed regulatory networks of transcription factors and histone modifications to identify groups of regulators that work together. For example, we found that the Polycomb Repressive Complex 2 (PRC2) is involved with three distinct clusters each interacting with different sets of transcription factors. Notably, the combined dataset is made into web-based application software where users can perform enrichment analyses or download the data in various formats.

Introduction

Gene expression in mammalian cells is regulated by transcriptional complexes that include transcription factors, histone modifiers, other chromatin and DNA modifiers, and co-regulators proteins that bring these factors together in a transient manner. These cellular components are increasingly profiled using the ChIP-seq/chip (ChIP-X) technologies to identify the locations on the genome where such cellular components bind. Integrating datasets from many sparse ChIP-X studies applied to mammalian cells into a single computable resource is challenging but can lead to a new level of global understanding of gene expression regulation in mammalian cells. In 2010, a database called ChIP-X Enrichment Analysis (ChEA) was published [START_REF] Lachmann | ChEA: transcription factor regulation inferred from integrating genome-wide ChIP-X experiments[END_REF]. This database originally contained integrated data from 87 publications reporting the binding of 92 mammalian transcription factors to their putative targets. Since then, we continually expanded this database where it currently contains 200 transcription factors from 221 publications for a total of 458,471 transcription-factor/target-gene interactions. Such dataset is potentially useful for two types of applications: 1) exploring the global transcriptional regulatory landscape of mammalian cells; 2) identifying the most likely regulators given lists of differentially expressed genes using gene-list enrichment analyses.

While the initial version of the ChEA database has been proven useful for several applications, since its publication in 2010 other similar databases have been published. For example hmChIP provides systematic access to ChIP-X experiments deposited in to GEO [START_REF] Chen | hmChIP: a database and web server for exploring publicly available human and mouse ChIP-seq and ChIP-chip data[END_REF]. ChIP-Array combines gene expression changes together with ChIP-X experiments to provide querying capabilities [START_REF] Qin | ChIP-Array: combinatory analysis of ChIP-seq/chip and microarray gene expression data to discover direct/indirect targets of a transcription factor[END_REF]. Similarly to ChIP-Array, TranscriptomeBrowser integrates various types of binary regulatory interactions including those extracted from ChIP-X experiments [START_REF] Lepoivre | TranscriptomeBrowser 3.0: introducing a new compendium of molecular interactions and a new visualization tool for the study of gene regulatory networks[END_REF]. Two other related tools that aggregate ChIP-X experiments are called CistromeMap [START_REF] Qin | CistromeMap: a knowledgebase and web server for ChIP-Seq and DNase-Seq studies in mouse and human[END_REF] and CistromeFinder [START_REF] Sun | CistromeFinder for ChIP-seq and DNase-seq data reuse[END_REF]. CistromeMap and CistromeFinder combine many ChIP-seq and DNase-seq publications for integrative analysis. Complimenting these efforts are databases and tools that identify transcription-factor/target-genes interactions by other computational and experimental methods. For example a database called HTRIdb aggregates mammalian transcription-factor/target-gene interactions from lowthroughput studies [START_REF] Bovolenta | HTRIdb: an open-access database for experimentally verified human transcriptional regulation interactions[END_REF] and there are many tools that detect such interactions computationally using position weight matrices. However, not much meta-analysis has been performed on the networks that such datasets and tools produce to obtain a global view of the transcriptional regulatory landscape in mammalian cells. In addition, the data from these tools is not easily utilized for gene-list enrichment analyses. Here we performed an initial integrative analysis of the ChIP-X data we collected, as well as provide a state-of-the-art HTML5 freely available web application that can be used to query the datasets with lists of differentially expressed genes by performing gene-list enrichment analyses.

The Histone Modification Dataset

ChIP-seq for histone modification datasets were collected from the Roadmap Epigenomics project (http://www.roadmapepigenomics.org) deposited to the GEO database (http://www.ncbi.nlm.nih.gov/geo/). Previous studies [START_REF] Pepke | Computation for ChIP-seq and RNAseq studies[END_REF] have indicated that the use of control samples substantially reduces DNA shearing biases and sequencing artifacts, therefore for each experiment, an input control sample was matched according to the description in GEO. ChIP-seq experiments without matched control dataset were not included. BED files were first downloaded and standardized to contain the first six columns as defined by the UCSC genome browser BED file format by adding placeholders if needed. Unlike narrow peaks of transcription factor binding sites, histone modification marks have broad peaks and can be extended to as long as 100kb [START_REF] Pepke | Computation for ChIP-seq and RNAseq studies[END_REF]; therefore the peak-calling software Sicer [START_REF] Zang | A clustering approach for identification of enriched domains from histone modification ChIP-Seq data[END_REF] was used to identify significant peaks. Sicer uses a spatial clustering approach that considers enrichment information from neighboring regions to call broader domains. Each read was shifted by 75bp towards the center of the DNA fragment; window size of 200bp and gap size equals to the window size, or three times the window size, were used according to the authors' guide. However, for large datasets, window size of 500bp and gap size of 1500bp were applied because smaller parameters resulted in memory limit errors. For broad histone marks such as H3K27me3 and H3K36me3, the window size of 500bp and gap size of 1500bp were set to detect significant peaks. Peaks were identified with only one read mapped to the reference genome, not allowing for redundant reads. False discovery rate (FDR) of 0.001 was used to find significant peaks as compared to the control.

The significant peaks for each experiment were collected and associated with annotated human genes. Chromosome coordinates of the human hg19 build were downloaded from NCBI; the longest readout was kept for genes having multiple isoforms. A gene is considered associated with a significant peak if the peak overlaps with the region from -10kb of transcription starting site (TSS) to +5kb of transcription ending site (TES) of the gene. Peaks that overlap with +2kb of TSS of known genes were defined as enriched at promoter regions. Genes that have significant peaks at promoter regions were ranked according to their distance from the middle point of the associated peak to the transcription starting site. For experiments that identified more than 2000 genes around TSS and TES, the top 2000 genes were taken to generate the gene matrix transpose (GMT) gene-set library file, whereas all genes were taken for experiments that recovered less genes. The resulted dataset contains 27 types of histone modifications for 64 human cell lines and tissue types.

The ENCODE Transcription Factor Dataset

ChIP-seq datasets for transcription factor binding sites were downloaded from the ENCODE project [START_REF]An integrated encyclopedia of DNA elements in the human genome[END_REF] (ftp://encodeftp.cse.ucsc.edu/pipeline/hg19/) on July 9, 2012. A total of 920 experiments applied to 44 cell lines profiling 160 transcription factors were processed. For each experiment, the peak file in broadPeak or narrowPeak format was extracted. For each experiment dataset, the peaks in the broadPeak and narrowPeak files were mapped to the TSS of human genes from the hg19 build if the peak overlaps with the region of (-5kb, +2kb) surrounding the TSS. Most of the experiments have biological replicates. Therefore, only the unique genes (intersection) with peaks deposited within the defined promoter region from both replicates were extracted for further analysis. The GMT file was generated in the same way as for the histone modification datasets, where the top 2000 genes were taken for experiments that yielded larger number of target genes.

Updating the Manual Part of the ChEA Database

The manual part of the ChEA database is continually updated by manually processing the supporting tables from ChIP-seq publications. Target gene lists are extracted from such publications. Only human or mouse experiments are considered. All gene identifiers are converted to Entrez Gene Symbols. E-mail alerts for abstracts that contain the key term of "ChIP-seq" or "ChIP-chip" point to publications that may contain data that is suitable for this database. In some cases when the authors only provide the peaks' coordinates we process the data to identify and rank the associated genes.

The ChEA2 Web Application

ChEA2 uses HTML5, CSS3, and JavaScript on the frontend to render the page, make the SVG visualizations interactive, and retrieve JSON data from the backend server asynchronously using AJAX. The frontend makes heavy use of the jQuery JavaScript library and other dependent plugins to simplify page manipulation/selection, enable touch gestures, add sortable tables, and create tooltips in combination with using the D3 JavaScript library to generate SVG visualizations. ChEA2 also utilizes JSP to generate some of the more dynamic pages that interact with the backend. The backend server side uses an Apache Tomcat 7 application server to host Java servlet web apps and a Microsoft IIS 6 web server to serve the web pages. Apache Maven is used to simplify the process of compiling the code, processing the web resources, and deploying the web app to the Tomcat server. The application is best viewed with Chrome or Firefox, but it is compatible with all modern browsers that support SVG and HTML5. The code-based is mostly borrowed from the recently published tool Enrichr [START_REF] Chen | Enrichr: interactive and collaborative HTML5 gene list enrichment analysis tool[END_REF].

Canvas Visualization

The canvas images in Figure 1 represent an alternative way to visualize networks. Instead of connecting nodes with edges, the network nodes are placed onto a square toroidal grid and then clustered together using simulated annealing to maximize local connections. The brightness of a node on the finished grid corresponds to the strength of its connections with its neighbors. The annealing process is performed by swapping two randomly selected nodes on the grid and recalculating the global fitness. After annealing, the canvas is exported as a JavaScript Object Notation (JSON) file which is then sent to the Visualizer module. The Visualizer creates SVG images with HTML5 and the JavaScript Library D3. These images were created with the recently published tool, Network2Canvas [START_REF] Tan | Network2Canvas: Network Visualization on a Canvas with Enrichment Analysis[END_REF].

The manual portion of the ChEA2 database currently contains 200 transcription factors from 221 publications for a total of 458,471 transcription-factor/target interactions. In addition, the data processed from ENCODE includes 920 experiments applied in 44 cell-lines profiling 160 transcription factors for a total of ~1.4 million transcription-factor/target-gene interactions. The data from the NIH Epigenomics Roadmap project contain 27 different types of histone marks in 64 different human cell-lines for a total of 438,064 interactions. The first step in our analysis is to visualize each of the gene-set libraries: the manual ChEA, ENCODE, and histone modifications on a canvas. The canvas visualization is a compact alternative to visualize networks as ball-and-stick diagrams [START_REF] Tan | Network2Canvas: Network Visualization on a Canvas with Enrichment Analysis[END_REF]. In these canvases, each ChIP-X experiment is represented as a square on a grid of squares. The location of the experiments/squares is optimized so a gene-set with a similar gene-set are adjacent to each other. The similarity measure for the three grids is based on set similarity computed by the Sets2Networks algorithm [START_REF] Clark | Sets2Networks: network inference from repeated observations of sets[END_REF]. Once such arrangement is determined, the squares are color-coded by the level of similarity each experiment/square has with its neighbors. The brighter the square the more similar it is to its neighboring squares. The canvases clearly identify clusters within the gene-set libraries/networks (Fig. 1). These clusters include a stem-cell/pluripotency set of factors and the PRC2 group in the manual ChEA canvas; the H3K36me3, H3K27me3, mixture of H3K27me3/H3K9me3 clusters in the histone modifications canvas; and CTCF/RAD21, TAF1/TAF7/YY1/HEY1 clusters in the ENCODE canvas; but there other clusters. While some of these clusters are likely functional, where the factors physically interact to regulate the same sets of genes, it should be considered that some factors are more frequently profiled so there is some research bias in the formation of the various clusters on these canvases.

Next, we visualized the manual ChEA gene-set library and the ENCODE gene-set library using multi-dimensional scaling (MDS) with a distance measure similarity computed using the Sets2Networks algorithm [START_REF] Clark | Sets2Networks: network inference from repeated observations of sets[END_REF] (Fig. 2). This visualization identifies the same clusters but has the advantage of showing the labels for most factors as well as provides a visual of the distance similarity among all factors. Then, we visualized the manual ChEA gene-set library as a heatmap using hierarchical clustering (Fig. 3). The heatmap identifies three clusters that include the members of the PRC2 complex: EZH2, EED, SUZ12, and JARID2. Besides these core components of PRC2, the transcription factor MTF2, and the histone methyltransferase SETDB1 as well as RNF2 and PHC1 which are both members of the PRC1 complex, cluster with the canonical PRC2 components. Interestingly, the transcription factor p53 also clusters with these factors. However, these three clusters also include various other factors that are unique for each one of the three clusters. Using known protein-protein interactions from the literature [START_REF] Berger | Genes2Networks: connecting lists of gene symbols using mammalian protein interactions databases[END_REF], we see that many of these additional factors were previously identified to interact with members of the PRC2 complex, suggesting that these factors directly physically associate with the PRC2 complex members to regulate the same set of genes in different contexts (Fig. 3, right). This potentially identifies multiple roles for the PRC2 complex in different cell types. As we see below, when combining the manual ChEA dataset with the ENCODE and histone modification datasets the PRC2 complex members strongly associates with the H3K27me3 modification which is already well established. Next, we visualized the manual ChEA and ENCODE networks as ball-and-stick diagrams. We only connected the transcription factors that were used as immunoprecipitation baits in ChIP-X experiments (Fig. 4). Therefore all nodes are transcription factors and the edges connect transcription factor profiled by ChIP-X to their direct binding targets which are other transcription factors. The manual ChEA network is made of 197 factors connected through 5344 edges whereas the ENCODE network is made of 150 factors connected through 2430 links with clustering coefficients of 0.15 and 0.09 respectively. These clustering coefficients are much higher than the clustering observed for random networks of the same size. Since the density of links in these networks is very high, not much of the network structure is revealed by the ball-and-stick visualization. However, the hubs of the networks clearly stand out. These hubs are likely master regulators or well-studied factors. Interestingly BHLHE40 (also called DEC1, HLHB2, SHARP-2, STRA13, and STRA14) a less known transcription factor appears as a hub in the ENCODE network. This is because this factor is a target of many other factors as well as a regulator of many of the other profiled factors. It is possible that this transcription factor has an important global role: integrating information from many transcription factors and acting as a master regulator of many factors. This central role may have been yet mostly overlooked.

Next, we combined all three gene-set libraries into one heatmap that cluster the entries based on similarity of gene-list content (Fig. 5). Euclidean distance of the Jaccard score was used as a similarity measure and the linkage type was average. Multiple experiments forming clusters are highlighted and listed on the right. In general the histone modification experiments cluster together by modification type and the transcription factors form their own clusters. Focusing on one cluster that contains a mixture of transcription factors and histone modification experiments, we mapped all the known protein-protein interactions that connect the factors, as well as the protein interactions between these factors and the known enzymes that remove or add the chemical modification group on the histone marks from the cluster (Fig. 6). Links connect the enzymes associated with these modifications were created by a literature search and the other links represent known experimentally determined protein-protein interactions collected from available online databases [START_REF] Berger | Genes2Networks: connecting lists of gene symbols using mammalian protein interactions databases[END_REF]. We find that some factors interact more exclusively with the enzymes that remove the marks, while other factors interact more exclusively with the enzymes that add the mark. Moreover, two writers: EP300 and CREBBP and one remover HDAC1 are highly connected. This could be a literature bias since these factors are well-studied, but also can represent the type of complexes that are formed to regulate the same subset of genes. Finally, we performed enrichment analysis, using all three gene set libraries: ChEA-manual, ENCODE and histone modifications, on the entire gene set library created from the Mouse Genomics Informatics Mammalian Phenotype ontology (MGI-MP) [START_REF] Eppig | The Mouse Genome Database (MGD): comprehensive resource for genetics and genomics of the laboratory mouse[END_REF]. This mapping identifies relationships between annotated single gene knockout mouse phenotypes and groups of transcriptional regulators. Genes associated with mouse phenotypes (columns) were used for enrichment analysis and the corrected p-values of each term from the three gene-set libraries: manual ChEA, ENCODE and histone modifications (rows) were used for drawing the hierarchical clustering heatmap. Clusters of mouse phenotypes are highlighted with green labels and clusters of ChEA2 elements are labeled with orange colors (Fig. 7). For example, a group of transcription factors including GATA2, EGR1, BCL11A, FLI1, and NFKB are highly enriched for abnormal immune system as expected. The mouse phenotype clusters of abnormal neuronal system (Cluster 3 in Fig. 7) and abnormal embryonic/early development (Cluster 5) share similar groups of enriched transcription factors and histone modification profiles. This may be due to the observation that neuronal specific genes are suppressed in embryonic stem cells by specific histone modifications. The map includes many potentially interesting small clusters that identify relationships between factors and phenotypes. For example, we highlight two small clusters that connect abnormal pigmentation related phenotypes and abnormal lysosomes to the transcription factors: USF1/2, ATF3, BHLHE40, MAX, NF2, and TFEB. Indeed, TFEB has a well-established role in autophagy [START_REF] Decressac | TFEBmediated autophagy rescues midbrain dopamine neurons from α-synuclein toxicity[END_REF] and ATF3 is known to be associated with stress responses [17]. Corre and Galibert summarized the roles of USF1/2 in various contexts including the role of these factors in pigmentation determination and stress responses [18]. While there is some related knowledge about the role of some of these factors in determining these phenotypes, our analysis suggests that these factors may work together to regulate the same sets of genes that are responsible for the induction of specific functions that lead to specific phenotypes. In addition to the preliminary meta-analysis, we performed on the three gene-set libraries ChEA2 datasets, we also developed a web-based application that provides access to the data and enables users to query their lists of differentially expressed genes (Fig. 8). The tool is implemented with HTML5 and the JavaScript library D3 and it is freely available online at http://amp.pharm.mssm.edu/ChEA2.

Conclusions

Here, we assembled a large compendium of ChIP-seq/chip experiments applied to mammalian cells. We began an initial meta-analysis of such complex dataset by abstracting each experiment to a gene list. Such abstraction discards much important information such as peak height and exact binding locations. Peak height and exact binding locations information are likely critical in better assessing cooperation among regulators. In addition, including such quantitative data in enrichment analyses is likely to improve the inference of regulators given sets of differentially expressed genes. Moreover, we must consider the fact the putative target genes determined by ChIP-seq/chip contain many false positives. Integrating results from transcription factor knockdowns followed by genome-wide expression can potentially filter putative binding targets with real functional targets. Another consideration is that our network analysis is very preliminary. We did not consider measuring network properties such as connectivity distributions and identification of network motifs. Since the datasets contain a large-scale directed graphs made of transcription factors regulating other transcription factors, insights can be obtained from structural and dynamical analyses of such graphs. Regardless of these limitations, we have integrated an important large-scale dataset made from most available ChIP-X studies into a computable format and began the global analysis of this potentially useful resource. This project paves the way for more extensive computational studies that would further unravel the transcriptional networks that are responsible for regulating the complex system of the mammalian cell. In addition, the data collection and organization, as well as the interactive light-weight web-based data visualization of ChEA2 exemplify an application from systems biology that deals with a complex interdisciplinary challenge to extract knowledge from massive datasets [19].

Figure Legends

Fig. 1. Canvas representation of transcription factors and histone modifications networks using Network2Canvas (N2C) [START_REF] Tan | Network2Canvas: Network Visualization on a Canvas with Enrichment Analysis[END_REF]. Fig. 2. Multi-dimensional scaling scatter plot of the a) manual ChEA; and b) ENCODE datasets, using the Sets2Networks distance measure [START_REF] Clark | Sets2Networks: network inference from repeated observations of sets[END_REF].

Fig. 3 .

 3 Fig. 3. Ball-and-stick representation of the transcription factor regulatory networks from ChEA (a) and ENCODE (b) based on direct regulatory interactions between transcription factors and other transcription factors. Node size and color represents connectivity degree.

Fig. 4 .

 4 Fig. 4. Hierarchical clustering heatmap of the ChEA adjacency matrix based on direct targets. Direct protein-protein interactions connecting transcription factors that form clusters on the heatmap are highlighted for several clusters.

Fig. 5 .

 5 Fig. 5. Hierarchical clustering of all three gene set libraries: ChEA, ENCODE and histone modifications. Experiments were clustered based on their target-set similarity.

Fig. 6 .

 6 Fig. 6. Protein-protein interactions (PPI) between histone modifying enzymes and transcription factors in one of the clusters from Figure 5. The histone modifications (orange square nodes) and transcription factors (green eclipse nodes) are members of a cluster in Fig. 5.

Fig. 7 .

 7 Fig. 7. Hierarchical clustering of gene sets causing abnormal phenotypes in mice and enriched transcription factors and/or histone modifications. Generalized descriptions of mouse phenotype clusters are listed at the bottom, with numbers matching to the corresponding clusters.

Fig. 8 .

 8 Fig. 8. Screenshots from the ChEA2 web-application. a) Users can enter lists of genes or upload a file containing lists of genes. b) Once the users submit valid gene lists, they are presented with enrichment analysis results that rank transcription factors and histone modifications as bar-graphs, tables, networks of enriched factors and canvases that highlight the enriched factors or enriched histone modification experiments.

16 Fig. 1

 161 Fig.1

  Fig. 2 ChEA

Fig. 4

 4 Fig. 4

Fig. 5

 5 Fig. 5

Fig. 6

 6 Fig. 6

Fig. 7

 7 Fig. 7

Fig. 8

 8 Fig. 8

Acknowledgements

This work is supported in part by NIH grants R01GM098316, R01DK088541, U54HG006097-02S1, P50GM071558, and Irma T. Hirschl Career Scientist Award.