Adrien Vogt-Schilb
email: avogtschilb@worldbank.org

Stéphane Hallegatte
email: shallegatte@worldbank.org

Christophe De Gouvello
email: cdegouvello@worldbank.org

Marginal Abatement Cost Curves and Quality of Emission Reductions: A Case Study on Brazil

Decision makers facing emission-reduction targets need to decide which abatement measures to implement, and in which order. This paper investigates how marginal abatement cost (MAC) curves can inform such a decision. We re-analyse a MAC curve built for Brazil by 2030, and show that misinterpreting MAC curves as abatement supply curves can lead to suboptimal strategies. It would lead to (i) under-investment in expensive, long-to-implement and large-potential options, such as clean transportation infrastructure, and (ii) over-investment in cheap but limited-potential options such as energy-efficiency improvement in refineries. To mitigate this issue, the paper proposes a new graphical representation of MAC curves that explicitly renders the time required to implement each measure.

Policy relevance

In addition to the cost and potential of available options, designing optimal short-term policies requires information on long-term targets (e.g., halving emissions by 2050) and on the speed at which measures can deliver emission reductions. Mitigation policies are thus best investigated in a dynamic framework, building on sector-scale pathways to long-term targets. Climate policies should seek both quantity and quality of abatement, by combining two approaches. A "synergy approach" that focuses on the cheapest mitigation options and maximizes co-benefits. And an "urgency approach" that starts from a long-term objective and works backward to identify actions that need to be implemented early. Accordingly, sector-specific policies may be used (i) to remove implementation barriers on negative-and low-cost options and (ii) to ensure short-term targets are met with abatement of sufficient quality, i.e. with sufficient investment in the long-to-implement options required to reach long-term targets.

Various options are available to reduce greenhouse gas (GHG) emissions: fuel switching in the power sector, renewable power, electric vehicles, energy efficiency improvements in combustion engines, waste recycling, forest management, etc. Policy makers have to compare and assess these different options to design a comprehensive mitigation strategy and decide the scheduling of various actions (i.e. decide what measures need to be introduced and when). This is especially true concerning the emissionreduction measures that require government action (e.g., energy-efficiency standards, public investment, public planning).

Marginal abatement cost (MAC) curves are largely and increasingly used in the policy debate to compare mitigation actions [START_REF] Kesicki | Marginal abatement cost curves: a call for caution[END_REF][START_REF] Esmap | Planning for a low carbon future: Lessons learned from seven country studies[END_REF][START_REF] Kolstad | Social, economic and ethical concepts and methods[END_REF]. A MAC curve provides information on abatement costs and abatement potentials for a set of mitigation measures. They can serve as powerful tools to communicate that large amounts of emission reductions are technically possible. They also show that some emission reductions can pay for themselves due to co-benefits such as energy efficiency gains or positive impact on health, and that many others will be inexpensive (in terms of net present social value). This information can help governments decide how ambitious their mitigation strategy will be, and make informed domestic and international commitments (in the UNFCCC context, for instance). It is also helpful for policy makers searching for synergies and co-benefits, for instance between emission reduction and economic development.

The academic literature on MAC curves has extensively discussed the plausibility of energy efficiency options that would reduce emissions at net negative costs. In general, MAC curves do not factor in implementation barriers on these options, such as split incentives, lack of information, behavioral failures, or lack of resources (Allcott and Greenstone, 2012; [START_REF] Kesicki | Marginal abatement cost curves: a call for caution[END_REF]. 1 According to this literature, overcoming such barriers may be costly enough to reduce significantly the economic benefits from energy savings. To date, identifying specific barriers and cost-effective ways of working around them remains a policy-relevant challenge (e.g., [START_REF] Giraudet | Double moral hazard and the energy efficiency gap[END_REF].

The issue discussed here is different. Because they rank options according to their cost -from the least to the most expensive -MAC curves look like abatement supply curves (Fig. 1), and are frequently interpreted as such (e.g. [START_REF] Haab | Environmental economics: Look! it's a carbon supply curve[END_REF]DECC, 2011). According to this interpretation, the optimal emission-reduction strategy would be to "implement the cheapest measure first, preferring measures with a lower total saving potential but more cost-effective than those with a higher GHG saving potential in absolute terms" [START_REF] Wächter | The usefulness of marginal CO2-e abatement cost curves in Austria[END_REF]. In this paper we show why this strategy is not the optimal one, we propose a new graphical representation of MAC curves that avoid this misinterpretation, and we derive broader policy implications on the design of climate mitigation strategies.

In addition to the cost and potential, a key parameter of emission reduction options is the speed at which they can be implemented. Speed is limited by factors such as (1) long capital turnover, (2) slow technological diffusion, (3) availability of skilled workers, (4) availability of relevant specific capital, such as production lines, (5) availability of funds or (6) institutional constraints. 2 As a consequence, some high-abatement-potential measures, such as switching to renewable power or retrofitting existing energy-inefficient buildings, may take decades to implement. While the cost and potential displayed in a MAC curve are frequently assessed with such maximum speed in mind, the diffusion speed itself is almost never displayed in the MAC curve or generally disclosed to decision-makers.

With a simple theoretical model, [START_REF] Ha-Duong | Influence of socioeconomic inertia and uncertainty on optimal CO2-emission abatement[END_REF] find that this technical inertia (using the wording by [START_REF] Grubb | The economics of changing course: Implications of adaptability and inertia for optimal climate policy[END_REF] means that the optimal quantity of shortterm abatement depends on long-term objectives -an extensive literature based on integrated assessment models has reached the same conclusion (e.g. [START_REF] Luderer | Implications of weak near-term climate policies on long-term mitigation pathways[END_REF][START_REF] Bertram | Carbon lock-in through capital stock inertia associated with weak near-term climate policies[END_REF][START_REF] Riahi | Locked into copenhagen pledges -implications of shortterm emission targets for the cost and feasibility of long-term climate goals[END_REF]. Using a theoretical MAC curve, Vogt-Schilb and Hallegatte (2014) show that the quality of abatement is also important. The authors argue short-term abatement targets should be reached with some of the high-potential but long to implement measures that will make deeper decarbonization possible in the long term, even if these are not the less expensive measures available in the short term. As a consequence, focusing on short-term targets (e.g., for 2030) without considering longer-term objectives (e.g., for 2050 and beyond)

Figure 1: A measure-explicit marginal abatement cost curve. The general appearance of the curve makes it easy to misinterpret it as an abatement supply curve, leading to the misguided conclusion that the "abatement demand" X should be met with measures 1 to 4 only (possibly using the carbon price Y).

would lead to carbon-intensive lock-ins, making it much more expensive (and potentially impossible) to achieve the long-term objectives. 3 In this paper, we apply Vogt-Schilb and Hallegatte's method on a MAC curve built at the World Bank for studying low-carbon development in Brazil in the 2010-2030 period [START_REF] De Gouvello | Brazil Low-carbon Country Case Study[END_REF]. Lack of data beyond 2030 does not allow us to investigate how using only the 2010-2030 MAC curve to design a mitigation strategy would lead to suboptimal choices in view of longer-term objectives (2050 and beyond). We can however investigate this problem by assuming that we want to achieve an objective for 2030, and that we use the MAC curve to design a mitigation strategy for the 2010-2020 period only.

We find that a strategy for 2010-2020 that disregards the 2030 target under-invests in clean transportation infrastructure such as metro and train; and over-invests in marginal, cheap but low-potential options, such as heat integration and other improvements in existing refineries. In other words, developing clean transportation infrastructure in the short term is appealing only if the long-term abatement target is accounted for. In addition, we find that not developing clean transportation infrastructure in the short term (by 2020) closes the door to deeper emission reductions in the middle (2030) and longer term. Loosely speaking, the 2020 strategy provides a sensible quantity of abatement by 2020, but abatement is of insufficient quality to reach the 2030 target. These results stress the need 3 A related line of argumentation is on learning by doing and directed technical change [START_REF] Gerlagh | Optimal timing of climate change policy: Interaction between carbon taxes and innovation externalities[END_REF][START_REF] Acemoglu | The environment and directed technical change[END_REF][START_REF] Kalkuhl | Learning or lockin: Optimal technology policies to support mitigation[END_REF]. Many of the technologies used to reduce emissions -for instance more efficient cars or renewable energy -are still in the early stage of their development, such that their cost will decrease as their deployment continues. Many authors have found in a variety of settings that this is a sound rationale to use expensive options in the short term (e.g., [START_REF] Rosendahl | Cost-effective environmental policy: implications of induced technological change[END_REF][START_REF] Del Rio Gonzalez | Policy implications of potential conflicts between short-term and long-term efficiency in CO2 emissions abatement[END_REF][START_REF] Azar | The elusive quest for technologyneutral policies[END_REF]. In the present work, we account for technical progress only to the (limited) extent that it can be captured by the slow technological diffusion encompassed in our diffusion speed constraint.

for policymakers to take into account long term targets and the limited speed at which emission-reductions may be implemented when deciding on short-term action.

We derive two conclusions from this work. First, MAC curves do not report a very important piece of information, namely the implementation pace of each measure and option. We suggest that when MAC curves are produced, they should be presented together with the corresponding emission reduction scenario -using the graphical representation that [START_REF] Pacala | Stabilization wedges: Solving the climate problem for the next 50 years with current technologies[END_REF], [START_REF] Williams | The technology path to deep greenhouse gas emissions cuts by 2050: The pivotal role of electricity[END_REF] and [START_REF] Davis | Rethinking wedges[END_REF] call wedge curves -making the dynamic aspect of the mitigation scenarios more explicit (Fig. 2). Note that this proposal concerns only the graphical communication of abatement measures, their impact on greenhouse gas emissions over time, and their cost; without any prescription on the method used to assess those numbers.

For instance, emission reduction potentials and costs are frequently assessed from expert surveys (e.g. [START_REF] Esmap | Planning for a low carbon future: Lessons learned from seven country studies[END_REF]. MAC curves built this way would be greatly improved by an explicit discussion of implementation barriers and factors limiting the pace at which emission reductions may be achieved with each particular measure (see Appendix B for suggested guidance for the experts in charge of collecting the information to build a MAC curve). This information would be particularly useful for decision makers if it permits identifying distinct bottlenecks (e.g. availability of skilled workers) that can be translated into specific policies (e.g. training).

Emission reduction scenarios, costs and potentials can also be derived from energy system models [START_REF] Kesicki | Marginal abatement cost curves: Combining energy system modelling and decomposition analysis[END_REF]. 4These models account for the limited ability to implement emission-reduction measures by building in particular on maximum investment speeds [START_REF] Wilson | Future capacity growth of energy technologies: are scenarios consistent with historical evidence?[END_REF][START_REF] Iyer | Diffusion of low-carbon technologies and the feasibility of long-term climate targets[END_REF], making them suitable for studying path dependency in emission reduction strategies (Kesicki, 2012a). MAC curves built this way can also be presented next to the corresponding wedge curve, as in Fig. 2. In this case also, the policy debate is improved by an explicit discussion of how the growth constraints are calibrated in the models [START_REF] Wilson | Future capacity growth of energy technologies: are scenarios consistent with historical evidence?[END_REF][START_REF] Iyer | Diffusion of low-carbon technologies and the feasibility of long-term climate targets[END_REF].

Second, climate change mitigation policies are designed for a relatively short term horizon (e.g., 2020 or 2030), while mitigation objectives go beyond this horizon (e.g., the EU has a 2050 objective). Most importantly, stabilizing climate change and tackling other environmental threats will require a reduction in emissions to nearzero levels by the end of the century [START_REF] Collins | Long-term climate change: projections, commitments and irreversibility[END_REF][START_REF] Steinacher | Allowable carbon emissions lowered by multiple climate targets[END_REF]; following the wording by [START_REF] Sachs | Pathways to deep decarbonization[END_REF], any climate stabilization target requires deep decarbonization.

An ideal policy would be to announce well in advance a perfectly credible long-term target to a forward-looking market. In practice, however, governments have limited ability to commit, and markets cannot perfectly anticipate future regulations [START_REF] Golombek | Carbon taxes and innovation without commitment[END_REF][START_REF] Brunner | Credible commitment in carbon policy[END_REF]. Following the World Bank (2012, p. 153), we thus suggest to combine a "synergy approach" focusing on mitigation options that provide co-benefits in terms of development, economic growth, job creation, local environmental quality, or poverty alleviation, with an "urgency approach", based on defining long-term objectives and working backward to identify which measures are needed early to achieved stated objective. Accordingly, sector-specific mitigation policies have two roles: (i) to remove implementation barriers on negativeand low-cost options, and (ii) to ensure short-term targets are met without under-investing in the ambitious and longto-implement abatement measures required to achieve otherwise-difficult-to-enforce long-term targets. In other words, these policies should ensure that the mitigation strategy reaches not only the desired quantity of abatement at a given date, but also a sufficient quality to make further emission reduction possible.

This second argument for sector-specific policies, in line with [START_REF] Waisman | The Imaclim-R model: infrastructures, technical inertia and the costs of low carbon futures under imperfect foresight[END_REF], remains a novelty in the academic literature: to date, such policies have been discussed as a way to tackle several market failures or policy objectives, including learning by doing [START_REF] Sandén | Near-term technology policies for long-term climate targets-economy wide versus technology specific approaches[END_REF][START_REF] Fischer | Combining policies for renewable energy: Is the whole less than the sum of its parts?[END_REF]; to correct for the effects of misperceived energy savings [START_REF] Tsvetanov | Re-evaluating the role of energy efficiency standards: A behavioral economics approach[END_REF][START_REF] Parry | Are energy efficiency standards justified[END_REF]; to complement an imperfect carbon-pricing mechanism [START_REF] Lecuyer | Can uncertainty justify overlapping policy instruments to mitigate emissions?[END_REF]; or as a political economy constraint [START_REF] Hallegatte | Green industrial policies: When and how[END_REF][START_REF] Jenkins | Political economy constraints on carbon pricing policies: What are the implications for economic efficiency, environmental efficacy, and climate policy design? Energy Policy[END_REF][START_REF] Rozenberg | Transition to clean capital, irreversible investment and stranded assets[END_REF].

The rest of the paper is structured as follows. In section 1, we review different types of MAC curves. While the construction of MAC curves sometimes requires to investigate the diffusion speed of emission-reduction options, MAC curves do not report separately the long-term abatement potential and the diffusion speed. In section 2, we reanalyze the data from the Brazilian MAC curve. We extract the cost, long-term potential and diffusion speed of each emission-reduction measure, and use them in a simple optimization model to investigate the least-cost emissionreduction schedule, depending on whether the objective is to reach a 2030 target or the corresponding 2020 target. We conclude in section 3.

Existing MAC curves

We call measure-explicit MAC curves (MAC curves for short) these which represent abatement costs and potentials of a set of mitigation measures. MAC curves have been developed since the early 1990s [START_REF] Rubin | Realistic mitigation options for global warming[END_REF], and have reached a wide public after McKinsey and Company published assessments of the cost of abatement potentials in the United States [START_REF] Mckinsey | Reducing US greenhouse gas emissions: How much at what cost?[END_REF] and at the global scale [START_REF] Enkvist | A cost curve for greenhouse gas reduction[END_REF]. This type of curve is increasingly used to inform policy makers. [START_REF] Kesicki | Marginal abatement cost curves: a call for caution[END_REF], who also offer a richer historical perspective). Depending on their implicit definition of the abatement potential of a measure, two types of measure-explicit MAC curves can be distinguished.

Full potential MAC curves

The full-potential approach gives information on how much GHG could be saved if the measure was used at its technical maximum. It is calculated against a reference or baseline technology, as for instance those used in the present [START_REF] Wächter | The usefulness of marginal CO2-e abatement cost curves in Austria[END_REF], taking into account the carbon intensity and imperfect substitutability of different technologies. For instance, this approach assesses what fraction of passenger vehicles can be replaced by electric vehicles (EV), accounting for limited driving range and exiting mobility practices. Given emissions from baseline vehicles option instead of the baseline option, divided by discounted avoided emissions. Marginal and average costs are equal only if the unit cost of abatement is constant. Note that potentials spreading over large range of abatement costs may be split into smaller potentials of nearly constant abatement cost [START_REF] Kesicki | Marginal abatement cost curves: Combining energy system modelling and decomposition analysis[END_REF], for instance reporting gas for base load and gas for peak power separately.

(e.g. 140g/km today in Europe) and emissions from EVs (say 30g/km), one can compute an amount of emissions avoidable using electric vehicles. [START_REF] Rubin | Realistic mitigation options for global warming[END_REF] use this approach. For instance, they assess the potential of nuclear power (in the US) as the quantity of GHG that would be saved if nuclear replaced all the fossil fuel capacity used for base load and intermediate load operation in 1989.

The main value of full potential MAC curves is descriptive: they highlight to which extent some key measures could reduce emissions in the long-run. One weakness is that full-potential MAC curves cannot easily represent the competition between two measures.6 Finally, full-potential MAC curves do not require investigation of possible diffusion constraints, but these may be assessed separately to build resulting emission reduction scenarios (e.g. World Bank, 2013).

Achievable potential MAC curves

Achievable-potential MAC curves have a prospective dimension, as they are built for a date in the future. This approach fully acknowledges that large-scale diffusion of new technologies can take decades [START_REF] Grübler | Technological change and the timing of mitigation measures[END_REF][START_REF] Grübler | Dynamics of energy technologies and global change[END_REF][START_REF] Wilson | Future capacity growth of energy technologies: are scenarios consistent with historical evidence?[END_REF]. In this context, the abating potential of a technology is an assessment of the abatement that could be achieved with such a technology if it was implemented at a given speed, starting at a given date. For instance, this approach takes into account that even ambitious fiscal incentives in favor of electric vehicles would induce a limited increase of EV sales, resulting in a limited share of EVs in the fleet, hence limited emission reductions from EVs by 2020 or 2030. The potential achievable by a given date is therefore lower or equal than the full potential reported on full potential MAC curves.

A key advantage of the achievable-potential approach is that it requires investigation of reasonable assumptions regarding the possible implementation speed of a measure (e.g. 1% of the dwellings can be retrofitted each year). This information is key for a policy maker scheduling emissionreduction investments. Unfortunately, assessed diffusion speeds are not displayed in the resulting MAC curve, and are not always discussed in the accompanying reports.

Most expert-based MAC curves published in the gray literature are constructed this way -see for instance McKinsey (2009, p. 46), or Pellerin et al. (2013, p. 22). The few curves built using integrated assessment models are also achievable-potential MAC curves [START_REF] Kesicki | Marginal abatement cost curves: Combining energy system modelling and decomposition analysis[END_REF]. Achievable-potential MAC curves are built from emissionreduction pathways (Fig. 2), that are investigated taking into account at least some inter-temporal dynamics and sector-specific constraints. It is thus logically inconsistent to conclude from an achievable-potential MAC curve that emission-reduction should be implemented sequentially in the "merit order", cheapest first. The original emission-reduction pathways already provides an answer to when and where to reduce GHG emissions. Unfortunately, achievable-potential MAC curves have been frequently used overlooking their caveats, in particular in the media and policy debate [START_REF] Haab | Environmental economics: Look! it's a carbon supply curve[END_REF][START_REF] Kesicki | Marginal abatement cost curves: a call for caution[END_REF].

One weakness of the achievable potential is that it makes the slow diffusion process indistinguishable from the full potential. The reader of a MAC curve does not know, for instance, if a small potential for abatement from residential building retrofit means that residential buildings are already almost entirely retrofitted in the region (the full potential is low), or it if means that only a small fraction of buildings may be retrofitted during the period (the diffusion is slow).

The MAC curve we reanalyze in this paper is an achievablepotential MAC curve. In each economic sector, emission reduction scenarios have been assessed taking into account constraints on implementation and maximum diffusion speeds (de Gouvello, 2010).

MAC curves at the World Bank: MACTool

The World Bank develops and promotes a piece of software called MACTool, which can produce achievablepotential MAC curves. One aim of the MACtool is to provide policy makers with a common framework to analyze available mitigation measures. MACTool takes as inputs the key socio-technical parameters of a set of large mitigation measures, and macroeconomic variables. For instance, technology options to produce electricity are characterized by required capital and operation expenditures, as well as their lifetime, energy efficiency and type of fuel used. Physical constants as the carbon intensity of each fuel are factored in. The user must also specify at least one scenario on the future macroeconomic variables of interest, such as the price of fossil fuels and the future demand for electricity. Finally, the user must provide scenarios of future penetration of (low-carbon) technologies and measures, in both a baseline and at least one emissionreduction pathway [START_REF] Esmap | Modeling tools and e-learning: MACTool[END_REF].

As outputs, MACTool computes the amount of GHG saved by each measure in the long run (in MtCO 2 ), and the cost of doing so (in $/tCO 2 ). This information is illustrated with two figures: an achievable potential MAC curve, and an abatement wedge curve.

The tool itself does not provide information on what is achievable, this information comes directly from the input scenarios. Input scenarios therefore need to be built taking into account the constraints on technology diffusion and implementation speed. For instance, these scenarios may come from integrated assessment models that factor such constraints in, or be built by sector experts who guesstimate possible penetration scenarios [START_REF] Kesicki | Marginal abatement cost curves: a call for caution[END_REF].

In addition to the classical abatement cost and abatement potential, MACTool reports the investment needed in different emission reduction scenarios. MACTool can also compute the carbon price signal that would be required to trigger investments from the private sector, taking into account any private discount rate. These can be different from the social discount rate to reflect different opportunity costs of capital in sectors where funding is restricted, different risk premiums in different sectors, and particular fiscal regimes.

Proof of concept: Re-analyzing the case of Brazil by 2030

In a theoretical framework, Vogt-Schilb and Hallegatte (2014) find that using a MAC curve as a supply curvethat is disregarding constraints on implementation speed and focusing on short-term targets -would lead to suboptimal strategies, making the longer-term target more expensive to reach. In some cases, doing so would even lead to a carbon-intensive lock-in, making the longer-term target impossible to reach. They show how a simple optimization model that factors implementation speed in the analysis can be used to avoid this problem.

Here, we perform a proof of concept for these ideas, reanalyzing the data used at the World Bank to create a MAC curve for Brazil with MACTool (de Gouvello, 2010).

We first extract the long-term potential and emissionreduction speed from the emission-reduction pathway that was provided to MACTool, and use them to calibrate the model.

We then take the point of view of a social planner who chooses in 2010 an emission-reduction schedule to comply with an emission target, in two different simulations. In the first one, an emission-reduction target is set for year 2030 and the optimal emission strategy is derived. Then, the quantity of abatement obtained in 2020 in this optimal strategy is used as a target for 2020, and the MAC curve is used to design a mitigation strategy between 2010 and 2020, disregarding the longer-term objective. Finally, we investigate differences of the optimal emission reductions up to 2020 in the two simulations.

We find that because of technical inertia, using a MAC curve without taking into account long-term objectives would lead to insufficient short-term investments in metro, rail, waterways, and bullet train, all options with high potential, large costs and slow implementation speed. Instead, the abatement target is met by implementing marginal energy-efficiency improvement in refineries, which provide "lower-quality" abatement. Indeed, while these options are lower cost than clean transportation infrastructure, they have a much lower abatement potential in the long term, meaning that using them in the short term not opening the door to deeper reductions in the long-term.

Methods and data

We use a spreadsheet program based on the model proposed by Vogt-Schilb and Hallegatte (2014). The program provides the least-cost emission-reduction schedule that complies with the abatement target. As inputs, it requires a list of measures, characterized by a marginal abatement cost, a maximum diffusion speed, and a maximum abatement potential (Appendix A).

Note that the abatement potential may evolve through time. For instance, if available technology limits intermittent wind power to 20% of the electricity production and electricity production is expected to grow over time, then the abating potential of wind power grows over time. On the other hand, if natural resources provide only few opportunities to build dams, the abating potential of hydro power is fixed, regardless of total electricity demand growth. We thus extend the model by Vogt-Schilb and Hallegatte (2014) to allow for growing abatement potentials (see below and Appendix A).

We use data collected at the World Bank to build a MAC curve (using MACTool) for Brazil (de Gouvello, 2010). The MAC curve provides a list of emission-reduction measures, their marginal abatement cost, and the potential achievable by 2030.

While the list of measures and their cost can be used directly in our spreadsheet program (see the first two columns of Tab. 1), our program requires the full-abatement potential and diffusion speed. Since the diffusion speed and the full-abatement potential were not reported separately, we have to reconstruct them with indirect methods, using the emission-reduction pathways that were provided to MACTool. For each measure, the shape of the emissionreduction pathways can be classified in one out of three cases.

In the first case, emission-reduction pathways may be approximated by a two-phases piecewise-linear function as This particular emission-reduction measure illustrates that many emission reduction pathways (the plus signs +) may be approximated by a piecewise-linear curve (in red). The slope of the first piece provides the diffusion speed for that measure. The second part is interpreted as the maximum potential, that grows over time.

in Fig. 3. In this case, the diffusion speed is given by the slope of the first piece, and the second phase is interpreted as the growing full potential. About half the measures fall in this category.

Other emission-reduction pathways may be approximated by a single linear diffusion (Fig. 4a). In this case, the full potential is not binding before 2030. We calibrate the diffusion speed from the slope of the penetration pathway, and denote the lack of data on the full potential with a dot (•) in the two last columns of Tab. 1.

In some other cases the emission-reduction pathway lacks the first phase; abatement immediately "jumps" to a growing full-potential (Fig. 4b). We denote them with a dot in the diffusion speed column in Tab. 1. There is usually a handful of such cases in MAC curves exercises. One example from the Brazilian study is solid residues management. In the emission-reduction pathway, solid residues management is able to reduce emissions by more than 40 MtCO 2 in one year, and then grow at less than 1 MtCO 2 /yr. From the perspective of the user of a MAC curve, it is unclear whether this should be considered as a shortcoming in the data (if the investigation could not identify the constraints that limit the diffusion of solid residues management), or a realistic emission-reduction pathway (if solid residues management can actually save lots of GHG in a short time lapse). 7 To avoid this situation 7 Livestock and forest management is a particular example. In the emission-reduction pathways, this measures allows to save 229 MtCO 2 , that is almost one third of the total abatement potential by 2030, as soon as 2010. Since Brazil has already managed to reduce drastically its emissions from deforestation (-80% between 2004 and 2009), the study considered that this mitigation option is already enforced. Sustaining such effort over a long period will require that productivity gains in the livestock sector free-up pasture land fast enough to accommodate the growth of the livestock-agriculture sector without deforesting, as recommended in the Brazil Low-carbon study [START_REF] De Gouvello | Brazil Low-carbon Country Case Study[END_REF] For some abatement measures, the data needed to calibrate our model cannot be derived from the emission-reduction pathways. Traffic optimization (a) is an example of measure for which the long term potential is not binding because it cannot be reached before 2030. Solid residues (b) exemplifies that for some other measures, the diffusion speed cannot be assessed -either it was not investigated, or the measure can reach its full potential in less than one year.

in the future, we recommend that the terms of reference for the experts in charge of collecting data on emission reductions options should explicitly ask to report possible diffusion speeds (Appendix B). Finally, some emission-reduction measures (reforestation, air conditioning and rapid bus transit) were included in the list while lacking either a marginal abatement cost or an emission scenario. These measures, as well as those for which the diffusion speed could not be estimated, are discarded for the rest of the analysis. The remaining options allow to reduce Brazilian emissions in 2030 by 223 MtCO 2 (compared with 812 MtCO 2 in the original MAC curve).

Results

In a first simulation, we run our spreadsheet model to design the socially optimal strategy to achieve 223 MtCO 2 of emission reductions by 2030. The optimal emissionreduction strategy has the following characteristics (for transparency and reproducibility purposes, detailed results are displayed in Appendix C).

First, all negative-cost measures are introduced at full speed from year 2010, independently of the emission-reduction target. Indeed, these measures are desirable per se, as they bring more benefits than costs even in the absence of any carbon pricing or climate change impacts. 8Second, the least-cost strategy is to implement the positive-cost measures as late as possible, to benefit from the discount rate. This means that under climate targets expressed as an emission reduction in one point in time, such as -30% by 2030, the two-phase penetration pictured in Fig. 3 is not optimal for positive-cost measures. A better solution is to delay the implementation such that the maximum potential is reached just in time, when the target needs to be achieved. 9

Finally, the optimal emission reduction pathway to achieve 223 MtCO 2 in 2030 leads to 127 MtCO 2 of emission reductions in 2020.

To investigate how focusing on short-term targets may lead to suboptimal outcomes, we run a second simulation with the only constraint of reducing emissions by 127 MtCO 2 in 2020. We then investigate how the "optimal" solution provided by our model in this case compares to the first simulation.

In line with Vogt-Schilb and Hallegatte (2014), the least-cost strategy for 2010-2020 uses different emissionreduction options, depending on whether the strategy aims at a short-term target (127 MtCO 2 in 2020) or at a longerterm one (223 MtCO 2 in 2030). This is shown in Fig. 5, which depicts emission reductions achieved by 2020 in the two strategies, for selected emission reduction options. We chose the five emission-reduction measures with the highest difference between the two scenarios. The simulation that ends in 2020 uses notably less investment in metro and other clean transportation infrastructure, and more heat integration and other marginal improvements in existing refineries than what the 2030 simulation does by 2020.

Indeed, clean transportation infrastructure is characterized by a large abatement potential, and high cost per ton of CO 2 avoided. As illustrated in Fig. 6, these options are not implemented when short-term target masks 9 This is a downside of targets expressed in terms of reductions at one point in time. If the climate mitigation target was expressed in terms of a carbon budget (consistently with climate change physics, [START_REF] Zickfeld | Setting cumulative emissions targets to reduce the risk of dangerous climate change[END_REF], then the two-phase penetration target would be optimal (Vogt-Schilb and Hallegatte, 2014, section 4). the longer-term target. In addition, clean transportation infrastructure also takes a long time to implement, meaning that in the 2030 scenario, it is implemented as fast as possible -confirming the need for short term investment in clean infrastructure, as recently advocated by [START_REF] Waisman | The Imaclim-R model: infrastructures, technical inertia and the costs of low carbon futures under imperfect foresight[END_REF]; [START_REF] Framstad | Energy intensive infrastructure investments with retrofits in continuous time: Effects of uncertainty on energy use and carbon emissions[END_REF]; [START_REF] Kopp | Turning the Right Corner: Ensuring Development through a Low-Carbon Transport Sector[END_REF]; [START_REF] Lecocq | The economics of targeted mitigation in infrastructure[END_REF] and [START_REF] Avner | Carbon price efficiency: Lock-in and path dependence in urban forms and transport infrastructure[END_REF].

Moreover, because it takes so long to build clean transportation infrastructure, not starting doing it before 2020 closes the door to deeper emission reductions by 2030. Indeed, reaching the 2030 target requires the implementation of 95 additional MtCO 2 of abatement between 2020 and 2030. However, a 2020-2030 strategy would be able to save 84 MtCO 2 additionally at best, since not enough time would be left to deploy time intensive solutions. This new low-carbon scenario would therefore be short 11 MtCO 2 or 12% in 2030 compared to the first best. In other words, the 2030 target becomes impossible to achieve after 2020, as the limited diffusion speed prevents high-abatementpotential options to achieve their optimal 2030 level in only 10 years. This is an example of how delayed action in key sectors can create carbon lock-ins.

Conclusion

In order to put the economy on the track to deep decarbonization, 9 MtCO 2 of abatement achieved with metro may be worth more than 11 MtCO 2 achieved with energyefficiency improvements in refineries; for metro avoids locking the transportation system in carbon-intensive patterns, while energy-efficiency improvement in refineries has limited long-term potential.

Regardless of the process used to generate them, MAC curves cannot communicate this type of information to decision makers: they appear as static abatement supply curves, leaving any caveat regarding the dynamic aspect of mitigation strategies to method sections or footnotes. An easy solution to mitigate this issue may be to systematically display flipped MAC curves next to the corresponding emission-reduction pathway, also known as a wedge curve (Fig. 2).

More generally, the abatement potential and cost are not sufficient information to schedule emission-reduction measures. Both a long-term objective and the speed at which each option may deliver abatement are instrumental in deciding on the quantity and quality of short-term emission reductions.

With this information, decision makers can design policies aiming to achieve two objectives. The first is to remove implementation barriers on negative-and low-cost options. The second is to ensure short-term targets are met with abatement of sufficient quality -that is without under-investing in the ambitious abatement measures required to achieve long-term targets.

Figure 2 :

 2 Figure2: A "flipped" achievable-potential MAC curves next to the corresponding emission reduction scenario (wedge curve). By displaying together the cost, the potential, and the time required to implement the options, confusion on how to interpret MAC curves may be avoided.

Figure 3 :

 3 Figure3: Emission reductions achieved over time thanks to recycling. This particular emission-reduction measure illustrates that many emission reduction pathways (the plus signs +) may be approximated by a piecewise-linear curve (in red). The slope of the first piece provides the diffusion speed for that measure. The second part is interpreted as the maximum potential, that grows over time.

Figure 4 :

 4 Figure4: Emission reductions achieved over time thanks to traffic optimization and management of solid residues. For some abatement measures, the data needed to calibrate our model cannot be derived from the emission-reduction pathways. Traffic optimization (a) is an example of measure for which the long term potential is not binding because it cannot be reached before 2030. Solid residues (b) exemplifies that for some other measures, the diffusion speed cannot be assessed -either it was not investigated, or the measure can reach its full potential in less than one year.

Figure 5 :

 5 Figure5: Comparison of emission reduction achieved in 2020 with a set of measures when the 2020 target is the final target vs. when it is a milestone toward a more ambitious 2030 target. The picture shows the five emission-reduction for which the difference between the two strategies are the largest.

Figure 6 :

 6 Figure6: Two achievable-potential MAC curves, built for 2020 and 2030. The 2020 MAC curve (a) suggests that the 2020 target (D2020) can be met using only options 1-4, disregarding option 5 before 2020. But then only a fraction of option 5 could be implemented between 2020 and 2030. The 2030 MAC curve (b) however shows that options 1-5 should be implemented by 2030 to meet the D2030 target. For option 5 to deliver all the abatement listed by 2030, it should be implemented before 2020.

Table 1 :

 1 Author manuscript accepted in Climate Policy doi:10.1080/14693062.2014.953908 Calibrated speed, cost and potential of the measures in the Brazilian study. A dot (•) denotes lack of reliable data.

	Measure	MAC	Diffusion speed Potential in 2010 Potential growth
		$/tCO 2	ktCO 2 /yr	MtCO 2	ktCO 2 /yr
	Combustion optimization	-28.4	955	3.3	218
	Heat recovery	-59.6	168	0.6	37
	Steam recovery	-62.4	339	1.1	77
	Furnace heat recovery	-12.8	1780	8.6	743
	New processes	25.8	1200	4.5	265
	Other Energy Efficiency	-7.5	162	0.6	35
	Thermal Solar	-34.8	228	0.8	50
	Recycling	-23.6	679	2.3	155
	Natural gas	0	397	1.3	90
	Biomass	4.3	716	•	•
	Reforestation	•	•	26.9	1002
	Wind	64	138	1.2	0
	Comb. Heat Power	-43.2	1517	5.7	241
	Solar heat	83.9	18	•	•
	Air conditioning	419.1	•	0	0
	Residential Lightning	-91.9	•	0.1	0
	Cooler	5.2	79	•	•
	Motor	-5.8	13	•	•
	Industrial Lightning	-36.2	3	•	•
	Commercial lightning	-27.3	9	•	•
	GTL	0.6	1021	•	•
	New Refineries	16.4	352	•	•
	Refineries Heat Integration	10.9	510	3.1	37
	Refineries Fouling Mitigation	45.8	59	0.5	0
	Refineries Advanced Control	79.1	59	0.5	0
	Ethanol	1.8	1444	•	•
	Rail and Waterways	23.3	494	•	•
	Bullet train	376.3	45	0.9	0
	Rapid transit bus	42	•	0	0
	Metro	95.7	1007	•	•
	Traffic optimization	0.2	232	•	•
	Bike Lanes	2.6	120	•	•
	Solid residues	2.1	•	40.4	732
	Resid. wastewater	7.8	513	•	•
	Indust. Wastewater	80.4	•	8	333
	Restauration	•	5899	•	•
	Livestock and Forest	0.7	•	229.4	6542
	Tilage	-0.2	2578	17.6	185

). Author manuscript accepted in Climate Policy doi:10.1080/14693062.2014.953908 (a) No potential (b) No speed

The investor's MAC curves commissioned by the EBRD are noticeable exceptions, as they factor energy subsidies, the high cost

of capital that the private sector faces, and positive transaction costs in their assessment of the abatement cost of each option[START_REF] Nera | The demand for greenhouse gas emissions reductions: An investors' marginal abatement cost curve for turkey. Prepared for the european bank for reconstruction and development[END_REF][START_REF] Nera | The demand for greenhouse gas emissions reduction investments: An investors' marginal abatement cost curve for ukraine. Prepared for the european bank for reconstruction and development[END_REF][START_REF] Nera | The demand for greenhouse gas emissions reduction investments: An investors' marginal abatement cost curve for kazakhstan. Prepared for the european bank for reconstruction and development[END_REF]).2 In this paper we assume implementation barriers make the implementation of measures slower, without affecting their cost. In economic theory, an alternative approach is to consider adjustment costs that capture a trade-off between implementing options quickly and implementing them at low cost[START_REF] Vogt-Schilb | How inertia and limited potentials affect the timing of sectoral abatements in optimal climate policy[END_REF][START_REF] Lecuyer | Optimal transition from coal to gas and renewable power under capacity constraints and adjustment costs[END_REF].

While existing MAC curves in the gray literature are mainly derived from expert surveys, the academic literature frequently studies emission reduction pathways with energy system models or integrated assessment models.

While the literature consistently calls these curves marginal abatement cost curves, in most occasions the cost of each option is computed as an average cost, as the net present cost of using that Author manuscript accepted in Climate Policy doi:10.1080/14693062.2014.953908

For instance, if fuel-cell vehicles are much more expensive than EVs, but do not suffer from limited autonomy, the optimal strategy would be to use EVs when possible (say for 25% of the fleet), and fuel cell vehicles otherwise. In this case, the full-potential MAC curve could depict an abatement potential of 25% of private-mobility related emissions for EVs, and

75% for fuel cell. In the absence of EVs, fuel cell vehicles could abate 100% of private mobility emissions, but this information would not appear in the curve.Author manuscript accepted in Climate Policy doi:10.1080/14693062.2014.953908

Author manuscript accepted in Climate Policy doi:10.1080/14693062.2014.953908

Remember that our framework accounts for implementation barriers that lower the speed at which emission reduction options may be implemented, but do not increase their cost.

The model assumes that abatement costs are linear, such that marginal and average cost coincide.

In the model proposed by Vogt-Schilb and Hallegatte (2014), abatement potentials do not evolve over time. This is the only extension we propose.

Acknowledgments

We thank Pierre Audinet, Mook Bangalore, Luis Gonzalez, Jean-Charles Hourcade, Pedzi Makumbe, Baptiste Perissin-Fabert, Julie Rozenberg, Supachol Suphachalasai, three anonymous referees and seminar participants at the French Ministère du Développement Durable, the World Bank, and the International Energy Agency for useful discussions on the work performed here. All remaining errors are the authors' responsibility. We thank the ESMAP (World Bank) for financial support. The views expressed in this paper are the sole responsibility of the authors. They do not necessarily reflect the views of the World Bank, its executive directors, or the countries they represent. A previous version of this paper is Vogt-Schilb et al. (2014)

Author manuscript accepted in Climate Policy doi:10.1080/14693062.2014.953908

Appendix A. Model

We extend the model proposed by Vogt-Schilb and Hallegatte (2014). As inputs, the model takes a set of measures (indexed by i), their respective abatement potential A i,t , (marginal) abatement costs c i , 10 maximum diffusion speeds v i , an abatement target a T , set for a date in the future T (e.g. 2020 or 2030), and a discount rate r.

The model computes the least-cost schedule a i,t of emission reductions done with each measure i at each time t:

The model takes into account the constraint set by maximum abatement potentials: 11

The second constraint on emission reduction is that they cannot grow faster than the diffusion speed v i , such that:

Finally, the abatement target sets the following constraint:

An Excel implementation of this model is available online.

Appendix B. Information collection guidance

The following proposes guidance on how data on emission reduction measures could be collected to take into account the findings of this paper. The objective is to collect data that can be used to build emission reduction pathways and MAC curves in order to inform climate mitigation policies. Asking specifically to disclosure assumptions on the diffusion speed of each option (3c) should help identify bottlenecks preventing some measures to be implemented.

Note that collecting this data does not require more work that what is currently done to build MAC curves from expert surveys; clarifying the difference between implementation speed and full technical potential may actually facilitate the data-gathering process.

Of course, this sketch should be adapted to local conditions; for instance, it should account for existing plans and projections when defining emission baseline and abatement potentials.

Inventory of existing GHG emissions

(a) Provide the list of GHG emissions at a given date in the recent past. Chose the most recent date for which data is available . (b) Provide a breakdown of these emissions by sector, e.g. power generation, industry, buildings, transportation, agriculture. Use sub-sectors where possible, for instance as provided by the International Standard Industrial Classification. (c) Describe current output of these sectors.

i. Use physical measures of output when possible, e.g: A. In the transportation sector, use passengerkilometer and ton-kilometer. B. In the power sector, use MWh/yr. C. In the residential sector, use number of inhabitants at given comfort. ii. Express these emissions in CO 2 equivalent using accepted conversion factors. 2. Prospective: provide projections of future GHG emissions reported in 1 using the same breakdown. Report relevant drivers, such as population projections, GDP growth, etc. 3. List available emission-reduction measures (a) Full technological potentials i. Provide emission intensity of each activity (e.g., gCO 2 /km).


Jodi Schneider
email: jodi.schneider@inria.fr

Paolo Ciccarese
email: paolo.ciccarese@gmail.com

Tim Clark
email: timclark@harvard.edu

Richard D Boyce

Using the Micropublications ontology and the Open Annotation Data Model to represent evidence within a drug-drug interaction knowledge base

Keywords: Linked Data, drug-drug interactions, evidence bases, Micropublications, Open Annotation Data Model, knowledge bases

Semantic web technologies can support the rapid and transparent validation of scientific claims by interconnecting the assumptions and evidence used to support or challenge assertions. One important application domain is medication safety, where more efficient acquisition, representation, and synthesis of evidence about potential drug-drug interactions is needed. Potential drug-drug interactions (PDDIs), defined as two or more drugs for which an interaction is known to be possible, are a significant source of preventable drug-related harm. The combination of poor quality evidence on PDDIs, and a general lack of PDDI knowledge by prescribers, results in many thousands of preventable medication errors each year. While many sources of PDDI evidence exist to help improve prescriber knowledge, they are not concordant in their coverage, accuracy, and agreement. The goal of this project is to research and develop core components of a new model that supports more efficient acquisition, representation, and synthesis of evidence about potential drug-drug interactions. Two Semantic Web models-the Micropublications Ontology and the Open Annotation Data Model-have great potential to provide linkages from PDDI assertions to their supporting evidence: statements in source documents that mention data, materials, and methods. In this paper, we describe the context and goals of our work, propose competency questions for a dynamic PDDI evidence base, outline our new knowledge representation model for PDDIs, and discuss the challenges and potential of our approach.

Introduction

Scientific knowledge depends on the verification and integration of large systems of interconnected assertions, assumptions, and evidence. These systems are con-tinually growing and changing, as new scientific studies are completed and new documents are published. The state of current knowledge in any given domain can be difficult for any one individual to fully grasp, because bits of knowledge are updated at frequent intervals.

In the biosciences, this problem has taken on particular importance, due to an exponential growth in the aggregate publication rate. Manually curated databases are used to record certain types of knowledge. To update and maintain these databases, curators must make knowledge-intensive decisions, identifying the best available evidence in the current scientific literature. Maintaining such databases is challenging because there is limited tracking of the source information.

In an ongoing project, we are experimenting with using the Micropublications Ontology4 [Clark2014] and the Open Annotation Data Model5 [W3C2013] to create an audit trail between assertions, evidence, and source documents, so that assertions and evidence can be flagged for update in flexible and intelligent ways. Updates may be needed when the underlying sources change, when a particular method for establishing an assertion is discredited, etc. Our goal is to provide better linkages between an assertion recorded in a knowledge base and its supporting evidence (i.e., data, materials, and methods) found in source documents.

In the remainder of the paper, we describe the competency questions for our evidence base and the new evidence model that we are creating, which combines the Micropublication Ontology and the Open Annotation Data Model, and adapts them to the existing evidence modeling of the Drug Interaction Knowledge Base6 [Boyce2007,Boyce2009]. We then reflect on how the new model performs for our goal of creating an audit trail between assertions, evidence, and source documents.

Context and goals

Our work is in the context of a larger project on organizing and synthesizing scientific evidence from the biomedical literature on potential drug-drug interactions. Potential drug-drug interactions (PDDIs), defined as two or more drugs for which an interaction is known to be possible, are a significant source of preventable drug-related harm (i.e., adverse drug events, or ADEs). The combination of poor quality evidence on PDDIs, and a general lack of PDDI knowledge by prescribers, results in many thousands of preventable medication errors each year. While many sources of PDDI evidence exist to help improve prescriber knowledge, they are not concordant in their coverage [Saverno2011], accuracy [Wang2010], and agreement [Abarca2003]. Difficulties with synthesizing evidence, and gaps in the scientific knowledge of PDDI clinical relevance, underlie such disagreement.

To address these problems, our research group is studying the potential benefit of applying recent developments from the Semantic Web community on scientific discourse modeling and open annotation. The goal is to develop core components of a new PDDI knowledge representation model that will support a more efficient acquisition, representation, and synthesis of PDDI evidence. The desired knowledge representation will provide better linkages between PDDI assertions and their supporting evidence, by directly connecting to annotated section(s) of relevant source documents.

Approach

Our new approach will draw upon the current version (1.2) of the Drug Interaction Knowledge Base [Boyce2007,Boyce2009], the Open Annotation Data Model [W3C2013], and the Micropublications Ontology [Clark2014].

The Drug Interaction Knowledge Base (DIKB) is a static, manually constructed evidence base that indexes assertions and evidence of PDDI for over 60 drugs. Its taxonomy of assertion types and evidence types [Boyce2014] is a starting point for the new knowledge base. The current version of the DIKB implements a version of the SWAN semantic discourse ontology [Ciccarese2008] to represent evidence relations. Specifically, the knowledge base uses swanco:citesAs-SupportingEvidence and swanco:citesAsRefutingEvidence to link to an entire source document as a supporting or refuting citation. At the time the DIKB 1.2 was constructed (2007-2009), annotation methodologies were less well developed. Consequently, version 1.2 of the DIKB stores quotes as textual strings manually copied from source documents. The text has been enriched with metadata about the source section, but it is non-trivial to return to the appropriate segment of the text from this information.

Our use of the Open Annotation Data Model (OA) reflects a change in the state of the art. OA is an "an interoperable framework for creating associations between related resources, annotations, using a methodology that conforms to the Architecture of the World Wide Web"7 . In particular, OA allows an evidence database to provide explicit connections from quotes to their source documents. For example, as shown in Figure 1, an OA resource can be used to quote a specific part of a drug product label (also known as a summary of product characteristics) to indicate evidence that escitalopram inhibits CYP2D6. In general, OA enables queryable links between selections from source documents (as target) to the instances of data, methods, and materials (as body) that we want to model to support drug interaction knowledge base use cases.

Similarly, the Micropublications Ontology improves the depth with which evidence can be represented and queried. The most important feature of the Micropublications model, in our view, is its ability to represent the data, methods, and materials that act as support for a claim, and to transitively close chains Fig. 1. The Open Annotation ontology (oa) can be used to quote the evidence (here a method described in a DailyMed product label) and associate it with an instance of the Micropublication ontology (mp). The annotation records quoted text (oa:exact) as the target, while the body of the annotation is a mp:Method instance, Me1, supporting the Claim Escitalopram inhibits CYP2D6 shown in Figure 2. We use existing terminology from the DIKB ontology to specify the section of the DailyMed product label and to indicate the dikbEvidence type.

of claims8 and citations across the literature to their fundamental supporting evidence. A mp:Micropublication mp:argues a mp:Claim based on connecting any number of mp:Representations. The whole Micropublication is a Representation, as are Data and Methods (including Materials and Procedures), whether textual or pictoral. A mp:Representation may mp:support or mp:challenge any other mp:Representation, making the evidence explicit and queryable.

Competency Questions

To design an appropriate enhancement of the DIKB model with Micropublications and the Annotation Ontology, we need to understand what sorts of questions experts would like to retrieve about the PDDIs. The competency questions below were elicited from experienced editors of clinically oriented drug compendia during the process of developing DIKB 1.2. Most fall into three categories: finding assertions and evidence; assessing the evidence; and enabling updates. A second area of interest is statistical information about the evidence base which is useful for various analytics related to knowledge base maintainance.

Finding assertions and evidence

Finding assertions:

(a) List all assertions that are not supported by evidence (b) Which assertions are supported (or refuted) by just one type of evidence? (c) Which assertions have evidence from source X (e.g., product labeling) (d) Which assertions have both evidence for and evidence against from a single source X? 2. Finding evidence:

(a) List all evidence for or against assertion X (by evidence type, drug, drug pair, transporter, metabolic enzyme, etc.) (b) What is the in vitro evidence for assertion X? the in vivo evidence? (c) List all evidence that has been flagged as rejected from entry into the the knowledge base (d) Which single evidence items act as support or rebuttal for multiple assertions of type X (e.g., substrate of assertions)?

4.2 Assessing the evidence:

1. Understanding evidence coming from a given study: (a) What data, methods, materials, are reported in evidence item X? (b) Which evidence items are related to and follow-up on evidence item X? (c) Which research group conducted the study used for evidence item X? (d) Are the evidence use assumptions for evidence item X concordant? unique? non-ambiguous?

2. Verifying plausibility of an evidence item: (a) Has evidence item X been rejected for assertion Y? If so, why and by whom? (b) Which other assertions are being supported/challenged by this evidence item? (c) What are the assumptions required for use of this evidence item to support/refute assertion X? 3. Checking assertions about pharmacokinetic parameters (i.e., area under the concentration time curve (AUC)) (a) How many pharmacokinetic studies used for evidence items in the DIKB could be used to support or refute an assertion about pharmacokinetic paramater X (e.g., 'X increases AUC')? (b) How many pharmacokinetic studies in the DIKB used for evidence items for assertion X are based on data from the product label? (c) What is the result of averaging (or applying some other statistical operation) to the values for pharmacokinetic parameter X across all relevant studies used for evidence items? 4. Checking for differences in the product labeling:

(a) Are there differences in the evidence items that were identified across different versions of product labeling for the same drug? (b) What version of product labeling was used for evidence item X? Original manufacturer or repackager? Most current label or outdated? Is the drug on market in country X or not? American or country X?

Supporting updates to evidence and assertions

1. Changing status of redundant and refuted evidence: (a) Remove a older version of a redundant evidence item (b) Change the modality of a supporting evidence item to be a refuting evidence item 2. Updating when key sources change:

(a) Get all assertions that are supported by evidence items identified from an FDA guidance or other source document just released as an updated version.

Understanding the evidence base

1. Statistical information about the evidence base: (a) Number of assertions in the system (b) Number of evidence items for and against each assertion type (c) Show the distribution of the levels of evidence for various assertion types (e.g., pharmacokinetic assertions)

Figure 2 shows how the new Micropublications model of evidence on PDDIs would represent some of the evidence supporting and challenging the assertion escitalopram does not inhibit CYP2D6. We created the example by hand using a sample assertion and evidence items from the DIKB version 1.29 .

C1 S2

Steady state levels of racemic citalopram were not significantly different in poor metabolizers and extensive CYP2D6 metabolizers after multiple-dose administration of citalopram, suggesting that coadministration, with escitalopram, of a drug that inhibits CYP2D6, is unlikely to have clinically significant effects on escitalopram metabolism.

Escitalopram does not inhibit CYP2D6

In vitro studies did not reveal an inhibitory effect of escitalopram on CYP2D6. S1

D1

Coadministration of escitalopram, a substrate for CYP2D6, resulted in a 40% increase in Cmax and a 100% increase in AUC of desipramine. The Micropublications ontology is used to structure the evidence relating to data, methods, and materials, and the overall indication that evidence mp:supports or mp:challenges a mp:Claim. We qualify Claims (C1 in the figure) by reusing identifiers from DRON10 [Hanna2013] and the Protein Ontology11 [Natale2011].

The new model reuses the DIKB evidence taxonomy12 to provide epistemic qualification (SQ2, SQ5, SQ6 in the figure) to statements (S1, S2, and S3 in the figure), data (D1 in the figure), methods (Me1 in the figure), and materials (not shown in this example). The Open Annotation Data Model (previously shown in Figure 1) is used to link quotes taken from source documents back to their originating information artifacts. The approach to modeling other DIKB assertions would be similar to this example.

Discussion

Expected Benefits

Certain benefits accrue from upgrading from the current DIKB. Many of the competency questions (Section 4) are not supported in the DIKB 1.2. The new model is designed to support these and additional questions relevant in the domain. Visual inspection of the model suggests that we will be able to answer some competency questions quite naturally. In particular, finding the assertions that are not supported by evidence already in the evidence base, the evidence that should be checked most thoroughly (e.g. evidence that by itself supports multiple assertions), and the data, methods, and materials associated with a given evidence item as described in source documents.

Further, as a Linked Data resource, our new knowledge base will also enable innovative queries using knowledge from other sources about tagged entities (i.e., drugs and proteins) represented in the evidence base. Unlike the current DIKB, we will be able to render annotations in their original context. We also expect to be able to support distributed community annotation/curation, since MP and OA take account of provenance, and since OA is being increasingly adopted by a variety of annotation tools.

Modeling challenges

Our project does raise certain modeling challenges. To date, MP has not been used to represent both unstructured claims and the related logical sentences. Figure 1 shows the assertion escitalopram does not inhibit CYP2D6 as unstructured text. However, the DIKB requires that 1) assertions about PDDIs be formulated by experts prior to collecting evidence, and 2) that the assertions be represented both as unstructured statements and sentences in a logical formalism. Careful thought is being put into how to properly accommodate this use case. Such challenges are to be expected since MP is a relatively new ontology and since this is a new application of it.

Another challenge is to ensure that, as the evidence base scales, competency questions can be answered efficiently. To address this, we building the model using an iterative design-and-test approach. In this process, efficient querying is a key requirement.

Other issues

For enabling synthesis over the PDDI information, the model is not the only concern. Applying this model will require integration work. One challenge is inherent to scholarly documents: the existing evidence items within the DIKB refer to many data, materials, and methods that exist only in PDF documents accessible only through proprietary portals or academic library systems. Consequently, resolving annotations requires a method for pointing to proprietary oa:targets.

Conclusions & Future Work

We are currently iterating and refining the PDDI evidence and annotation model. Once it is stable, we plan to use the new model to represent as Linked Data evidence collected by an evidence board consisting of drug experts. The evidence collection effort is planned as part of a research project funded by the National Library of Medicine ("Addressing gaps in clinically useful evidence on drug-drug interactions", 1R01LM011838-01) and will focus on PDDI assertions for a number of commonly prescribed drugs (anticoagulants, statins, and psychotropics). We plan to implement a pipeline for extracting PDDI mentions from a variety of publicly available sources, including published journal articles indexed in PubMed or PubMed Central, FDA Guidance Documents, and drug product labels from the National Library of Medicine's DailyMed website 13 . Candidate PDDI assertions will be linked by machine to the Internet-accessible versions of the information artifacts used as evidence.

An existing Micropublication plugin for Domeo [Ciccarese2014] is being modified as part of the project. Our plan is to use the revised plugin to support the evidence board with the collection of the evidence and associated annotation data. It will also enable the broader community to access and view annotations of PDDIs highlighted in a web-based interface. We anticipate that this approach will enable a broader community of experts to review each PDDI recorded in the DIKB and examine the underlying research study to confirm its appropriateness and relevance to the evidence base.

The usability of the annotation plug-in is critically important so that the panel of domain experts will not face barriers to annotating and entering evidence. This will require usability studies of the new PDDI Micropublication plugin. Another issue is that many PDDI evidence items can be found only in PDF documents. Currently, the tool chain for PDF annotation is relatively weak: compared to text and HTML, PDF annotation tools are not as widely available and not as familiar to end-users. Suitable tools will have to be integrated into the revised plugin.

Knowledge representations combining MP and OA have the potential to allow more granular and reusable representation of evidence (data, materials, and methods), which are needed for synthesizing contested knowledge at the state of the art from scientific documents. The knowledge representations we are now creating will be beneficial for integrating PDDI evidence, and we hope they will inspire an increased use of linked data for evidence synthesis in other domains.

Fig. 2 .

 2 Fig. 2. A model of the evidence for and against the assertion escitalopram does not inhibit CYP2D6. This is based on the Micropublications ontology, and reuses the evidence taxonomy (dikbEvidence), terms (dikb), and data from the DIKB. The Drug Ontology (DRON) and Protein Ontology (PRO) are reused in semantic qualifiers. A more detailed view of Method Me1 is shown in Figure 1.

http://purl.org/mp/

http://www.openannotation.org/spec/core/

http://purl.net/net/drug-interaction-knowledge-base/

http://www.openannotation.org/spec/core/

'Assertion' in DIKB terminology corresponds to a 'Claim' in the Micropublications model; this variation in terms is because the term 'claim' is used in a different sense in medical billing.

http://dbmi-icode-01.dbmi.pitt.edu/dikb-evidence/escitalopram_does_not_ inhibit_cyp2d6.html

http://purl.obolibrary.org/obo/dron.owl

http://pir.georgetown.edu/pro/

http://bioportal.bioontology.org/ontologies/DIKB

http://dailymed.nlm.nih.gov/dailymed/about.cfm

Acknowledgments

This work was carried out during the tenure of an ERCIM "Alain Bensoussan" Fellowship Programme. The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreement n o 246016, and a grant from the National Library of Medicine (1R01LM011838-01). We thank Carol Collins, Lisa Hines, and John R Horn for serving on the Evidence Panel of "Addressing PDDI Evidence Gaps", and for contributing to the competency questions presented here.


Krystian Samp
email: samp.krystian@gmail.com

Cédric Beuzit

Jodi Schneider
email: jschneider@pobox.com

Unifying the Shift and Narrow Strategies in Focus+Context Exploratory Search

Keywords: Focus + Context, Exploratory Search, Browsing, Shift and Narrow, Academic Search, Saffron. ACM Classification H.5.m. Information interfaces and presentation (e.g., HCI): Miscellaneous Human Factors, Design, Measurement

In this paper we discuss two existing exploration strategies -Shift and Narrow -employed by Focus+Context techniques, and how they are supported in the user interface of Saffron, a web-based system enabling exploration of academic topics, authors, and publications. The Shift strategy enables the user to shift focus between different resources while the Narrow strategy enables the user to narrow the focus. Current systems typically support only one of these approaches or include them as separate interaction modes. Saffron supports both strategies in a unified user interface. An initial user study indicates that participants use and appreciate both strategies being supported simultaneously.

INTRODUCTION

Focus+Context is one of the predominant approaches for building browsing interfaces [START_REF] Herman | Graph Visualization and Navigation in Information Visualization: A Survey[END_REF]. Focus provides a user with local detail while global Context provides access to other parts of the information space.

Context is the main facilitator of exploration and can support different browsing strategies. Current interfaces typically choose to support one of the two strategies to which we refer in this paper as Shift and Narrow. The Shift strategy enables the user to shift the focus from one resource to another while the Narrow strategy enables the user to progressively narrow the focus area. For example, while viewing a page about a topic, clicking on a new topic could either: a) Shift to a page about the new topic (irregardless of the current results) b) Narrow to results that meet BOTH the current context AND the new topic.

In this paper we focus on the exploration of academic collections consisting of three types of resources: topics, authors, and publications. In exploratory interfaces for academic collections, local detail provides information about a single "in focus" resource. By contrast, the global context consists of the resources most related to the current focus. For example, consider a publication as the "in focus" resource. Its local detail includes the title, abstract and year of publication. Meanwhile the global context could consist of the authors, the topics covered in the publication, and the conference where the publication was presented. Notice that the global context provides interactive information that can enable further exploration.

Current exploratory interfaces for academic collections generally support either the Shift or Narrow strategy. Both strategies prove useful but there has been little work integrating support for the two strategies in a unified interface.

In this paper we describe the Shift and Narrow strategies and demonstrate how they are supported in a unified user interface of our exploratory search tool Saffron1 -a web system for exploring academic collections. We argue that interfaces integrating both strategies are feasible and useful.

First we present related work. Next we describe the Shift and Narrow strategies and present the guiding design principles for supporting both strategies. We then describe how the two strategies are supported in Saffron's unified exploratory user interface. A user study follows where we collect initial feedback from participants. We close the paper with discussion and conclusions.

RELATED WORK

ArnetMiner [START_REF] Tang | ArnetMiner: extraction and mining of academic social networks[END_REF] is an academic expert finder based on social network mining and extraction algorithms. It enables browsing by statistics (e.g., H-index), geo-location, and topics. Expert pages provide author information such as a bio, affiliation, education (local detail) as well as links related to the person's expertise (global context): topics, conferences, and publications. ArnetMiner is an example of a system supporting the Shift strategy. Following links from the global context results in shifting the focus from one resource to another. [START_REF] Hearst | NLP support for faceted navigation in scholarly collections[END_REF] present a system that extracts topics from publication titles and abstracts. These topics are then used to construct the global context which enables multifaceted browsing. As in the typical multi-faceted interface the user can progressively add or remove constraints, in this case the extracted topics, to the result set. The system is an example of a system supporting the Narrow strategy. Following links from the global context results in narrowing the focus to a smaller subset of the information space.

Hearst and Stoica

Jones and Paynter [START_REF] Jones | Topic-based browsing within a digital library using keyphrases[END_REF] present two user interfaces for exploring academic collections. The Kniles interface supports the Shift strategy by providing keyphrases that link to lists of the documents to which the phrases have been assigned. The Phrasier interface supports the Narrow strategy: the user can select multiple phrases from a list to narrow the result set of the recommended documents.

The ACM Digital Library [START_REF][END_REF] is an example of a system that provides both Shift and Narrow browsing capabilities. The links from conferences to publications, to authors, to authors' colleagues, to subject areas and beyond, support the Shift strategy. The Narrow strategy is realized in the browsing interface based on the ACM Computing Classification System -a predefined hierarchy of computer science subjects. The user can drill down the hierarchy from general areas to more specific subjects which link to relevant publications.

Another system providing both Shift and Narrow strategies is Microsoft Academic Search [START_REF]Microsoft Academic Search[END_REF]. The user can narrow the focus to a particular field of study (e.g., Computer Science) and futher to a sub-domain (e.g., Human-Computer Interaction) and then move between resources in a manner similar to ArnetMiner and ACM Digital Library.

The last three systems are examples of how Shift and Narrow strategies are currently supported. They are either supported by two separate interfaces (ACM Digital Library, the Kniles and Phraiser interfaces) or they can be used in succession: first narrow and then shift between resources (Microsoft Academic Search). The two strategies are not supported at all times during the exploration, and rather than a single unified user interface, each system uses two different interfaces, one for Shift and the other for Narrow. We argue in this paper that it is feasible and beneficial to support both strategies simultaneously. In the Shift strategy (see Figure 1 left) a resource from the current context (Figure 1 middle) is selected to become the new focus. The user shifts from one location in the information space to another. The Shift strategy is used to perform broad exploration and understand connections in the information space [START_REF] Hearst | Search user interfaces[END_REF]. As the user's understanding develops, some parts of the information space might be selected for more detailed exploration.

SHIFT AND NARROW EXPLORATION STRATEGIES

In the Narrow strategy (see Figure 1 right) a resource from the current context (Figure 1 middle) is selected to be combined with the current focus. The role of the Narrow strategy is to provide information on a more narrowed area of the information space. Narrow is used when the user indicates interest in a specific aspect and formulates more focused questions [START_REF] Kuhlthau | Inside the search process: Information seeking from the user's perspective[END_REF][START_REF] Marchionini | Exploratory search: from finding to understanding[END_REF].

GUIDING DESIGN PRINCIPLES

To guide the design process of a unified user interface we developed a set of guiding principles for supporting the Shift and Narrow strategies in Focus+Context techniques. Based on our own experiences and user feedback these principles have been refined over time. In this section we present these principles.

Supporting the Shift and Narrow Strategies

First we define three common guiding design principles for supporting both the Shift and Narrow strategies:

1. The context should include resources of different types to diversify opportunities for further exploration.

2. The context should include resources that enable both Shift and Narrow strategies. Note that the same resource cannot always be used to support both strategies (the next section gives an example).

3. The context should communicate visually which resources can be used for Shift vs Narrow interactions.

Supporting the Shift Strategy

We define four guiding design principles for supporting the Shift strategy:

1. To keep the conceptual model coherent the new context should share some information with the previous context. As a minimum the new context should include the previously "in focus" resource.

2. The new context should contain new information not available in the previous context to enable access to new locations of the information space.

3. The interface should explain how the new context resources are related to the current focus.

4. The context should balance between the resources that are most related to the focus and those that provide the best diversity and reach to other, different parts of the information space.

Supporting the Narrow Strategy

We define four guiding design principles for supporting the Narrow strategy:

1. The new context should prioritize information relevant to the combination of the selected resources. The context should not be just a Boolean AND of the other resources' contexts.

2. The new context does not have to contain less information but it should be more specific.

3. Narrowing need not be limited to two resources but can enable combining several at once.

4.

The user interface should provide mechanism to remove resources from the current combination (focus) and to revert back to previous steps.

SAFFRON

Saffron is a web-based exploratory search system enabling exploration of academic topics, publications and authors. Saffron extracts topics from the publications and connects the topics to both authors and publications with weighted links. Saffron has been deployed at several conferences including LREC 2011, EKAW 2012, and SIGIR 2013. It is also one of the official browsers for the ACL Anthology [START_REF]ACL Anthology[END_REF].

In this section we demonstrate how both the Shift and Narrow strategies have been designed into Saffron's user interface.

Initial screen

In the initial Saffron screen (see Figure 2) the user can select one of the hot topics2 or perform a keyword search for topics, authors, and publications. In the case of search, the query is matched against the available metadata, the results are returned, and the user can select one of the resources.

Topic, Author, and Publication Pages

Selecting a topic, author, or publication brings the user to the page for that resource. There is a consistent visual structure across the pages. Resource pages always consist of the local detail (at the top) and the global context (at the bottom). The global context displays a series of ranked lists of resources, each of a single type.

An example for a topic page is shown in Figure 3. The local detail shows the definition 3 and trends of the topic. The global context gives three ranked lists of resources: cooccurring topics, experts related to this topic, and related publications.

The global contexts are adapted to make sense for each resource. For an author page the global context gives coauthors, expertise topics, and publications. For a publication page, the global context gives the publication's authors, key topics of the publication, and similar publications.

To keep the context compact, each ranked list in the context displays only a limited number of resources. However, the user can expand each list by pressing the "more" button located on the top right-hand side of each list (see Figure 3).

Exploration with Shift and Narrow

The global context of the current resource gives the user two interactive options for exploration.

Supporting the Shift Strategy with Context Resources

The user can click on any resource from the global context to move to that resource page. In this manner the user shifts the current focus. For example, from a topic page (Figure 3), the user can shift to a co-occuring topic, an expert, or a publication; similarly for the other global contexts. This means that the user can start with a topic, move to a cooccurring topic, move to an expert, move to one of the expertise topics, and end up in a publication. This mechanism supports the Shift strategy.

Supporting the Narrow Strategy with Context Resources

To narrow, the user can click on a plus button located next to a resource in the global context (see Figure 3 and Figure 4). The system combines the new resource with the current focused resource, generating a page showing the local detail and the global context for the new combination. For example, combining a topic with one of its co-occurring topics will show a page giving the trend for how many times both topics co-occurred over the years (local detail) followed by a list of topics co-occurring with the two selected topics, a list of experts on the two topics, and publications covering the two topics (global context).

Another example is combining an author with one of his/her expertise topics. In this case the global context will include the author's publications on the selected topic, other topics mentioned by the author with the selected topic, and who worked with the author on the selected topic. The "plus button" mechanism supports the Narrow strategy.

Note that not all resources in the global context have a corresponding plus button as not all combinations make sense. For example, Saffron does not allow combining an author and one of his/her publications, since it is not clear what this would mean.

Currently Saffron supports combining topics with other topics (up to three topics), an author with a topic, and a topic with an expert.

Shifting through Breadcrumbs

As the user shifts and narrows in the information space Saffron constructs a navigation path in the form of the breadcrumbs located at the top of each page (see Figure 3). Shifting between resources merely substitutes the current resource in the breadcrumbs while narrowing adds more resources (the additional constraints) to the breadcrumbs.

Each individual breadcrumb item can be removed with an X button. Alternatively, the user can click on any of the breadcrumb items in order to navigate to the page for the corresponding resource. This mechanism is another way of supporting the Shift strategy although a very limited one as it only allows the user to move to previously visited parts of the information space and not new unexplored areas.

Summary

In contrast to other systems described earlier in the Related Work, in Saffron's user interface the Shift and Narrow facilities are interwined and coexist at any given point of the exploration. The user can shift between resources and at any point decide to narrow down the focus. While in the narrowed view the user can step back, narrow further, but also directly shift to any resource of the current global context.

INITIAL USER STUDY

We conducted an initial user study to collect feedback on the Shift and Narrow mechanisms as implemented in the Saffron interface. We recruited 12 researchers from a large research institute and rewarded them with candy for their time.

Apparatus

We used an instance of Saffron based on the proceedings from the last years of the following conferences: CLEF, TREC, NTCIR, ACL, LREC, ISWC, ESWC, WWW, and WRRS. The collection consisted of a total of 12,461 publications, 35,121 authors, and 17,985 topics. We made sure that the recruited participants were interested in the collection (i.e., they were working in the domains of IR, NLP, and Semantic Web).

The Saffron instance was instrumented to log all user interactions such as click on a topic, click on a plus button next to a topic, remove breadcrumb item, and so on.

Procedure and Tasks

The study consisted of three parts:

Explanation (5 minutes). Participants were briefed on: a) the procedures of the study, b) what Saffron is, c) all of the user interface functionality. We explained the user interface functionality as we were interested in how participants use Shift and Narrow mechanisms rather than how easy it is to learn the purpose of these mechanisms.

Unstructured Exploration (15 minutes). We asked participants to explore the collection based on their interests. We mentioned that participants could look into their own research areas but also into less known research areas that they found interesting while browsing.

Questionnaire and Interview. We asked participants to rank on a 1-7 Likert scale how much they agree / disagree with a number of statements investigating the usefulness of the Shift and Narrow functionality (these statements are provided later). We further had a free-form interview (5-20 minutes) with each participant in which they were encouraged to express their experiences with the system.

Results

We collected 682 interactions in total. There were an average of 57 interactions (SD 26) per participant.

All participants, except one, employed both the Shift and Narrow interface mechanisms. In total 296 interactions (43% of all interactions) were Shift moves (clicks on a topic, person, or publication) while 102 interactions (15%) were Narrow moves (clicks on a plus button of a topic or person). There were 98 interactions (14%) to remove a breadcrumb item and 68 searches (10%). The number of other types of interactions were much smaller.

For the Shift moves participants used topics the most (148 times), approximately as twice often as authors (80) and publications (68). For the Narrow interactions participants again used topics the most (92), followed by authors [START_REF] Tang | ArnetMiner: extraction and mining of academic social networks[END_REF].

Participants combined two topics 68 times, three topics 30 times, and one topic with one person 24 times.

Figure 5 shows the questionnaire results. As shown in Q3 most participants find it useful to have both strategies available at the same time.

DISCUSSION

The feedback gathered in our initial study hints at the usefulness of the Saffron's user interface combining Shift and Narrow strategies.

User Interactions and Subjective Perception

The Shift and Narrow interactions were the top two actions performed by participants and accounted for over 58% of all collected interactions. All participants, except one, employed both strategies which suggests they found them useful on different occasions. This is also reflected in the subjective responses. Participants found the combination of the two interaction mechanisms useful for exploration. These data support the idea supporting the Shift and Narrow strategies in a single unified interface.

Informal Feedback

During the interview session we received rich feedback.

Participants were enthusiastic about the system and about the two interface mechanisms for exploration. All participants were able to explain the difference between clicking on a resource and clicking on the corresponding plus button.

One participant did not use the plus button and explained that this was because she focused on a specific area and was able to reach it using search. However, the participant pointed that the ability to combine resources would be useful in other scenarios.

Eight participants pointed that not all combinations are supported by the interface and that they wanted to use them for exploration. Further questions revealed that they were interested in combining two authors, multiple topics with expert, and also publication with its author. The last combination (publication with its author) was something we initially thought would make little sense. But the participant explained:

"

.] when I continue moving between the pages and find an interesting publication I'd like to know which topic I searched for initially"

These participants were indirectly asking for the visual history of the navigation actions. The history should explain the navigation trajectory and provide the ability to shift to any of the previous steps.

CONCLUSIONS

In this paper we described the Shift and Narrow exploration strategies and argued that it is both feasible and beneficial to combine both in a single unified user interface. We demonstrated one such integration in the Saffron user interface for exploring academic collections. In the initial study we found that participants use the Shift and Narrow interface mechanisms and find them useful in exploration.

Participants were enthusiastic about the interface mechanisms and understood their different purposes.

The results also hint at the challenges associated with combining the Shift and Narrow strategies in a single interface. One such challenge is to better explain the users their trajectory through the information space: when and where the user took the Shift and Narrow turns. Another challenge is to construct an effective global context that balances opportunities for the Shift and Narrow strategies. The context will have to assure diverse reach into other parts of the information space as well as meaningful combinations for more specific interests.

We hope that future exploratory interfaces can consider the possibility of interwining both strategies in a single interface. And that our design principles can provide helpful guidance.

In the future work we would like to conduct a formal study with structured tasks. We also plan to extend current Saffron interface and support more combinations requested by participants. Finally, we are interested in testing the unified interface with other collections.

Figure 1

 1 Figure 1 presents the concepts of the Shift and Narrow exploration strategies.

Figure 1 A

 1 Figure 1 A pictorial representation of the Shift (left) and Narrow (right) exploration strategies in the Focus+Context technique, starting from the Initial Focus+Context view (middle). (middle) The focus is represented by a small black circle. Context resources of different types are depicted as different gray shapes. (left) The Shift strategy selects a new resource from the initial context as a new focus. (right) The Narrow strategy adds a second resource (black rectangle) from the initial context to the current focus.

Figure 3 The

 3 Figure 3 The Saffron page for the topic "Natural Language Processing". At the top, the local detail shows a trend graph. Below, the global context links to co-occurring topics, experts on the topic, and publications on the topic. These links support the Shift strategy. Plus buttons (+) next to the co-occurring topics and experts support the Narrow strategy: they combine the topics and experts with the current focus topic. The current view is reflected in the Breadcrumbs at the top.

Figure 4

 4 Figure 4 Two mechanisms for supporting the Shift and Narrow exploration strategies. Clicking on a topic shifts the focus while clicking on a plus button (+) narrows the focus.

Figure 5

 5 Figure5Questionnaire results on the Likert scale 1 being "Strongly Disagree" and 7 being "Strongly Agree". The "plus button" corresponds to the Narrow strategy whereas the "links to other resources" correspond to the Shift strategy.

  In this case I'd expect to see the papers of the selected author that are similar to the paper included in my selection." Some participants pointed that the interface should better explain how the current resource (focus) has been reached. They understood the function of breadcrumbs but expressed that it was not enough. Participants provided examples:

"When I combine three topics and then move to an expert page I end up with one breadcrumb [the one representing the expert] and if I remove that breadcrumb I come back to the starting page instead of going back to my three combined topics."

another participant said: "I'd like to see on a topic page that I arrived there from a search result page [..

http://saffron.deri.ie

Saffron uses a combination of topic frequency, inverted document frequency, trend, and topic length to determine which topics are the "hot topics" in the collection.

Pulled from DBPedia if available.Figure 2 Saffron's starting page. The user can select one of the hot topics or perform a keyword search.

ACKNOWLEDGEMENTS

This work was supported by Science Foundation Ireland under Grant No. SFI/09/CE/I1380 (Líon2).