Gauthier Sornet
email: gauthier.sornet@univ-orleans.fr

Fabrice Dupros
email: f.dupros@brgm.fr

Sylvain Jubertie
email: sylvain.jubertie@univ-orleans.fr

A Multi-level Optimization Strategy to Improve the Performance of Stencil Computation

Keywords: Stencil computation, Vectorization, Performance model Stencil computation, Vectorization, Performance model Stencil computation, Vectorization, Performance model

Stencil computation represents an important numerical kernel in scientific computing. Leveraging multi-core or many-core parallelism to optimize such operations represents a major challenge due to both the bandwidth demand and the low arithmetic intensity. The situation is worsened by the complexity of current architectures and the potential impact of various mechanisms (cache memory, vectorization, compilation). In this paper, we describe a multi-level optimization strategy that combines manual vectorization, space tiling and stencil composition. A major effort of this study is to compare our results with the Pochoir framework. We evaluate our methodology with a set of three different compilers (Intel, Clang and GCC) on two recent generations of Intel multi-core platforms. Our results show a good match with the theoretical performance models (i.e. roofline models). We also outperform Pochoir performance by a factor of x2.5 in the best case.

Introduction

Stencil computation is widely used in numerical simulations. For instance, many physical models based on PDE (Partial Differential Equation) heavily rely on it. This is coming from the grid-based computation induced by finite-difference or finite-volume methods that are routinely used to solve various equations (elastodynamics, heat equation, shallow water equation, etc). This numerical kernel can represent a major part of the total elapsed time, particularly when an explicit method is implemented. In this case, the numerical kernel should be evaluated hundreds or thousands of times making critical any performance improvement.

Regarding the computation workflow, each numerical stencil could be described as a set of weighted neighbor cells that should be loaded and combined. The stencil morphology (i.e. the number of cells and the directions involved in the computation) has an impact on the overall performance of the kernel. For instance, the ratio between the number of memory accesses 1 This space is reserved for the Procedia header, do not use it

Introduction

Stencil computation is widely used in numerical simulations. For instance, many physical models based on PDE (Partial Differential Equation) heavily rely on it. This is coming from the grid-based computation induced by finite-difference or finite-volume methods that are routinely used to solve various equations (elastodynamics, heat equation, shallow water equation, etc). This numerical kernel can represent a major part of the total elapsed time, particularly when an explicit method is implemented. In this case, the numerical kernel should be evaluated hundreds or thousands of times making critical any performance improvement.

Regarding the computation workflow, each numerical stencil could be described as a set of weighted neighbor cells that should be loaded and combined. The stencil morphology (i.e. the number of cells and the directions involved in the computation) has an impact on the overall performance of the kernel. For instance, the ratio between the number of memory accesses 1 This space is reserved for the Procedia header, do not use it

Introduction

Stencil computation is widely used in numerical simulations. For instance, many physical models based on PDE (Partial Differential Equation) heavily rely on it. This is coming from the grid-based computation induced by finite-difference or finite-volume methods that are routinely used to solve various equations (elastodynamics, heat equation, shallow water equation, etc). This numerical kernel can represent a major part of the total elapsed time, particularly when an explicit method is implemented. In this case, the numerical kernel should be evaluated hundreds or thousands of times making critical any performance improvement.

Regarding the computation workflow, each numerical stencil could be described as a set of weighted neighbor cells that should be loaded and combined. The stencil morphology (i.e. the number of cells and the directions involved in the computation) has an impact on the overall performance of the kernel. For instance, the ratio between the number of memory accesses 1 and the number of operations on each cell should be minimized to improve the computational efficiency. This memory-bound situation is a concern on multi-core architectures regarding the limited improvement at the bandwidth level. The situation is worsened on many-core platforms and recent work underline significant efforts to extract the best level of performance of such platforms (e.g. Kalray MPPA-256 chips for seismic wave propagation stencil in [START_REF] Castro | Seismic wave propagation simulations on low-power and performancecentric manycores[END_REF]).

In this paper we consider the classical 7-point and 27-point stencils that exhibit different characteristics (arithmetic intensity, memory bandwidth demand). These stencils have already been widely studied in the scientific literature providing some good references [START_REF] Datta | Optimization and performance modeling of stencil computations on modern microprocessors[END_REF]. The main contributions of this paper are the following. First, we underline the effectiveness of each level of optimization (vectorization, tiling, stencil composition) based on a set of representative compilers (Clang, Intel and GCC). Moreover, we show the portability of our methodology on two recent dual-socket Intel platforms (Intel Ivy Bridge and Broadwell). Finally, our contributions are analyzed with respect to the expected peak performance of each numerical kernel learned from the roofline prediction model [START_REF] Williams | Roofline: An insightful visual performance model for multicore architectures[END_REF]. Stencils studied in this paper have also been implemented with the Pochoir reference compiler [START_REF] Tang | The pochoir stencil compiler[END_REF] in order to discuss our results.

The paper proceeds as follows. Section 2 describes the related work. Section 3 discusses the fundamentals of stencil computation. We motivate the choice of each kernel by underlying their theoretical behavior. Section 4 underlines the main challenges for optimal performance on multi-core architectures and presents our contributions. Section 5 discusses our experimental results and Section 6 concludes this paper.

Related work

Stencil-based computation is the building block of many numerical models. One of the critical aspects is to minimize the cost of the memory movements required to retrieve the data from the main memory to the fast local memories. There have been significant efforts on new techniques designed to improve the performance of these memory-bound kernels on different types of architectures [START_REF] Krotkiewski | Efficient 3d stencil computations using {CUDA}[END_REF][START_REF] Hikmet Dursun | A Multilevel Parallelization Framework for High-Order Stencil Computations[END_REF][START_REF] Schafer | High performance stencil code algorithms for gpgpus[END_REF]. Obviously, these efforts mainly tackle the improvement of data-reuse. Among them, space blocking techniques provide a first level of optimization but these approaches suffer from several limitations. First of all, the expected efficiency is bounded because of the very low reuse opportunity inherent to stencils. In [START_REF] Rivera | Tiling optimizations for 3D scientific computations[END_REF], the limited speedup obtained from such strategies is described in a three-dimensional Jacobi loop (17%). Additionally, the impact of low-level mechanisms such as prefetching, vectorization or cache bypass are underestimated as described in [START_REF] Datta | Optimization and performance modeling of stencil computations on modern microprocessors[END_REF][START_REF] Werner Augustin | Optimized Stencil Computation Using In-Place Calculation on Modern Multicore Systems[END_REF].

To overcome space tiling limitations, efforts have been focused on tiling the computational domain both in space and time directions. In this case, the algorithms exploit the iterative nature of the kernel. Time-skewing or cache-oblivious algorithms [START_REF] Frigo | Cache oblivious stencil computations[END_REF] rely on a similar idea to perform several time steps inside each subdomain. The main difference comes from the explicit blocking criterion in the first case whereas the oblivious approaches exploit a recursive cut in space in order to fit into cache memory. The search for the optimal spacetime tile shape is also a critical point, the so-called diamond tile shape exhibits the best performance [START_REF] Tareq | Multicore-optimized wavefront diamond blocking for optimizing stencil updates[END_REF][START_REF] Muranushi | Optimal temporal blocking for stencil computation[END_REF]. Regarding the growing importance of vectorization capabilities, the space-time decomposition could exhibit a high level of complexity for the compilers. Moreover, implementations at the application level is difficult as the organization of the code must be deeply revisited.

Much work has also been done on auto-tuning techniques. Several frameworks have been introduced to optimize stencil computations on modern architectures (e.g. PATUS [START_REF] Christen | Automatic code generation and tuning for stencil kernels on modern shared memory architectures[END_REF] or PLUTO [START_REF] Muthu Manikandan Baskaran | Compiler-assisted dynamic scheduling for effective parallelization of loop nests on multicore processors[END_REF]). Amongst them, the Pochoir stencil compiler [START_REF] Tang | The pochoir stencil compiler[END_REF] aroused much interest. It implements stencil computations in a domain-specific language (DSL) and exploits a hyperspace

Standard Stencils

7-point and 27-point Stencil Description

We consider two classical stencils commonly studied in the scientific literature [START_REF] Rivera | Tiling optimizations for 3D scientific computations[END_REF][START_REF] Datta | Auto-tuning the 27-point stencil for multicore[END_REF]. The first stencil is a 7-point 3D stencil which includes a central cell with its six nearest neighbors as described in figure 1. The second stencil is a 27-point 3D stencil composed of a total of 27 cells: 6 direct plus 20 indirect neighbors included in the 3 × 3 × 3 cube around the central cell as described in figure 1. Both stencils are of great interest because they are representative of compute-bound and memory-bound situations. Indeed, we define the Reuse Intensity (RI) as a factor of reused floats per loaded bytes per stencil iteration. It is determined by the number of cells of the stencil divide by the number of bytes of a floating point value. The 7-point stencil performs the accumulation of 7 cells and each cell contains a 32 bits (4 bytes) floating point value. In this case, the reuse intensity factor is 7/4 = 1.75 for 7-point stencil that is likely to be memory-bound. It means that each float per byte is reused 1.75 times. For the 27-point stencil, the RI is 6.75, indicating that this stencil is less memory-bound.

Classic Stencil Implementation

Stencil algorithms consist in traversing a grid and performing computation at each grid point. In the case of a three-dimensional Cartesian grid, the computation consists in using the neighbor points in the upward-downward, left-right and forward-backward directions to evaluate the value of the current grid point. The algorithm then iterates to the next point applying the same computation until the entire grid has been traversed. Considering the 7-point Jacobi stencil, one needs to deal with a domain of n 3 single precision values and adds neighbor weights to compute the result for the next iteration. On multi-core platforms, a classical way to extract parallelism is to exploit the nested loops coming from the spatial dimensions of the problem. One of the main advantages is a straightforward use of OpenMP directives that are applied to the external loop. A first level of optimization of this implementation is to use the collapse OpenMP option in order to extract the parallelism from the three spatial loops.

4 Optimizations

This section details the different techniques we propose to optimize stencil computation.

Manual Vectorization

Available in all modern processors, SIMD units are able to apply a single instruction on a vector of values.In our study, we consider using AVX units supported by our Intel platforms which are able to process vectors of eight floating point single precision values. Compilers may automatically vectorize some codes, but there is no guarantee that the process works nor that it provides the best vectorized code in terms of performance. Thus, we propose a hand-written vectorized version of the stencil computation to ensure that our code is correctly vectorized. We compared it with actual auto-vectorized codes produced by compilers. This version is written using compiler intrinsics and consists in computing eight contiguous cells at a time.

Tiling

Tiling consists in virtually decomposing the domain into tiles and adapting the traversal accordingly. In this case, we traverse the domain blockwise with a given tile size (tx; ty; tz). When the whole domain does not fit into the cache, the standard stencil implementation makes the processor reload data from DRAM several times by iteration. If we traverse this domain by tiles fitting into the cache, we can expect to get closer to the theoretical RI. Therefore, the challenge is to determine the size of the tile that minimizes the amount of cache misses and provides the best performance. Optimized tile sizes have been experimentally found. Their larger dimensions are in the unit-stride direction which follows the literature [START_REF] Datta | Optimization and performance modeling of stencil computations on modern microprocessors[END_REF][START_REF] Muranushi | Optimal temporal blocking for stencil computation[END_REF].

Stencil Composition

The standard stencil description underlines the impact of the Reuse Intensity (RI). For instance, the 7-point stencil exhibits poor theoretical performance. In this part, we introduce a strategy to build an equivalent of the 7-point stencil but with a higher density. This approach could be described as a similar strategy of spacetime stencil decomposition. Figure 2 provides an overview of the composition of two 7-point stencils. Regarding the reuse intensity, this new stencil is close to the 27-point stencil and is likely to improve the performance of the original 7-point stencil. Each evaluation of our composed stencil corresponds to two iterations of the original 7-point kernel, thus we perform twice more floating point operations. However, we expect to reach a higher level of performance. We remind that the RI of the 7-point stencil is 1.75 whereas the RI of the reformulated stencil with 25-point is 6.25.

The stencil composition is defined as follows. Let us consider two stencils S A and S B that must be applied consecutively, then it is possible to build a stencil S C that is the composition of S A and S B . If we consider that S A and S B are of dimension N where the size of each dimension are respectively S A .r i * 2 + 1 and S B .r i * 2 + 1 (1 ≤ i ≤ N ) then the composed stencil S C is also in size N where the size of each dimension is (S A .r i + S B .r i ) * 2 + 1). We can determine that the extra-compute ratio between the successive application of S A and S B in comparison with the application of S C is given by the following approximation :

( N i=1 2(S A .ri + S B .ri) + 1) -1 ( N i=1 2S A .ri + 1 + N i=1 2S B .ri + 1) -2 (1)
If we apply Equation 1 on the 27-point stencil, the half dimension is 1 and the extra-computation ratio is 2.29. In our case, for the 7-point stencil extra-computing ratio is 2.0 (2).

(25pts -1)Additions 2time steps * (7pts -1)Additions

(2)

5 Performance Analysis

Experimental Setup

We used two dual-socket platforms for our experiments:

• the first one with Intel Xeon E5-2697 v2 Ivy Bridge processors, for a total of twenty-four cores at 2.7Ghz;

• the second one with Intel Xeon E5-2697 v4 Broadwell processors, for a total of eighteen cores at 2.3Ghz.

Our codes are compiled with three different compilers: Clang 3.8, GCC 6.2 and ICC 17, with -O3 -march=native optimization flags, the second one implicitly enables AVX and FMA support on capable architectures. Multi-threading is performed using OpenMP. All results represent the maximum of ten runs to guarantee relevant values. The computation domain is composed of 512 3 single precision floating point values. Therefore, the memory footprint is one order of magnitude larger than the size of the last level of cache memory. For each run, 100 iterations are performed. The following versions are implemented for both the 7-point and the 27-point stencils, except for the composition which only concerns the first stencil: 1. No vectorization: vectorization is disabled by the -fno-tree-vectorize compilation flag.

2. Automatic vectorization: compilers may automatically vectorize the code.

3. Manual vectorization: manual vectorization is performed as described in Section 4.

Manual vectorization with tiling: this version adds tiling to the previous version.

5. 7-point stencil composition: this version applies the 25-point stencil resulting from the self composition of the 7-point stencil and consequently performs only 50 iterations.

6. Pochoir: this version requires the Intel compiler since it is based on the Cilk threading library instead of OpenMP.

Performance Characterization

Roofline Models

Figure 3 shows the roofline models obtained on Broadwell and Ivy Bridge platforms. It is a good visual tool to discuss both the performance of our baseline implementations but also to compare the impact of the reuse intensity (RI) of each stencil. First, we can observe that the achievable performance is strongly related to the RI. In the first case, the 7-point kernel is higtly memory-bound and the achievable peak performance is limited to an average upperbound of 100 GFLOPS on both platforms. For the 27-point stencil, the RI is higher (6.25 vs 1.75 for the 7-point) leveraging from the higher density of this kernel. Therefore, the upperbound achievable performance is 413 GFLOPS on Broadwell and 368 GFLOPS on Ivy Bridge. Nevertheless, these stencils are still memory-bound as their RI are below the compute-bound limit measured on each platform (8.4 on Broadwell and 7.2 Ivy Bridge platforms). Secondly, if we combine manual vectorization and space tiling optimization we almost reach 30% of the achievable peak performance for the 7-point and 27-point stencils on both platforms. In fact, the 7-point and 27-point stencils perform respectively 41 GFLOPS over 116 GFLOPS and 158 GFLOPS over 447 GFLOPS on the Broadwell platform. Moreover, the Ivy Bridge platform achieves 35 GFLOPS over 104 GFLOPS and 128 GFLOPS over 400 GFLOPS with respectively the 7-point and 27-point stencils. These results are rather good as we do not take into account advanced ceiling for our roofline models. For instance, one may consider that with only additions, processors may no use all its operator ports. Whereas a theoretical peak performance can be determined from all the available ports. As a result, our kernels have a significantly reduced achievable peak performance.

Comparison with the theoretical Results

The reuse intensity is a tool to check the level of performance of our implementations. The theoretical ratio between the RI of the 27-point and the 7-point stencil is 6.75/1.75 ≈ 3.85. We use this ratio to discuss the performance observed on the different architectures. Regarding the Clang compiler auto vectorization version, the ratio measured is 1.97 on Broadwell and 2.04 on Ivy Bridge. These values are far from the theoretical value. Additional investigations demonstrate that Clang compiler is unable to vectorize the standard OpenMP implementations.

Combination of the complexity of the nested loops and side effects from the use of OpenMP directives could explain this situation [START_REF] Klemm | Extending openmp* with vector constructs for modern multicore SIMD architectures[END_REF]. GCC and ICC compilers vectorize the code leading to better results. Another issue is the DRAM memory access delay. Any loaded data from DRAM into caches have to be reused as mush as possible in order to avoid some DRAM access delays. As we supposed a 3D grid bigger than caches, each grid cell is loaded at least once by iteration. At the best, they are loaded only once by iteration. To get close to it, it needs the spacial cache reuse to be optimized. This is the aim of the tile optimization. Table 1 shows our tiled vectorized peak performance ratios 27-point/7-point. The performance ratios of the manually vectorized and tiled versions (version 4) are closer to 3.85 than the ones for the automatic vectorized versions (version 2). Thus, we observe a good match between the theoretical RI ratio and these experiments. This version shows a reliable ratio of 3.13.

Machine

Multi-level Optimization

We evaluate the improvement coming from our optimizations including a new algorithmic formulation introduced for the 7-point stencil. Figure 4 shoes the results we obtained.

Impact of Vectorization

The comparison between manual and automatic vectorization underlines the difficulty for the compiler to fully exploit this level of parallelism. As described in the previous section, the Clang compiler is struggling to be efficient on the 27-point stencil that exhibits complex multithreaded nested loops. Indeed, the compiler uses a lot of uload/ustore and does not detect that the buffers are aligned. Moreover, in our case, the OpenMP transformations may inhibit the Clang automatic vectorization. For these reasons, the scalar version and the automatic vectorization version of Clang have the same performance. This is the reason why only the scalar version is shown on figures 4. Our manual vectorization strategy provides a significant improvement in comparison with the results obtained with the three compilers. We observe an average gain of 1.8× for Intel and GCC compilers and a maximum gain of a factor 4.2 for Clang compiler. As a matter of fact, the 7-point stencil can hardly benefit from these improvements as this kernel quickly saturates the memory bandwidth available.

Impact of Tiling

Tiling is a strategy to enhance data-reuse.We learn from figures 4 that the impact of this optimization is rather strongly dependent on the stencil characteristics.For instance, we observe limited gains for the 7-point stencil (less than 10% on Broadwell and about 0.3% on Ivy Bridge) whereas the 27-point stencil is able to benefit from this strategy (more than 25% on Broadwell and more than 12% on Ivy Bridge). This is mainly coming from the shape of the stencil and the ratio between the computation and the memory movement that offers more opportunities to reuse data in the 27-point case. The optimal tile size selection is another issue. We solve it for each stencil and architecture thanks to an experimental approach guided by the literature [START_REF] Datta | Optimization and performance modeling of stencil computations on modern microprocessors[END_REF][START_REF] Muranushi | Optimal temporal blocking for stencil computation[END_REF].

Impact of Stencil Composition

The last stage of our optimization strategy corresponds to a new formulation of the numerical kernel. We evaluate this strategy for the 7-point example as this kernel is more likely to leverage this improvement.

In terms of GFLOPS, we observe good performances with Clang, Intel and GCC compilers as the measured speedup on Broadwell is respectively 3.34×, 3.33× and 3.42×. On the Ivy Bridge, we also have respectively 2.8×, 3.74× and 3.26× speedups. The reuse intensity of the 25-point stencil (new formulation of the 7-point stencil) allows us to compute with a higher efficiency whatever the compiler or the platform. Additionally, we maintain the coherency with the theoretical analysis. Figure 4 shows the peak performance measured with the 25-point and the 27-point stencil. The measurements are very similar as both stencils exhibit very close data reuse capabilities. Elapsed times are reported in table 2. As explained in the subsection 4.3, the 25-point stencil is the result of the 7-point stencil self-composition but it requires 4 times more additions by iteration (from 6 to 24 additions). However, one iteration of the 25-point stencil is equivalent to 2 iterations of the 7-point stencil. Thus, to obtain the same numerical results, we only need half the number of iterations. As a result, we perform twice more additions for the whole computation. Then, in terms of elapsed time, the measured speedup on Broadwell is respectively divided by 2 and are equal to 1.67×, 1.67× and 1.7×. On the Ivy Bridge, we also have respectively 1.4×, 1.87× and 1.63× speedups. We are still investigating these odd Pochoir performances.

Machine

Conclusion and Perspectives

As shown in our experiments, optimizing stencil kernels is not straightforward since efficient optimization techniques depend on the stencil morphology, on the underlying architecture and on compilers. Taken independently, these optimization techniques are simple to implement, but they need to be combined in order to reach near-peak performance, and some parameters, like the tile size, may need to be tuned for each architecture. The Pochoir library relies on a complex space and time decomposition which outperforms our implementation (tiling and manual vectorization) only for the 7-point stencil on the Ivy Bridge architecture. However, our stencil composition is able to reach a similar level of performance on this architecture. In all other cases, our best implementation is faster than Pochoir but the speedup is highly variable depending on the architecture, on the stencil and on the compilers. Of course, faster implementations may be obtained since we do not reach the upper bound of the Roofline, but reducing the remaining gap may require to increase the code complexity. Despite encouraging results, the implementation of our contributions in some real applications may not seem straightforward nor reasonable. Indeed, these optimizations are tied to the underlying architecture which may prevent code portability. To solve this problem, we plan to use the same approach as Pochoir: splitting the interface from the optimized implementation. A Domain Specific Language (DSL) would be provided to the developer who focuses on the description of its stencil computation. Optimized implementation would be obtained using either a specific compiler, like Pochoir, or meta-programming techniques.

--------------------------

Figure 1 :

 1 Figure 1: Visual representation of the 7-point and 27-point stencils.

Figure 2 :

 2 Figure 2: The 25-point stencil corresponding to the composition of two 7-point kernels.

Figure 3 :

 3 Figure 3: Experimental Stream and Linpack roofline models on Broadwell and Ivy Bridge platforms.

Figure 4 :

 4 Figure 4: Impact of the multi-level optimization strategy on the 7-point and the 27-point stencils with the Clang, Intel and GCC compilers on a dual-socket Intel Xeon Broadwell (E5-2697v4) and a dual-socket Intel Xeon Ivy Bridge platform.

Table 1 :

 1 Ratios between 7-point and 27-point kernels of automatically vectorized version 2 and our best optimized version 4. The theoretical value ration is about 3.85.

		Clang compiler	Intel Compiler	GCC Compiler
		2-Auto 4-ManuTiled 2-Auto 4-ManuTiled 2-Auto 4-ManuTiled
	Broadwell	1.97	3.45	4.28	3.88	4.18	3.74
	Ivy Bridge	2.04	2.59	4.24	4.14	4.20	3.72

Table 2 :

 2 Elapsed time (in ms) of the 7-point (RI=1.75) and the 25-point (RI=6.25) stencil.

		Stencil	steps Clang ICC GCC Pochoir
	Ivy Bridge 7-point	100	2157 2288 2174	1316
		25-point	50	1520 1306 1339	2534
		27-point	100	3613 2393 2532	5289
	Broadwell	7-point	100	1875 1956 2117	2810
		25-point	50	938 1160 1276	2301
		27-point	100	2358 2183 2249	2893

Acknowledgments

The authors thank Philippe Thierry, senior principal engineer at Intel, for many interesting discussions and for providing us access to Broadwell and Ivy Bridge platforms. The work of G. Sornet is co-funded by the Région Centre-Val de Loire. 9


G Blount

Ph.D ; M Gorensek

Ph.D L Hamm

; K O'neil

Ph.D ; C Kervévan

CO2-Dissolved and Aqueous Gas Separation

Keywords: Carbon Capture, Enhanced Oil Recovery, EOR, Greenhouse Gas, GHG, Geothermal Energy, Dissolved CO2, Post-Combustion

Partnering in Innovation, Inc. (Pi-Innovation) offers a novel aqueous gas separation and carbon capture approach. Specific features in the design uniquely integrate multiple energy recovery processes and the potential for contaminant removal (i.e., oxides of sulphur (SOx), nitrogen (NOx), selenium (SeOx), and mercury (HgOx). CO2-DISSOLVED is an ANR multi-national project led by the BRGM, with Phase II funded as one of the first Geodenergies projects. Phase I explored the techno-economic feasibility of geothermal energy production combined with CO2 injection using a simplified version of the Pi-Innovation technology. Here we review the Pi-CO2capture process and the integration with dissolved injection applications.

Introduction

CO2 Dissolved and Partnering in Innovation Inc. (Pi-Innovation) are collaborating to integrate aqueous CO2 storage with aqueous gas separation. Aqueous capture and storage of CO2 storage has applications including coinjection with reservoir flooding, with reproduced water in oil production, and in geothermal circulation systems. Pi-Innovation offers a patented aqueous gas separation and carbon capture technology that is highly promising in terms of potential total system cost reduction and energy recovery.

The CO2-Dissolved project evaluated the storage potential and techno-economic feasibility of co-injecting dissolved CO2 in a geothermal doublet in the Paris Basin.

CO2-DISSOLVED

CO2-DISSOLVED is a multi-national project led by the Bureau de Recherches Géologiques et Minières (BRGM) and funded by the French National Research Agency (ANR) with Phase II funded as one of the first Geodenergies projects. Geodenergies is a French industry-driven initiative grouping 18 companies and research organizations aiming at: (1) structuring a community of expertise to promote subsurface energy technologies that are key to a global energy transition; (2) cross-fertilizing to develop 3 emerging industrial sectors: geothermal energy production,CO2 storage, and energy storage.

As the name "CO2-DISSOLVED" implies, the project focuses on the feasibility of the injection of dissolved CO2 in water, as opposed to supercritical CO2 injection. In this, and drawing from a process originally developed at a US Department of Energy (DOE) laboratory, Partnering in Innovation, Inc. (Pi-Innovation) offers a novel aqueous gas separation and carbon capture approach.

The CO2-Dissolved concept explores the storage of CO2 dissolved in water rather than in supercritical form; Kervévan et al. [START_REF] Kervévan | CO2-DISSOLVED: a novel concept coupling geological storage of dissolved CO2 and geothermal heat recovery -Part 1: Assessment of the integration of an innovative low-cost, water-based CO2 capture technology[END_REF] and Kervévan et al. [START_REF] Kervévan | CO2-DISSOLVED: A Novel Approach to Combining CCS and Geothermal Heat Recovery[END_REF]. Figure 1 illustrates the carbonation of spent geothermal water in an injection well associated with a geothermal doublet Kervévan [START_REF] Kervévan | Successfully combining CO2 storage and geothermal heat extraction[END_REF]. The dissolved form of CO2 storage in a saline aquifer is neutral to negatively buoyant, and shifts the geologic storage mechanism to the solubility trapping realm. Supercritical trapping relies on structural and stratigraphic trapping, and residual trapping, which is positively buoyant. Positive buoyancy represents a greater risk of uncontrolled release of CO2 to the surface. Figure 2 is a modification of the typical IPCC [START_REF]IPCC Special Report on Carbon Dioxide Capture and Storage. Prepared by Working Group III of the Intergovernmental Panel on Climate Change[END_REF] geologic storage effectiveness diagram illustrating the range of CO2 trapping mechanisms versus storage security. Moving directly to solubility trapping offers lower brine displacement in the vicinity of the injection and significantly reduces the time required to achieve the highest level of geologic storage security.

The storage potential for CO2 in geothermal brine is limited by the solubility of the CO2 in the brine and the purity of the CO2 that is to be dissolved in the brine; due to gas competition in the dissolution process. This project targets low to medium range CO2 emitters (~10 to 150 kilotons/year) that could be compatible with a single doublet geothermal installation adjacent to the facility.

The coupling of geothermal heat with CO2 storage adds the potential for energy use and revenue generation, and adds the opportunity for valorization of the injection operations. In France, 650 potentially compatible industrial sites have been identified. These sites represent 25% of the stationary emissions for the country Kervévan [START_REF] Kervévan | Successfully combining CO2 storage and geothermal heat extraction[END_REF].

Pi-CO2

Partnering in Innovation Inc. offers post combustion CO2 capture using water as the physical mass transfer solvent. The patented Pi-CO2 process can be deployed on land or offshore, and has the capacity to remove SOx, NOx, and vaporized metals from the flue gas stream along with the CO2. The relatively complete treatment of flue gases adds the potential for substantial cost savings, providing capture incentives to a wide variety of flue gas sources typically containing moderately high CO2 contents (i.e., coal, cement, burnt lime, refining, smelting, biomass combustion, incineration, etc.).

The CO2 capture component of the process takes advantage of the solubility difference between CO2 and N2 in water at higher pressures and lower temperatures. Figure 3a illustrates the saturation concentrations of CO2 and N2 in water at 30 C, versus pressure. Figure 3b illustrates the saturation concentration of CO2 at various temperatures and pressures. Capture of CO2 occurs as water is circulated through a closed loop absorber and pressure swing desorber; the water in the closed loop is used "over and over again". High hydrostatic pressure magnifies the CO2 carrying capacity of the water. Lower temperature locations also further increased the CO2 carrying capacity. High hydrostatic pressure is achieved by placing the absorber/desorber in a water filled large diameter shaft, or through submergence in an offshore setting. Lower water temperatures can be acquired through submergence in relatively cold ocean or lake settings or by heat exchange with a natural cold water source.

The configuration of the absorber/desorber system is down-flow for the absorber and up-flow for the desorber. The absorber contains many stages that support sequential loading of CO2 into the water. Figure 4 shows the transfer of CO2 from the gas phase to the liquid phase in a down-flow absorber having 16 stages of mass transfer, operating at 30 C, with approximately 14% CO2 in the flue gas (similar to coal). The absorber will be more efficient with higher CO2 concentration emissions (i.e., cement, smelting and refining cat-crack), and less efficient at lower CO2 concentrations (i.e.,, natural gas fired turbines, natural gas fired power plants).

Figure 4 Step-wise CO2 transfer from the gas-phase to water-phase with a multi-stage down-flow cascading absorber.

The absorber efficiency will increase with: higher hydrostatic pressure, lower temperature, optimum bubble fraction, with longer absorber stages, and with an increase in the number of stages. The presence of free and dissolved N2 in the water in the absorber acts as a poison with respect to clathrate formation at the lowest conceivable temperature and highest conceivable pressure that the system is likely to be operated.

The desorber is configured to support sequential desorption promoting higher CO2 concentration in the final CO2 product and supporting gas lift pumping within the desorber. Gas lift pumping within the desorber drives the circulation within the absorber/desorber loop because the upward flow in the desorber achieves down-flow in the absorber.

Gas lift pumping is a well understood technology. However, most gas lift pumps are not typically on the scale of the Pi-CO2 process (diameter of the desorber riser and depth of submergence), and are not generally driven by effervescing gases (greater efficiency). Pump efficiency is controlled by the diameter of the rising pipe, bubble dispersion, gas volume, submergence ratio, and the height in which the pumped water is lifted; Awari et al. [START_REF] Awari | Performance analysis of air-lift pump design[END_REF], Kassab et al. [START_REF] Kassab | Air-lift pumps characteristics under two-phase flow conditions[END_REF], and Barrut et al. [START_REF] Barrut | Mass Transfer efficiency of a vacuum airlift-Application to water recycling in aquaculture systems[END_REF]. The best performance in volume of water pumped can be achieved with: the largest diameter riser pipe, a high submergence ratio (submergence/lift height), an optimum volume of gas to make bubbles, and good bubble dispersion.

Fan et al. [START_REF] Fan | Experimental study on the performance of an air-lift pump for artificial upwelling[END_REF] performed a study of gas lift pumping with a relatively large diameter rising pipe, and relatively low water lift. Pi-Innovation developed a simple model using the empirical data derived from the study. A comparison with the modeling work of Fan et al. [START_REF] Fan | Experimental study on the performance of an air-lift pump for artificial upwelling[END_REF] suggests that there is more than sufficient gas to support pumping in the Pi-CO2 process. The challenge may be to selectively remove gas to maintain the optimum gas volume for optimum pumping in the desorber.

An important part of the Pi-CO2 process is the energy recovery from the pressurized non-dissolved gas that exits the top of the absorber. The process uses turbo-machinery to recover a large fraction of the compression energy penalty by combining the heat of compression with the pressurized gas. The Pi-CO2 process has similar parasitic energy requirements when compared to competing technologies (~22%) in a coal fired setting, but can take advantage of system waste heat for electricity production that offsets CO2 to an equivalency of 16 to 18%. The process can be driven by direct natural gas firing with alternate fuels, or driven with electricity. Direct firing can offer greater energy efficiency. The turbo-line can be electrically linked or mechanically linked as in some Compressed Air Energy Storage turbo-lines. Figure 5 shows a generalized sketch of the Pi-CO2 system.

Uniquely, the Pi-CO2 process removes SOx, NOx, vaporized metals, and CO2, in a single integrated process, The oxides are removed in compression condensate and at inter-cooler and after-cooler steps during flue gas compression. Figure 6 shows a simple chemical model output based upon Oxyfuel combustion flue gas compression empirical data, sourced from White et al. [START_REF] White | Purification of oxyfuel-derived CO 2 for sequestration or EOR[END_REF] and Huckaby [START_REF] Huckaby | NETL Office of Research and Development -Oxyfuel Combustion Progress Update Thermofluids; Slide Presentation. Annual NETL CO2 Capture Technology for Existing Plants R&D Meeting[END_REF].

SOx and NOx removal is usually required prior to CO2 capture because the presence of these compounds accelerates the degradation of typical capture chemicals (amines, sorbents, etc.); Rezaei and Jones [START_REF] Rezaei | Stability of Supported Amine Adsorbents to SO2 and NOx in Postcombustion CO2 Capture. 1[END_REF], Supap et al. [START_REF] Supap | Kinetics of sulfur dioxide and oxygen induced degradation of aqueous monoethanolamine solution during CO2 absorption from power plant flue gas streams[END_REF], Rohr et al. [START_REF] Rohr | Potential toxicological effects of amines used for carbon capture and storage and their degradation products[END_REF], and El Moudir [START_REF] El | Experimental Studies for Development of a Purification Process for Single and Mixed Amine Solvents[END_REF]. Based upon CAPEX and OPEX costs from Cichanowitz [START_REF] Cichanowicz | Current Capital Cost and Cost Effectiveness of Power Plant Emissions Control Technologies[END_REF], SOx pretreatment, alone, prior to CO2 capture, can add as much as $25 to $30 (US Dollars) to the cost of a metric ton of CO2 removed. The addition of heat to the turbo-line can allow for excess power generation to offset capital cost. Turbomachinery is a very large part of the capital cost of the process. The potential to generate electricity is also useful because it can offset a portion of higher carbon sourced power on the grid in some locations which can improve the overall carbon reduction efficiency. Figure 7 illustrates this overall carbon reduction with electricity offset. Energy demand is typically considered an important measure of operating cost between competing CO2 capture technologies. The Pi-CO2 process efficiently uses the power required for operation by recovering a large fraction of the compression energy penalty, using the energy of compression of the dissolved gas fraction to drive absorber/desorber pumping, and provides an opportunity to generate electricity offsets with the turbo-line.

CO2 capture with Pi-CO2 uses no expensive chemicals, has no significant chemical degradation, and no chemical wastes. The overall footprint of the technology is less because the absorber/desorber system is deployed in the subsurface or subsea. The capital cost can be lower when pretreatment for SOx, and NOx are considered, and future capital costs can be lower when Hg0 removal is considered. Operational costs are also low due to the use of long life turbomachinery for compression and energy recovery rather than chemical intensive processes.

Integration of CO2-DISSOLVED and Pi-CO2

The integration of aqueous CO2 storage with aqueous gas separation can offer lower risk geologic storage scenarios in situations where water is available for injection into saline aquifers. It is possible to capture CO2 from various industrial sources and produce a high purity CO2 product for use as supercritical CO2, or use relatively pure CO2 dissolved in water with the Pi-CO2 absorber technology, and directly inject the water into the subsurface. It is also possible to capture the CO2 from flue gas with the Pi-CO2 process absorber and inject the absorber discharge into the subsurface.

Figure 1

 1 Figure1CO2-Dissolved: geothermal doublet injection well; Modified from BRGM, Kervévan[START_REF] Kervévan | Successfully combining CO2 storage and geothermal heat extraction[END_REF]

Figure 2 CO2

 2 Figure 2 CO2 Trapping Mechanisms and Relative Storage Security Over Time

Figure

  Figure 3a CO2 and N2 Saturation Concentration versus partial pressure at 30 C dissolved in pure water and 3b CO2 Solubility in pure water across a range of temperature and partial pressure conditions

Figure 5 Figure 6

 56 Figure 5 Generalized Pi-CO2 capture processes showing general energy recovery and flue gas contaminant removal

Figure 7

 7 Figure 7 Overall carbon reduction with electricity offset

Available online at www.sciencedirect.com © 2017 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). Peer-review under responsibility of the organizing committee of GHGT-13.