Pete Beckman
email: beckman@anl.gov

Emmanuel Jeannot
email: emmanuel.jeannot@inria.fr

Swann Perarnau

Ron Brightwell

Workshop on Resource Arbitration for Dynamic Runtimes (RADR)

The question of efficient dynamic allocation of compute-node resources, such as cores, by independent libraries or runtime systems can be an nightmare. Scientists writing application components have no way to efficiently specify and compose resource-hungry components. As application software stacks become deeper and the interaction of multiple runtime layers compete for resources from the operating system, it has become clear that intelligent cooperation is needed. Resources such as compute cores, in-package memory, and even electrical power must be orchestrated dynamically across application components, with the ability to query each other and respond appropriately. A more integrated solution would reduce intra-application resource competition and improve performance. Furthermore, application runtime systems could request and allocate specific hardware assets and adjust runtime tuning parameters up and down the software stack. The goal of this workshop is to gather and share the latest scholarly research from the community working on these issues, at all levels of the HPC software stack. This include thread allocation, resource arbitration and management, containers, and so on, from runtime-system designers to compilers. We will also use panel sessions and keynote talks to discuss these issues, share visions, and present solutions.

Scope

Over the last five years, the number of nodes in large supercomputers has remained largely unchanged. In fact, the Oak Ridge National Laboratory computer leading the Top500 list, Summit, has fewer nodes than its predecessor, which is 20 times slower. Machines are getting faster not by adding nodes, but by adding parallelism, cores, and hierarchical memory to each compute node. This shift in how computers are scaled up makes it imperative that parallel computer resources within a node be carefully orchestrated to achieve maximum performance. Dynamically allocating and managing threads and the mapping of these threads to cores is a challenge that requires cooperation and coordination between the different components of the software stack.

At the application level, a software component might use pthreads to express and coordinate concurrency. The application might also be linked to computational libraries, such as PETSc or Intel's MKL that could be multithreaded. Moreover, other parts of the application may use OpenMP parallel section (which are implemented with threads). Furthermore, the runtime system may need its own parallel resources, for example, to spawn progress engines for message libraries or remote invocation handlers. Currently, each component in this complex software stack is unaware of the other pieces. Therefore, threads can compete for cores and cause profound slowdowns to intranode collective operations and barriers. Moreover, currently, no mechanisms exist to query for unused cores, to reserve some of them, or to check which part of the application is using them. Resource allocation and partitioning by the operating system must be adaptive and well connected to user-level software components. There has been research progress in this field. Tools such as hwloc can provide information on systems, such as topology. However, hwloc is not designed to handle direct allocation and partitioning resources. Likewise, it does not provide the interfaces required for software components to negotiate how to improve performance of the application through cooperative sharing of resources. Other approaches have been proposed, such as application composition, dynamic topology management or topology-aware core selection. Resource partitioning, enforced by the operating system using containers or within multi-kernels are also being investigated. Each of those approaches brings its own set of benefits and challenges, that need to be discussed within the community, compared with each other, and evaluated against benchmarks and use cases yet to be identified. As a relatively new and specific research area, it is difficult for researchers to find a place where to submit papers and discuss solution with the whole community. Therefore, we think it is of great interest for the HPC community to provide a venue to present these work in all their specificity and foster new discussions.

Program Committee

Program Chairs:

• Pete Beckman Program Committee:

• Dorian Arnold, Emory University.

• Denis Barthou, Bordeaux INP.

• Siegfried Benkner, University of Vienna.

• George Bosilca, Univ Of Tennessee.

• James H Cownie, Intel.

• Carter Edwards, Nvidia.

• Hal Finkel, Argonne Ntl Lab.

• Karl Fuerlinger, LMU, München.

• Balazs Gerofi, U. Tokyo -Riken.

• Brice Goglin, Inria.

• Raymond Namyst, Univ. Of Bordeaux.

• Stephen Olivier, Sandia Ntl Lab.

• Tapasya Patki, Lawrence Livermore Ntl Lab.

• Marc Perache, CEA.

• Swann Perarnau, Argonne Ntl Lab.

• Rolf Riesen, Intel.

• Sameer Shende, U. of Oregon.

• Christian Terboven, RTW Aachen.

Program

This year, we have selected one paper to be presented at this workshop and invited three keynote speaker. The topics covers by these talks will show the importance and the variety of the RADR problematic.

We hope to see you at this first edition of RADR and tat this workshop will be a place for fruitful and lively interactions.

RADR 2019 Invited Speaker Dynamic Resource Management: An Application Perspective Anshu Dubey Mathematics and Computer Science Division, Argonne National Laboratory Flash Center for Computational Science, University of Chicago

Abstract

The state of practice among system software developers and applications developers is largely that of mutual exclusion. System software aims to make its services transparent to the applications codes, while applications tend to do everything on their own. In reality neither is optimal. Applications know where there are most fruitful opportunities for resource and data orchestration. When the applications do it themselves they may eliminate the possibility of using future developments in system software that may be beneficial. And when system software does is transparent to the applications, it leaves a lot of domain specific knowledge on the table that could have been exploited for better performance. In this presentation I will present a resource orchestration approach that we are developing for FLASH, a Multiphysics Multicomponent application code. This approach relies upon close cooperation between applications and tools developers, and aims to be as future-proof as possible given what we know of machine and software development trends.

Bio

Anshu Dubey is a computer scientist in the mathematics and computer science division at Argonne National Laboratory and a Senior Scientist at large at the University of Chicago. She leads the earth and space sciences sub-area of applications development in the US-DOE Exascale Computing Project. She is also the chief software architect for FLASH, a multiphysics multiscale HPC software that is used by multiple science and engineering domains as their community code. She is interested in all aspects of HPC scientific software including numerics, design and productivity issues.

Figure 1 :

 1 Figure 1: Software stack with different parts using threads

An Outlook to Node Resource Management in HPC: Challenges and Opportunities.

Balazs Gerofi Riken

Abstract Node architecture in high-performance computing (HPC) is becoming increasingly complex. Heterogeneous processing elements combined with deep memory hierarchies and complex NUMA topologies require careful mapping of application components to node resources so that efficient utilization of the hardware can be attained. Applications, on the other hand, are also increasingly complex and dynamic in behavior which further complicates efficient resource utilization. This talk will address multiple aspects of the node resource management issue and is comprised of three parts. Two ongoing efforts at RIKEN will be discussed as motivating case studies: mpipin, a portable, MPI implementation agnostic process and workflow placement tool, and the utility thread interface (UTI) that provides and API for applications/libraries to better describe thread behavior to the underlying operating system. Finally, a recent international effort that aims to combine multiple existing resource management pieces (including the two RIKEN ones) will be introduced.

Bio

Dr. Balazs Gerofi is a research scientist at the RIKEN Center for Computational Science, where he is involved with system software research and development for high performance computing. He actively participates in the design and development of the Post K supercomputer, Japan's next-generation flagship supercomputer after the K Computer. Balazs earned his M.Sc. degree and Ph.D. degree in computer science from the Vrije Universiteit Amsterdam and The University of Tokyo, respectively. His research interest covers operating systems, high performance computing, cloud computing, and fault-tolerant computing. Balazs is a member of the IEEE Computer Society and the Association for Computing Machinery (ACM).


Abdanour Irbah

Luc Damé

Redouane Mecheri

Djelloul Djafer

Solar oblateness and its variations in phase with the 22 yr magnetic cycle

Solar oblateness and its variations in phase with the 22 yr magnetic cycle

Abdanour Irbah 1 , Luc Damé 1 , Redouane Mecheri 2 and Djelloul Djafer 3 (1) LATMOS/IPSL, UVSQ, Université Paris-Saclay, Sorbonne Université, CNRS, 11 BD D'Alembert Guyancourt 78280 France (2) Centre de Recherche Astronomie Astrophysique et Géophysique, CRAAG 16340, Bouzaréah, Algeria (3) Unité de Recherche Appliquée en Energies Renouvelables, URAER, Centre de Développement des Energies Renouvelables, CDER, 47133, Ghardaïa, Algeria

The internal dynamics of the Sun is manifested by disturbances on the photosphere surface, inducing directly small deviations from its sphericity. The solar shape then reflects the internal state of the Sun and processes taking place herein. Its temporal variations are useful for modelling the Sun, particularly in relation to its activity cycle. The solar oblateness i.e. Pole-Equator radius difference, is therefore a key input for models.

Oblateness measurements since the beginning of the 20th century are still very controversial. Many measurements were carried out, raising questions about its average value and its temporal variations. Gravitational models and helioseismology, giving access to radial profiles of latitudinal differential rotation and the internal magnetic field, have identified acceptable values of oblateness, mostly induced by the centrifugal force on superficial layers with a weak contribution of the gravitational quadrupole moment J 2 . Modern oblateness measurements confirmed expected values. Time series recorded from the ground, balloons, and lately Space showed variations but they were inconclusive. They were in phase or in antiphase with solar activity while others reported no obvious variations [1]. Oblateness measurements obtained from HMI are therefore a major asset for explaining those of the past and the inconsistencies reported since they now cover almost an entire cycle. Solar oblateness is computed from images recorded during roll procedures, i.e. the calibration mode of SDO where the spacecraft is rotated around HMI line-of-sight. The roll procedure is run twice a year with solar images recorded in the continuum near the Fe I absorption line (617.3 nm) in a narrow band (70 mA). The processing method is explained with Figure 1. It allows estimating the solar shape that is fitted with Legendre polynomials up to the fourth order. The dimensionless distortion coefficients C 2 (quadrupole) and C 4 (hexadecapole) are thus estimated, which makes it possible to deduce solar oblateness.

Roll sequences, ran between October 2010 and July 2018, resulted in an average solar oblateness of 8.8 ± 0.8 mas (6.4 ± 0.6 km), in good agreement with measurements of the last two decades and consistent with helioseismology-based models. Time variations were observed and appear in anti-phase with Cycle 24 sunspot number taken as proxy of solar activity (Fig. 2). C 2 variations are in phase with activity while C 4 presents anti-symmetric variations with respect to the maximum solar activity time. The oblateness measurements made in Cycle 24 with HMI and other Space missions are plotted against sunspot number (Fig. 3). A linear relationship between these solar parameters is observed (R 2 =77%). The negative slope confirms that oblateness variations are in anti-phase with activity. A similar analysis was performed on oblateness measurements made in Cycle 23 with the Heliometer (Pic-du-midi, France) and from Space (MDI and RHESSI). The linear regression vs. sunspot number (R 2 =79%) shows a positive slope indicating that oblateness variations are in phase with activity of Cycle 23 (Fig. 3). Finally, concerning the previous cycle, anti-phase variations with activity were reported from measurements made on balloons in the descent of Cycle 22 [2].

In summary, the Sun initiates a physical process that results in an oblateness pulsation with a period of twice the 11 yr solar cycle. The Sun has maximum swelling during odd cycles and vice versa for even ones, i.e. the solar shape oscillates like the magnetic field having extreme values during its polarity inversion. The measurement time relative to the oscillation period of solar oblateness explains the controversy reported in the literature for previous measurements.

The magnetic field was first mentioned in 1967 to model oblateness [3]. Why it oscillates with the solar magnetic field is a new challenge.

Fig. 1

 1 Fig. 1 Processing method illustrated with the roll of July 22, 2015. (a) Azimuthal variations of solar radius obtained from images (332); each line is a solar shape. Solar active regions (rectangle) affect shapes. (b) To compensate for the spacecraft rotation, they are shifted to have their western equator at a zero azimuth angle: active regions are spread out along columns and CCD defaults on oblique directions. (c) A time drift of solar radius is observed (black) and corrected (red). (d) The mean shape is computed but affected by active regions. (e) They are removed and the solar shape filtered then fitted with Legendre polynomials (red).

Fig. 2

 2 Fig. 2 Fit model parameters obtained from HMI during Cycle 24. (a) Temporal oblateness variations (red) are in anti-phase with sunspot number (black). (b) C 2 variations (blue dots) are in phase with activity. (c) C 4 (black dots) has anti-symmetric variations relative to the time of solar activity maximum.

Fig. 3 (

 3 Fig. 3 (a) All oblateness measurements made during Cycle 24 vs. sunspot number. A linear relationship is observed (R 2 =77%). The negative slope shows oblateness variations in antiphase with activity. (b) Same plot with all measurements recorded during Cycle 23. The linear regression (R 2 =79%) shows a positive slope, expressing oblateness variations in phase with activity.