Projects

D3: Multilevel Approaches and Adaptivity in Scientific Computing

Principal Investigators:

Prof. Bungartz, Prof. Gerndt

Scientific Researchers:

Prof. Michael Bader, Andreas Hollmann, Tobias Neckel, Martin Schreiber, Josef Weidendorfer, Tobias Weinzierl

Abstract

In subproject D3, the potential of invasive computing for scientific computing is explored. We examine how numerical algorithms need to be adapted to invasion-enabled hardware, and how invasive applications can be realised on standard HPC systems based on iOpenMP, an extension of OpenMP. We demonstrate the advantages of invasion in scientific computing via implementations of algorithmic building blocks on invasive hardware and, as a demonstrator, Tsunami simulations on classical multi- and many-core HPC systems. Thus, subproject D3 will have a strong impact on both scientific computing and invasive programming.

News

Synopsis

All hardware platforms for scientific computing follow the general trend to more parallelism, via multi- and many-core processors, GPUs, FPGAs, MPSoCs, and processor arrays. On the application side, the complexity of scientific computing algorithms is increasing as well. Hierarchy and dynamic adaptivity are essential paradigms for large-scale numerical simulations.

In contrast, programming models and the management of cores and memories in HPC architectures are still dominated by static allocation. Resources are allocated by batch scheduling systems, and the standard programming interfaces OpenMP and MPI focus on the efficient execution of applications on an initially given set of resources. Most applications on HPC systems are adapted to this static resource allocation. Either there is no additional parallelism in program phases that can be exploited or this parallelism is not exploited to not reduce the overall efficiency.

Hence, the goal of subproject D3 is to study invasion as a paradigm to bring together the varying amount of parallelism in multilevel adaptive applications with growing numbers of cores. We identify algorithmic approaches and best practices to implement key components of scientific applications on the basis of the invasion principle and on invasive hardware. In addition, we determine extensions to the resource management of HPC architectures and the standard programming interfaces MPI and OpenMP to enable invasive programming on future HPC systems for existing simulation codes.

In the algorithmic part, we study invasive approaches for dynamically adaptive multilevel solvers. Partitions are allowed to dynamically grow and shrink—w. r. t. partition size, refinement level, and multigrid levels—and especially to split and merge in terms of parallel threads according to the invasion paradigm, to tackle the resulting load-balancing challenges. The respective core algorithms are extracted from an existing hierarchical approach to compute wave propagation in the context of Tsunami simulation, which we choose as a realistic and challenging demonstrator. We implement these core algorithms using invasive programming languages and on first prototypes of invasive hardware.

To demonstrate the advantages of invasive computing to the HPC community, extensions to OpenMP and MPI for invasive programming are required to be minimal, to enable an incremental invasification, but also support data locality. We concentrate on iOpenMP, an invasive version of OpenMP, since OpenMP already supports flexible utilisation of a fixed set of resources, and since the first invasive hardware developments are of shared memory type, too. A prototypical implementation of iOpenMP is realised on standard shared memory HPC systems to serve as programming model for an invasive Tsunami simulation. The step towards clusters of such compute nodes, that is, hybrid systems, is planned for the second funding phase.

Hence, subproject D3 has a double impact: It tackles urgent problems in HPC and it helps to widen the scope of invasion towards HPC, which is essential for invasive computing to become a widely accepted paradigm for parallel programming.

Current state:

Application:

Example: Interactive adaptive simulation based on Shallow Water Equation and Discontinuous Galerkin Discretization. Parallelization is achieved via massive-tree splits and joins.

Visualization of the grid.

Invasive Resource Manager:

Scalability graph for the above mentioned problem. Depending on the probme size which is given by the number of cells, the scalability changes. Changing of scalability is forwarded to the resource manager.

Results:

Four shallow water simulations are started with a small delay. The resource manager distributes the cores to the application to maximize global throughput depending on the current scalability graph of the application. During start of the 2nd application (blue), less cores are assigned to it compared to the first application (red) since the scalability is worse.

Tsunami Simulation:

Tsunami Simulation for Tohoku Tsunami (In Germany known as the 'Fukushima Tsunami')
More information about simulation framework is available at Sierpinski

Publications