Scalable Tools Workshop

Dyninst and MRNet Update

Bill Williams and Sasha Nicolas

Abstract

The Dyninst suite of tools continues to evolve as we face the challenges of analyzing and instrumenting increasingly complex binaries in exascale environments. In the past year, we have continued work to improve our support for new architectures (ARM64), cross-architecture binary analysis, and performance improvements. As we work towards Dyninst 10.0, we want to share our vision for a better way to expose Dyninst's components to users and make our APIs easier to use.

Continuous Clock Synchronization for Accurate Performance Analysis

Johannes Ziegenbalg

Abstract

Clock synchronization is a key prerequisite for accurate performance analysis. This is particularly true for most HPC systems, due to the lack of a system-wide timer. The effects of different clock properties lead to false measurements and wrong conclusions. Often these errors remain undetected by the user. Different environmental effects cause continuous changes to clock properties over time. This behavior is ignored by the usual post-mortem approach for clock synchronization. In order to improve time measurement accuracy we implemented a method for continuous clock synchronization. In this talk we share our experiences and draw conclusions for tool development.

SNOflake - A Scalable Network Overlay for Multi-domain HPC Environments

Michael Brim

Abstract

SNOflake is a general purpose Scalable Network Overlay technology targeted at HPC environments. SNOflake supports deployments of overlay networks spanning many network domains (e.g., compute clusters, service nodes, and storage system). SNOflake extends the tree-based overlay network (TBON) technology pioneered by MRNet into a ring-of-TBONs architecture where one or more TBONs are placed on each network domain. This presentation will first cover the SNOflake design goals and architecture, and then discuss two ORNL software technologies, MELT and ExCDS, that leverage SNOflake for scalable end-to-end monitoring and management of HPC storage I/O.

Anticipating Aurora: Valgrind, AVX-512, and Intel HPC Analysis Tools

Tatyana Mineeva and Rashawn Knapp

Abstract

In collaboration with performance tool developers, our team enables open-source HPC analyzers and measurement tools for CORAL (Collaboration of Oak Ridge, Argonne, and Livermore) supercomputer installations at Argonne National Laboratory. We present our progress on adding support for AVX-512 vector instructions to Valgrind, a dynamic instrumentation framework, widely known for its memcheck tool. This support will allow Valgrind to instrument binaries containing AVX-512 instructions on the most current Intel(tm) Xeon Phi processor, codenamed Knights Landing (KNL), the recent Xeon Skylake server platforms, and Aurora, the successor to Argonne’s KNL based Theta. We will highlight the notable features of the AVX-512 extensions, and our contributions to date. Additionally, we provide an HPC focused overview of capabilities available with Intel performance analysis tools. Recently, a number of HPC-oriented features, including a roofline model and vectorization advice, have been added to Intel tools. We will show case several examples of using some of features on a single KNL node and on a modest-sized KNL cluster. Most of Intel’s performance tools are available at no cost to qualified open source contributors, students, and educators for non-commercial use.

State Of The Performance Tools (A Personal Opinion)

Doug Pase

Abstract

At Sandia, it isn't the unique challenges of exascale computing that are getting in the way of tool use. It is the mundane issues of reliability, robustness, being able to work with large scale applications, run on large scale systems under real life environments, etc. Tools are, for the most part, light enough, fast enough, and manage data well enough for very large core count applications. In the author's experience, where they fail is being able to run at all on production applications in production environments. Failures often occur when the application is large (in terms of LOC), or when production operating systems, compilers, languages, and libraries (e.g., Kokkos, Trilinos, or MKL) are used. It's not sufficient that a tool work for small applications on 100,000 cores if it doesn't also work on real codes running with systems, libraries, compilers and languages found in production environments.

Exposing Hidden Performance Optimizations in High Performance GPU Applications

Benjamin Welton

Abstract

The emergence of leadership class systems with nodes containing many-core accelerators, such as GPUs, has the potential to vastly increase the performance of distributed applications. Exploiting the additional parallelism that many-core accelerators offer is fraught with challenges. Developers and existing performance tools focus on a subset of these challenges, primarily the identification of CPU code that may be suited for many-core parallelization and improving the efficiency of existing many-core code. While this focus has resulted in application performance improvements, a significant amount of untapped performance still remains. Untapped performance opportunities take the form of missed unobvious many-core parallelization opportunities as well as inefficiencies in handling interactions with the accelerator, such as memory copies and synchronization. In this work we address three issues: (1) characterize the missed performance opportunities in many-core applications that are not detected by current performance tools and techniques, (2) design detection methods that can be used by performance tools to identify these missed opportunities, and (3) apply these techniques to five large scale scientific applications (Qball, QBox, Hoomd-blue, LAMMPs, and cuIBM), resulting in a reduction of their execution time by 18% and 87%.

Continuous Metric Collection and Assessment - an Ecosystem Approach

David Montoya

Abstract

We have traditionally assessed performance at the application internals and its immediate resources. With the complexities of increasing scale and unique architectures, performance has taken a more holistic perspective. How the computational environment is designed, its constraints, and ability to balance resource use is a key aspect in performance. There are varying perspectives of performance depending on differing stakeholders. This talk will review many of the drivers, approaches, and challenges of an integrated performance environment.

Extending Performance Monitoring Profile Guided Optimization (PGO) Capabilities

Michael Chynoweth

Abstract

Currently the last branch records are being used to determine the paths of execution and optimize code layout with profile guided optimizations. This presentation will cover the research we are accomplishing to take PGO beyond fixing text/code section issues and how it will be used to assist targeted optimizations through determining which portion of the pipeline is stalled and apply optimizations specifically for that bottleneck. Specifically we will cover aggressive loop optimizations, data layout, lock/cache contention determined through use of ~100 performance monitoring events and automatically apply the right compiler optimizations for the execution characteristics of each application.

TMAM: Top-down Microarchitecture Analysis Method simplifies Cycle Accounting

Ahmad Yasin

Abstract

Top-down Microarchitecture Analysis Method (TMAM) is an industry-proven systematic approach that identifies performance bottlenecks in out-of-order cores. Identifying true bottlenecks lets developers focus software tuning to remediate them and improve efficiently on same hardware. It further enables computer architects to better understand resource demands of emerging workloads when designing future hardware. TMAM simplifies cycle-accounting using microarchitecture-independent metrics organized in one single hierarchy which makes analysis simple. Using TMAM, the high-learning curve associated with each microarchitecture generation is replaced by a structured drill-down that guides the user to true performance limiters. This talk will present the key challenges that cycle-accounting faces and how the basic metrics of TMAM address them on Intel Core/Xeon products.

Isoefficiency in Practice: Configuring and Understanding the Performance of Task-based Applications

Sergei Shudler

Abstract

Task-based programming offers an elegant way to express units of computation and the dependencies among them, making it easier to distribute the computational load evenly across multiple cores. However, this separation of problem decomposition and parallelism requires a sufficiently large input problem to achieve satisfactory efficiency on a given number of cores. Unfortunately, finding a good match between input size and core count usually requires significant experimentation, which is expensive and sometimes even impractical. In this paper, we propose an automated empirical method for finding the isoefficiency function of a task-based program, binding efficiency, core count, and the input size in one analytical expression. This allows the latter two to be adjusted according to given (realistic) efficiency objectives. Moreover, we not only find (i) the actual isoefficiency function but also (ii) the function one would yield if the program execution was free of resource contention and (iii) an upper bound that could only be reached if the program was able to maintain its average parallelism throughout its execution. The difference between the three helps to explain low efficiency, and in particular, it helps to differentiate between resource contention and structural conflicts related to task dependencies or scheduling. The insights gained can be used to co-design programs and shared system resources.

Is it me, or is it the machine?

Judit Gimenez

Abstract

Sometimes programmers do a good job but the execution does not report the expected performance. The first step to improve the performance is to understand what is going on, and flexible tools like Paraver are key to face unknown problems that can be anywhere. During the tools training, I used to ask for a code from the attendees to be used for the demo, but sometimes no example is provided. In these cases I run the Lulesh benchmark. As a result, I have collected Paraver traces of Lulesh running on many different systems. The talk will present the high variability I have observed on the achieved performance with the same source code, as well as the ability of our performance tools to pinpoint what is causing the loss of performance.