Scalable Tools Workshop

What's New in Dyninst?

Tim Haines, James Kupsch, Josef (Bolo) Burger

Abstract

Binary analysis and instrumentation is a key technology to support performance profiling, debugging, testing, software security, and auditing. Dyninst is an open source suite of libraries providing binary analysis, instrumentation and control capabilities across several hardware architectures with an architecture-independent abstraction. It supports both dynamic (runtime) and static (binary rewriting) instrumentation of a binary program. Dyninst is opportunistic in that it uses symbol and debug information when it is available, but can operate without it, even on stripped binaries. Dyninst's analysis capabilities produce a control- and data-flow analysis of the program, identifying functions, loops, and basic blocks in the code. Dyninst allows fine-grained program instrumentation and modification based on a high-level (control flow graph) abstraction of a program.

Dyninst is structured as a suite of toolkit libraries, providing architecture independent interfaces to features such as instruction decoding, control flow analysis, data flow analysis, code generation, code patching (splicing) and symbol table processing. On the dynamic side, it also includes process control and stack walking support.

Dyninst has been used as the foundation for products from companies like Cray and Red Hat, as the basis for tools from national labs and research groups, and as a key component in hundreds of academic research projects. It continues to have the dual role of providing a foundation for new instrumentation and analysis research combined with support for key applications of binary analysis and instrumentation.

Since the last workshop in 2022, Dyninst has seen myriad improvements to functionality and code quality. Examples include support for AMD MI100 and MI200 GPUs; updates to support the latest compilers, language standards, libraries, and build tools; improvements to testing; and a wealth of correctness, performance, and usability enhancements discovered via user feedback, bug reports, our expanded test suites, and improved release testing. In this talk, we will provide a brief tutorial on how you can use Dyninst to take advantage of its capabilities and describe its new features and upcoming plans.

In this talk, we will review the features and structure of Dyninst, summarize the new developments, and present some practical examples of how you can use it.

Score-P: Beyond the Infinite HPC domain

Bert Wesarg, Bill Williams

Abstract

Score-P is a well-established framework for instrumentation and measurement of parallel codes. It has its origin in the classic HPC field. The HPC field is characterized by providing tightly coupled communication methods and shared file systems to applications, which Score-P also profits from. These are, and always will be, key features for HPC applications and measurement frameworks to achieve exceptional performance and scale on HPC systems. However, in recent years, other distributed frameworks have entered the HPC domain, which do not rely on these properties, but require immense computing power and energy consumption. Providing state-of-the-art performance analysis tools from the HPC community to these new communities will be very beneficial for them. Examples of such frameworks are common machine learning/AI frameworks like PyTorch or TensorFlow; or Java-based frameworks such as Hadoop.

We are in the process of extending Score-P to provide its own distributed communication layer via a client/server model based on TCP. In contrast with the common MPI communication used in HPC application, where Score-P uses the same communication layer as the application, this communication layer will be used only by Score-P. This allows it to support future frameworks, independent of what new communication methods those frameworks may use. We foresee a wide range of applications for such models, including, but not limited to commercial cloud infrastructure, embedded development, and lastly Jupyter Notebooks.

HPCToolkit: Experiences at Exascale

Jonathon Anderson

Abstract

With the arrival of Frontier, we have successfully used HPCToolkit to collect instruction-level call path profiles and traces for GPU-accelerated programs at exascale. To date, we have measured the performance of 65K MPI ranks of GPU-accelerated LAMMPS running for 15 minutes. HPCToolkit collected 5.5TB of CPU and GPU traces for this execution. Out-of-core strategies for data analysis and interactive presentation recently added to HPCToolkit make working with data at this scale practical. This talk will highlight these strategies and include a live demo.

Interactive Visualization of Binary Code for Investigating Compiler Optimizations

Shadmaan Hye or Kate Isaacs

Abstract

Investigating compiler optimizations requires navigating thousands of lines of disassembly and matching them to their generating source code. To alleviate this problem, we develop a web-based tool to interactive visual source and disassembly in a more intuitive way. Our tool incorporates automated analysis through the Dyninst library to present translations in the disassembly such as re-naming registers by variables, identifying functions, and displaying the structure of loops. We provide linked selection between source and disassembly, multiple source code files, and multiple disassembly interfaces to better support analysis processes. We are actively developing these interfaces to improve manual analysis of compiler optimizations.

Interactive Performance Visualization in Notebooks

Connor Scully-Allison

Abstract

In this talk, I will discuss the motivations and benefits for designing visualizations specifically to be embedded in literate programming environments like Jupyter notebooks. More specifically, I will detail how this intentional approach to visual + scripting design aids performance analysis in particular and demonstrate examples of visualizations I have developed for this purpose. I will also discuss some of the implementation details of loading these visualizations and passing data between the code used to render the visualization (Javascript) and the notebook's native language (Python). Finally, I will discuss some current and future directions of work in this space.

Towards Agile Development of High Performance Deep Learning Operators

Keren Zhou

Abstract

Deep learning programming models, such as TensorFlow and PyTorch, facilitate the development of innovative artificial intelligence applications. These models contain comprehensive libraries, which allow for efficient data manipulation, algorithm implementation, and model training. Nevertheless, these high-level programming models can sometimes fall short regarding computational efficiency. Low-level programming languages, such as CUDA and HIP, are often better for exploiting hardware-specific features, which can improve performance. However, to create custom operators - individual components of an algorithm - developers must put in considerable effort. In this talk, we introduce Triton, a Python-like language, specifically designed to streamline the development of deep learning operators. We will discuss the reasons behind Triton's development, its grammar, and the progress we have made so far. Triton differs from existing kernel programming languages in that it allows users to write understandable and high-performance GPU kernels. Triton has already been used in many popular open-source projects, such as PyTorch, PyTorch-Geometrics, and DeepSpeed.

Conquering Noise with Hardware Counters on HPC Systems

Felix Wolf

Abstract

With increasing system performance and complexity, it is becoming increasingly crucial to examine the scaling behavior of an application and thus determine performance bottlenecks at early stages. Unfortunately, modeling this trend is a challenging task in the presence of noise, as the measurements can become irreproducible and misleading, thus resulting in strong deviations from the actual behavior. While noise impacts the application runtime, it has little to no effect on some hardware counters like floating-point operations. However, selecting the appropriate counters for performance modeling demands some investigation. In this paper, we perform a noise analysis on various hardware counters. Using our noise generator, we add additional noise on top of the system noise to inspect the counters' variability. We perform the analysis on five systems with three applications in the presence of various noise patterns and categorize the counters across the systems according to their noise resilience.

PAPI 7.0 and Beyond

Anthony Danalis

Abstract

PAPI 7.0 marks a major update in the Performance API. With this release, PAPI extends the functionality it provides to its users further than ever before. Performance events and metrics originating in GPUs from all three major vendors are now fully supported through the same PAPI_start/PAPI_stop API. A brand new "sysdetect" functionality has been added, which is accessible to the users through both a new API and a new command line utility. Using this functionality, a user can detect a machine's architectural details, including the hardware's topology, specific aspects of the memory hierarchy, the number and type of GPUs and CPUs, etc. This paves the way for complex applications and runtime systems to programmatically discover the environment in which they are running and adapt their behavior accordingly. The Software Defined Events (SDE) functionality developed by PAPI has now become a stand-alone library. It offers multiple features that extend the notion of an event beyond the simple counter found in hardware devices. Using such events in conjunction with traditional counters offers application writers and tool developers opportunities for new types of analyses but also creates challenges in integrating the newly available information into existing performance analysis tools.

In this talk, we will discuss several of these new features, present findings related to the analysis of performance data and the effect of noise in performance counters, and initiate discussions with other tool developers on how to better integrate novel SDEs into their tools.

Detecting/Analyzing Unstable Performance Behavior

William Jalby

Abstract

Typically, the same loop or code region is executed billions of time during a whole application execution. Its execution time may vary significantly between each separate invocation (instance). However, classical (sampling) measurement techniques aggregate individual measurements making impossible to detect such “unstable” behavior across various instances. We will show how MAQAO/ONEVIEW provide mechanisms for detecting such instabilities and more generally how sampling and tracing techniques are used to improve measurement quality.

Efficient Branch Divergence Analysis Using Fine-Grained Binary Instrumentation on the AMD GPU

Hsuan-Heng Wu

Abstract

The instrumentation of binary code is a potent technique that has widespread applications, including the collection of fine-grained execution data for optimizing application performance. Our team's experience with the Dyninst binary analysis and instrumentation toolkits has demonstrated that performance data can be gathered efficiently for CPUs. We are now leveraging these techniques to gather performance data for AMD GPUs, thereby enhancing our understanding of their performance characteristics.

We begin by providing a brief introduction to the AMD ROCM environment and the necessary steps to support static GPU binary instrumentation for HIP applications. Subsequently, we discuss our strategies for reducing instrumentation overhead and outline the limitations of our approach. Finally, we evaluate our instrumentation overhead for collecting branch-divergence information (which can serve as an indicator of GPU hardware utilization)and benchmark it against that of NVIDIA's NVBIT and Intel's GTPIN, providing a comparative analysis.

minitest - a Framework for Testing a GPU Performance Tool

Marty Itzkowitz

Abstract

One of the challenges in developing GPU performance tools is measuring offloading on the GPU under various programming models, understanding the details of the offloading, including data copies, kernel launches and callstack attributions, and correlating those details with the CPU behavior. To address that challenge, we have developed minitest, a framework for testing HPCToolkit's GPU performance measurements. The framework consists of various scripts, and a set of directories for building a set of isomorphic applications whose behavior is well-understood.

Each application consists of a front-end and a back-end, compiled and linked with a particular compiler. The front-ends are either an OpenMP-threaded main program, a Posix-threaded main program, or a single-threaded main program. The back-ends support OpenMP-offloading to a GPU, vendor-specific offloading to a GPU, or non-offloading implementations. The applications are represented by a directory in a GPU-specific subdirectory of minitest, named by ... Each such directory contains only a Makefile. The GPU kernels are defined in an include file, included in all back-ends, thus ensuring that all the applications behave identically.

The minitest script takes a list of directories to traverse, and a list of tests to run in each directory. Each test is described by a data-descriptor, giving the number of worker-threads to invoke, and items specifying the arbitrary combination of data-collection options desired. As each test runs, it creates a log file, a measurements directory generated by hpcrun, and a database directory containing the results, as processed by hpcstruct and hpcprof. Those files and directories are all named with the data-descriptor, and each step of the data collection and processing is checked for errors. A mechanism is available to run each test under any of a list of GPU vendor versions.

Heterogeneous HPC Performance Analysis with trace Compass

Sébastien Darche

Abstract

The DORSAL group at Polytechnique Montréal, in collaboration with Ericsson, Ciena, AMD and EfficiOS, is conducting research on tracing, profiling, debugging and performance analysis tools, for large Cloud and HPC distributed applications. We will present our work on LTTng, Theia and Trace Compass, including support for analyzing the performance of HPC languages and frameworks such as MPI, OpenCL and HIP. Our work focuses on Trace Compass scalability, to handle large traces from hundreds or thousands of nodes, and low-overhead GPU tracing and profiling, by interfacing with ROCm.

AMD ROCm Tools Progress: Profiler/Tracer, Debugger, OMPT and Binary Instrumentation Support

Timour Paltashev

Abstract

ROCm is a brand name for the AMD ROCm open software platform or the ROCm open platform ecosystem (includes hardware like FPGAs or other CPU/GPU architectures). ROCm is focused on using AMD GPUs to accelerate computational tasks such as machine learning, engineering workloads, and scientific computing. ROCm supports a targeted set of hardware configurations with CPU accelerated by GPUs and FPGAs (expected). Along with support of HIP, OpenMP and OpenCL programming models, ROCm provides the set of profiling, tracing, and debugging APIs, tools and components supporting 3rd party HPC instrumentation tools like PAPI, Caliper, TAU, HPCToolkit, Score-P, TraceCompass/ Theia, LIKWID, ARM Forge and Perforce TotalView.

In this talk we will present the latest updates on rocprofiler V2 development which merges existing rocprofiler and roctracer libraries with new advanced functionality added. It will be followed by a brief overview of ROCm Debugger API and ROCGDB tool status. ROCm OMPT functionality support is necessary to enable HPCToolkit, TAU and Score-P working with OpenMP programming model on AMD platforms. We will also present few details about machine-readable XML ISA specification and related API project for AMD GPU product lines. It is targeted to support dynamic binary analysis and instrumentation of graphics and compute code objects in a variety of internal and 3rd party external code instrumentation tools, like Dyninst and others.