Scalable Tools Workshop

What's New in Dyninst?

Tim Haines, James Kupsch, Josef (Bolo) Burger

Abstract

Binary analysis and instrumentation is a key technology to support performance profiling, debugging, testing, software security, and auditing. Dyninst is an open source suite of libraries providing binary analysis, instrumentation and control capabilities across several hardware architectures with an architecture-independent abstraction. It supports both dynamic (runtime) and static (binary rewriting) instrumentation of a binary program. Dyninst is opportunistic in that it uses symbol and debug information when it is available, but can operate without it, even on stripped binaries. Dyninst's analysis capabilities produce a control- and data-flow analysis of the program, identifying functions, loops, and basic blocks in the code. Dyninst allows fine-grained program instrumentation and modification based on a high-level (control flow graph) abstraction of a program.

Dyninst is structured as a suite of toolkit libraries, providing architecture independent interfaces to features such as instruction decoding, control flow analysis, data flow analysis, code generation, code patching (splicing) and symbol table processing. On the dynamic side, it also includes process control and stack walking support.

Dyninst has been used as the foundation for products from companies like Cray and Red Hat, as the basis for tools from national labs and research groups, and as a key component in hundreds of academic research projects. It continues to have the dual role of providing a foundation for new instrumentation and analysis research combined with support for key applications of binary analysis and instrumentation.

Since the last workshop in 2022, Dyninst has seen myriad improvements to functionality and code quality. Examples include support for AMD MI100 and MI200 GPUs; updates to support the latest compilers, language standards, libraries, and build tools; improvements to testing; and a wealth of correctness, performance, and usability enhancements discovered via user feedback, bug reports, our expanded test suites, and improved release testing. In this talk, we will provide a brief tutorial on how you can use Dyninst to take advantage of its capabilities and describe its new features and upcoming plans.

In this talk, we will review the features and structure of Dyninst, summarize the new developments, and present some practical examples of how you can use it.

Levels of Detail in Performance Analysis. A RISCV Vector codesign case

Jesus Labarta

Abstract

Computers, as with many other complex systems, can be described and analyzed at many levels. In these systems, it is desirable for tools to be able to cover a huge dynamic range of granularities (scale) and to present detailed metrics and advanced statistics at all those levels. I will present a tool infrastructure to analyze the behavior of applications on the RISC-V long vector processor being designed in the EPI project. The framework is used to derive co-design insight for steering the work of application, compiler and RTL developers. The infrastructure uses Paraver to gain understanding on the behavior of programs from the coarse grain of user functions or runtime calls to the microscopic level of instruction execution.

HPCToolkit: Experiences at Exascale

Jonathon Anderson

Abstract

With the arrival of Frontier, we have successfully used HPCToolkit to collect instruction-level call path profiles and traces for GPU-accelerated programs at exascale. To date, we have measured the performance of 65K MPI ranks of GPU-accelerated LAMMPS running for 15 minutes. HPCToolkit collected 5.5TB of CPU and GPU traces for this execution. Out-of-core strategies for data analysis and interactive presentation recently added to HPCToolkit make working with data at this scale practical. This talk will highlight these strategies and include a live demo.

Interactive Performance Visualization in Notebooks

Connor Scully-Allison

Abstract

In this talk, I will discuss the motivations and benefits for designing visualizations specifically to be embedded in literate programming environments like Jupyter notebooks. More specifically, I will detail how this intentional approach to visual + scripting design aids performance analysis in particular and demonstrate examples of visualizations I have developed for this purpose. I will also discuss some of the implementation details of loading these visualizations and passing data between the code used to render the visualization (Javascript) and the notebook's native language (Python). Finally, I will discuss some current and future directions of work in this space.

Towards Agile Development of High Performance Deep Learning Operators

Keren Zhou

Abstract

Deep learning programming models, such as TensorFlow and PyTorch, facilitate the development of innovative artificial intelligence applications. These models contain comprehensive libraries, which allow for efficient data manipulation, algorithm implementation, and model training. Nevertheless, these high-level programming models can sometimes fall short regarding computational efficiency. Low-level programming languages, such as CUDA and HIP, are often better for exploiting hardware-specific features, which can improve performance. However, to create custom operators - individual components of an algorithm - developers must put in considerable effort. In this talk, we introduce Triton, a Python-like language, specifically designed to streamline the development of deep learning operators. We will discuss the reasons behind Triton's development, its grammar, and the progress we have made so far. Triton differs from existing kernel programming languages in that it allows users to write understandable and high-performance GPU kernels. Triton has already been used in many popular open-source projects, such as PyTorch, PyTorch-Geometrics, and DeepSpeed.

Interactive Visualization of Binary Code for Investigating Compiler Optimizations

Shadmaan Hye or Kate Isaacs

Abstract

Investigating compiler optimizations requires navigating thousands of lines of disassembly and matching them to their generating source code. To alleviate this problem, we develop a web-based tool to interactive visual source and disassembly in a more intuitive way. Our tool incorporates automated analysis through the Dyninst library to present translations in the disassembly such as re-naming registers by variables, identifying functions, and displaying the structure of loops. We provide linked selection between source and disassembly, multiple source code files, and multiple disassembly interfaces to better support analysis processes. We are actively developing these interfaces to improve manual analysis of compiler optimizations.

Conquering Noise with Hardware Counters on HPC Systems

Felix Wolf

Abstract

With increasing system performance and complexity, it is becoming increasingly crucial to examine the scaling behavior of an application and thus determine performance bottlenecks at early stages. Unfortunately, modeling this trend is a challenging task in the presence of noise, as the measurements can become irreproducible and misleading, thus resulting in strong deviations from the actual behavior. While noise impacts the application runtime, it has little to no effect on some hardware counters like floating-point operations. However, selecting the appropriate counters for performance modeling demands some investigation. In this paper, we perform a noise analysis on various hardware counters. Using our noise generator, we add additional noise on top of the system noise to inspect the counters' variability. We perform the analysis on five systems with three applications in the presence of various noise patterns and categorize the counters across the systems according to their noise resilience.

PAPI 7.0 and Beyond

Anthony Danalis

Abstract

PAPI 7.0 marks a major update in the Performance API. With this release, PAPI extends the functionality it provides to its users further than ever before. Performance events and metrics originating in GPUs from all three major vendors are now fully supported through the same PAPI_start/PAPI_stop API. A brand new "sysdetect" functionality has been added, which is accessible to the users through both a new API and a new command line utility. Using this functionality, a user can detect a machine's architectural details, including the hardware's topology, specific aspects of the memory hierarchy, the number and type of GPUs and CPUs, etc. This paves the way for complex applications and runtime systems to programmatically discover the environment in which they are running and adapt their behavior accordingly. The Software Defined Events (SDE) functionality developed by PAPI has now become a stand-alone library. It offers multiple features that extend the notion of an event beyond the simple counter found in hardware devices. Using such events in conjunction with traditional counters offers application writers and tool developers opportunities for new types of analyses but also creates challenges in integrating the newly available information into existing performance analysis tools.

In this talk, we will discuss several of these new features, present findings related to the analysis of performance data and the effect of noise in performance counters, and initiate discussions with other tool developers on how to better integrate novel SDEs into their tools.

Detecting/Analyzing Unstable Performance Behavior

William Jalby

Abstract

Typically, the same loop or code region is executed billions of time during a whole application execution. Its execution time may vary significantly between each separate invocation (instance). However, classical (sampling) measurement techniques aggregate individual measurements making impossible to detect such “unstable” behavior across various instances. We will show how MAQAO/ONEVIEW provide mechanisms for detecting such instabilities and more generally how sampling and tracing techniques are used to improve measurement quality.

OSACA – A Multi-Platform Static Code Analyzer for In-core Performance Prediction

Jan Laukemann

Abstract

The Open-Source Architecture Code Analyzer (OSACA) is a static analysis tool for in-core performance prediction of inner-most loop assembly loops. It is heavily inspired by the Intel Architecture Code Analyzer (IACA), an Intel-proprietary tool that unfortunately reached end-of-life in 2019 and only supports Intel microarchitectures. We therefore developed OSACA as Open-Source alternative, supporting various Intel, AMD, and ARM-based micro-architectures for code analysis. These analyses can be used to identify bottlenecks in performance-critical passages, as well as for performance modeling and performance engineering, for example as more accurate ceiling in the Roofline model or as in-core contribution in the Execution-Cache-Memory (ECM) model. Furthermore, OSACA was developed in a way that it does not require executable code and to allow the user to change the underlying hardware model in an easy and fast way, thus, being able to provide architectural exploration of novel, non-existing micro-architectures. It is not only available as standalone Python-based command line tool, but is also integrated in Godbolt’s Compiler Explorer and consequently easy to access for any user. While there do exist alternatives, most prominently the LLVM Machine Code Analyzer (llvm-mca) or uiCA, OSACA tries to combine accuracy and versatility.

In this talk, we will introduce OSACA to the workshop community, explain its usage in our group both for academic research, as well as in educational tutorials, and want to discuss new features. We are very interested in discussing potential future paths of OSACA and requirement requests from the tools community.

minitest - a Framework for Testing a GPU Performance Tool

Marty Itzkowitz

Abstract

One of the challenges in developing GPU performance tools is measuring offloading on the GPU under various programming models, understanding the details of the offloading, including data copies, kernel launches and callstack attributions, and correlating those details with the CPU behavior. To address that challenge, we have developed minitest, a framework for testing HPCToolkit's GPU performance measurements. The framework consists of various scripts, and a set of directories for building a set of isomorphic applications whose behavior is well-understood.

Each application consists of a front-end and a back-end, compiled and linked with a particular compiler. The front-ends are either an OpenMP-threaded main program, a Posix-threaded main program, or a single-threaded main program. The back-ends support OpenMP-offloading to a GPU, vendor-specific offloading to a GPU, or non-offloading implementations. The applications are represented by a directory in a GPU-specific subdirectory of minitest, named by ... Each such directory contains only a Makefile. The GPU kernels are defined in an include file, included in all back-ends, thus ensuring that all the applications behave identically.

The minitest script takes a list of directories to traverse, and a list of tests to run in each directory. Each test is described by a data-descriptor, giving the number of worker-threads to invoke, and items specifying the arbitrary combination of data-collection options desired. As each test runs, it creates a log file, a measurements directory generated by hpcrun, and a database directory containing the results, as processed by hpcstruct and hpcprof. Those files and directories are all named with the data-descriptor, and each step of the data collection and processing is checked for errors. A mechanism is available to run each test under any of a list of GPU vendor versions.

Heterogeneous HPC Performance Analysis with trace Compass

Sébastien Darche

Abstract

The DORSAL group at Polytechnique Montréal, in collaboration with Ericsson, Ciena, AMD and EfficiOS, is conducting research on tracing, profiling, debugging and performance analysis tools, for large Cloud and HPC distributed applications. We will present our work on LTTng, Theia and Trace Compass, including support for analyzing the performance of HPC languages and frameworks such as MPI, OpenCL and HIP. Our work focuses on Trace Compass scalability, to handle large traces from hundreds or thousands of nodes, and low-overhead GPU tracing and profiling, by interfacing with ROCm.

AMD ROCm Tools Progress: Profiler/Tracer, Debugger, OMPT and Binary Instrumentation Support

Timour Paltashev

Abstract

ROCm is a brand name for the AMD ROCm open software platform or the ROCm open platform ecosystem (includes hardware like FPGAs or other CPU/GPU architectures). ROCm is focused on using AMD GPUs to accelerate computational tasks such as machine learning, engineering workloads, and scientific computing. ROCm supports a targeted set of hardware configurations with CPU accelerated by GPUs and FPGAs (expected). Along with support of HIP, OpenMP and OpenCL programming models, ROCm provides the set of profiling, tracing, and debugging APIs, tools and components supporting 3rd party HPC instrumentation tools like PAPI, Caliper, TAU, HPCToolkit, Score-P, TraceCompass/ Theia, LIKWID, ARM Forge and Perforce TotalView.

In this talk we will present the latest updates on rocprofiler V2 development which merges existing rocprofiler and roctracer libraries with new advanced functionality added. It will be followed by a brief overview of ROCm Debugger API and ROCGDB tool status. ROCm OMPT functionality support is necessary to enable HPCToolkit, TAU and Score-P working with OpenMP programming model on AMD platforms. We will also present few details about machine-readable XML ISA specification and related API project for AMD GPU product lines. It is targeted to support dynamic binary analysis and instrumentation of graphics and compute code objects in a variety of internal and 3rd party external code instrumentation tools, like Dyninst and others.