Scalable Tools Workshop 2024

Triton Updates: Debugger, Profiler, Visualizer

Keren Zhou

Abstract

Over the past year, there has been substantial development on Triton, including third-party plugins, performance optimizations on NVIDIA Hopper GPUs, and the creation of several tools. In this presentation, we will focus on Triton's debugger, profiler, and visualizer. We will introduce an interactive debugging tool for easily running kernels written in Triton without the need for GPUs. This tool, supporting most Triton operations, allows users to instrument code and check results as NumPy arrays. We will also discuss Triton's kernel-level profiler, which provides low-overhead kernel-level profiling on NVIDIA and AMD GPUs. It can collect user annotations and program context, and then aggregate performance information into a concise profile for quick analysis. Additionally, we will introduce a visualization tool that displays tensor operations, such as memory load, store, and tensor computation, as 1D to 3D arrays. Invalid memory accesses are masked and highlighted to aid debugging in the visualizer. Furthermore, it allows for the examination of individual program instances through an interactive button.

Validating the Performance of GPU-Offloading with Differential Performance Models

Alexander Geiss

Abstract

Offloading work to the GPU is crucial to leverage many of today’s supercomputers. We do that expecting the offloaded part of an application to run faster than the pure CPU implementation, but is that true for your application – especially – when scaling up? We propose a semi-automatic workflow based on differential performance modeling to answer this question, even for individual parts of the application, so that potential issues are easier to locate. Our approach combines generation of empirical performance models based on combined CPU--GPU profiles with hardware characteristics to derive differential performance models that can be compared across device types. To complement the workflow, we have integrated the method into Extra-P and extended it with new features for interactive exploration of differential performance models.

Memory-Based, Locality-Driven Supercomputer Affinity

Edgar Leon

Abstract

Computer systems are becoming increasingly complex and may include many-core technologies, hybrid machines with throughput-optimized cores and latency-optimized cores, and multiple levels of memory. The complexity involved in extracting the performance benefits these systems offer challenges the productivity of computational scientists greatly. A significant part of this challenge involves mapping parallel applications efficiently to the underlying hardware. A poor mapping may result in dramatic performance loss. Furthermore, an application mapping is often machine-dependent breaking portability to favor performance.

In this talk, I will present mpibind, a memory-driven algorithm to map parallel hybrid applications to the underlying architecture transparently from the point of view of applications. This tool employs a simple interface for computational scientists and results in a full mapping of MPI tasks, threads, and GPU kernels to hardware processing units and memory domains. Furthermore, scientists do not have to deal with intricate details of the hardware topology and thus increasing their productivity. At Lawrence Livermore National Laboratory, we use mpibind to bridge the gap between performance and portability on commodity technology systems as well as advanced technology systems.

Invited Talk: AMD MI300A APU Architecture: Shader ISA Extension, Profiling and Instrumentation Support in ROCm

Timour Paltashev (speaker), Vladimir Indic and Nursultan Kabylkas

Abstract

The AMD Instinct™ MI300A APU is the 5th implementation of Exascale Heterogeneous Processor Concept. It consists of a mix of CPU and GPU chiplets along with a shared AMD Infinity Cache memory-side cache, eight stacks of HBM, I/O interfaces, and Infinity Fabric (IF) interconnect to provide the data movement among all these components.

Shader ISA were significantly updated to the CDNA-3 level, new counters and metrics added to support performance analysis model as well special hardware support for PC sampling mode of ROCm HPC/ML application monitoring. Machine-readable XML ISA specification with helper API is available with pending executable semantic description of GPU instructions.

Modeling compiler dependencies in Spack

Todd Gamblin

Abstract

Spack originally modeled compilers as simple node attributes, to allow users to easily swap compilers in and out of builds, and to allow users to link with packages built with different compilers. This model was flexible, but made it very hard to model the many ABI and consistency requirements that compiler toolchains impose on the code they build. It has been an effort over many years, but we are finally close to having compilers, their runtime libraries, and many of the relevant constraints that come with them modeled formally. This talk will cover the tractability challenges that we had to overcome to model toolchains properly, as well as the user-facing mechanisms Spack is starting to offer to model compilers, low-level runtime libraries, and their ABIs.