Scalable Tools Workshop 2022

A Fine-grained CPU-GPU Analysis Framework

Yueming Hao

Abstract

General-purpose GPUs have become common in modern computing systems to accelerate applications in many domains. However, inefficiencies abound in GPU-accelerated applications, which prevent them from obtaining bare-metal performance. Performance tools play an important role in understanding performance inefficiencies in complex code bases. Many GPU performance tools pinpoint time-consuming code and provide high-level performance insights but overlook one important performance issue - value-related inefficiencies, which exist in many GPU code bases. What's more, those GPU performance tools usually only focus on GPU kernels which may lose optimization opportunities of CPU-GPU interactions. In this talk, we will present our fine-grained CPU-GPU analysis framework which pinpoints value-related inefficiencies and optimizable CPU-GPU interactions.

This framework monitors application execution to capture values produced and used by each load and store operation in both CPU functions and GPU kernels, recognizes multiple value patterns, and provides intuitive optimization guidance. We address systemic challenges in collecting, maintaining, and analyzing voluminous performance data from CPU functions and many GPU threads to make it applicable to complex applications. We evaluate our framework on a wide range of well-tuned benchmarks and applications.

Fine Grained Binary Instrumentation for GPU Profiling

Hsuan-Heng Wu

Abstract

Instrumentation of binary code is a powerful technique that has been applied in many areas, including collecting fine-grained execution data for application performance tuning. Our experiences with the Dyninst binary analysis and instrumentation toolkits have shown that performance data can be collected efficiently for CPUs, and we are now applying these techniques to GPUs, specifically to the AMD family of GPUs.

We will first give a brief introduction to the AMD GPU environment. Then we will discuss our design for efficient GPU kernel instrumentation, including: device side resource allocation for storing performance data, efficient transfer of performance data from the GPU to CPU, and keeping the instrumentation overhead low compared to other approaches. We use binary instrumentation to avoid the need to recompile the kernel.

In our early experiments with these techniques, we have collected branch-divergence information, which can be used as an indicator of GPU hardware utilization, on several GPU kernels. We have also compared our instrumentation overhead to NVIDIA’s CUTPI for gathering similar information, and our overhead is consistently much lower than this other tool.

Using Hybrid Cores to Optimize Performance, Power and Throughput

Michael Chynoweth

Abstract

We are targeting using P-Cores for maximum performance in limited threading scenarios and E-Cores for scaling multi-threaded performance and when power is required by other IPs. This talk is going to cover how we are optimizing code generation, core selection and scheduling utilizing both online and offline telemetry to make the best decisions possible for the software. It will also give some hints on where Intel is taking hybrid performance going forward into the future.

Comparative Evaluation of Call Graph Generation by Profiling Tools

Onur Cankur

Abstract

Call graphs generated by profiling tools are critical to dissecting the performance of parallel programs. Although many mature and sophisticated profiling tools record call graph data, each tool is different in its runtime overheads, memory consumption, and output data generated. In this work, we perform a comparative evaluation study on the call graph data generation capabilities of several popular profiling tools -- Caliper, HPCToolkit, TAU, and Score-P. We evaluate their runtime overheads, memory consumption, and generated call graph data (size and quality). We perform this comparison empirically by executing several proxy applications, AMG, LULESH, and Quicksilver on a parallel cluster. Our results show which tool results in the lowest overheads and produces the most meaningful call graph data under different conditions.

Answer Set Solving in Spack: Tackling combinatorial software complexity head-on

Todd Gamblin

Abstract

The number of ways that software can be built and composed is not getting any smaller. These days every software language, ecosystem, and developer community has a package manager to help manage the growing number of libraries. Package managers convert abstract requests for software into concrete build or installation instructions. This involves: 1) managing complex compatibility constraints, 2) finding optimal solutions, and 3) solving an NP-complete problem.

Modeling compatibility semantics and optimization criteria in a way a solver can understand is a daunting task. This talk will describe our recent work to include increasingly complex package compatibility data in Spack using Answer Set Solving, and it will outline the remaining challenges we face as we try to model the low-level details of the HPC software ecosystem.

Building a DSL for ABI compatibility

Nate Hanford and Greg Becker

Abstract

Most package management systems use semantic versioning or similar conventions to describe the compatibility of packages. On the BUILD project, we are working to incorporate ABI compatibility information derived from binary analysis in order to get a more accurate picture of how binaries can be composed. Additionally, we are working on a domain-specific language (DSL) so that package authors can describe ABI breaks and other version incompatibilities without vain detailed analysis. This talk will cover our DSL and its potential use going forward in Spack and the broader community.