Analyzing GPU Energy Consumption in Data Movement and Storage - MIC
Pré-Publication, Document De Travail Année : 2024

Analyzing GPU Energy Consumption in Data Movement and Storage

Jonathan Miquel
Debjyoti Bhattacharjee
Diksha Moolchandani
Francky Catthoor
Lionel Torres
David Novo

Résumé

GPUs are the prevailing solution to execute high- performance tasks (e.g., machine learning training). As the peak performance of modern GPUs increases with each generation, so does their thermal design power (TDP). Hence, identifying energy bottlenecks in the GPU architecture is crucial to designing more efficient architectures in the future. However, due to the complex proprietary nature of modern GPU architectures, providing a detailed breakdown of the GPU energy consumption is not trivial. The goal of this work is to estimate a lower bound for the energy consumed by data movement and storage in modern GPU architectures, leveraging internal power sensors. We establish a basic energy model for modern GPUs, focused on data movement to/from the hardware-managed caches and software-managed memories. We propose a methodology to calibrate the energy model using microbenchmarks, performance counters, and the internal power sensor. We experimentally calibrate the model on an A100 NVIDIA GPU. Then, we challenge the consistency of the results by cross-validating with modified microbenchmarks with additional instructions. Finally, we use the calibrated energy model to evaluate breakdowns for workloads of increasing complexity (e.g., a ResNet-50 training iteration with different software optimizations). Our results show that data movement dominates the dynamic energy consumption of the GPU (up to 84%), with DRAM accesses being the main contributor.
Fichier principal
Vignette du fichier
delestrac2024analyzing.pdf (715.59 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04604802 , version 1 (07-06-2024)

Identifiants

Citer

Paul Delestrac, Jonathan Miquel, Debjyoti Bhattacharjee, Diksha Moolchandani, Francky Catthoor, et al.. Analyzing GPU Energy Consumption in Data Movement and Storage. 2024. ⟨hal-04604802⟩
60 Consultations
99 Téléchargements

Altmetric

Partager

More