UC Riverside

Understanding GPU memory hierarchy is essential for achieving high performance in scientific and machine learning workloads. This thesis analyzes memory latency on AMD Instinct™ MI300A, MI300X, and MI250X GPUs using a fine-grained pointer chasing microbenchmark. We characterize the scalar L1 (sL1), L2, AMD Infinity Cache™ (MALL), and HBM, revealing clear latency levels and architectural differences. The MI300A and MI300X, based on the AMD CDNA™ 3 architecture, show similar behavior, while the MI250X (AMD CDNA™ 2) differs due to the absence of a MALL. Compute partitioning has negligible impact on latency, but NUMA Partitioning per Socket (NPS) reduces latency by up to 1.42× in MALL and 1.31× in HBM. Notably, we find that NPS4 not only partitions the HBM stacks but also divides the 256 MB MALL into four 64 MB slices. We further analyze Translation Lookaside Buffer (TLB) behavior under varying parallelism levels and identify conditions where MALL latency rises. These findings provide actionable guidance for optimizing memory-bound workloads on AMD GPUs.