What are BaM and GIDS?
BaM is a system architecture that utilizes the low latency, extremely high throughput, large density, and endurance of SSDs. BaM’s goal is to provide efficient abstractions that enable GPU threads to make fine-grained accesses to datasets on SSDs and achieve much higher performance than solutions requiring CPU(s) to provide storage requests to serve GPUs. BaM acceleration uses a custom storage driver that is designed specifically to enable the inherent parallelism of GPUs to access storage devices directly. BaM is different from NVIDIA Magnum IO™ GPUDirect® Storage (GDS), as BaM doesn’t rely on the CPU to prepare the communication from GPU to SSD.
- The Micron® 9400 NVMe™ SSD Performance With NVIDIA Magnum IO GPUDirect Storage Platform
- Micron Collaboration With Magnum IO GPUDirect Storage Brings Industry-Disruptive Innovation to AI & ML
Graph neural network training using GIDS
To show the benefits of BaM and GIDS, we performed GNN training using the Illinois Graph Benchmark (IGB) heterogeneous full dataset. This dataset is 2.28TB large and would not fit into most platforms’ system memory. We timed the training for 100 iterations using a single NVIDIA A100 80GB Tensor Core GPU and varied the number of SSDs to provide a broad range of results, as seen in Figure 1 and Table 1.
![](http://media-2q.bodegapuenteajuda.com/-/media/client/global/images/blogs/content-images/2024/january/bam/mazzie-bamblog-figure-1.png?h=1353&w=2709&la=en&rev=573da641f0ec4762832b34667985b0d6&hash=767295F8BF6A098BB9A360F20B4B0DE9)
Figure 1: GIDS Training Time for IGB-Heterogenous Full Dataset - 100 Iterations
![](http://media-2q.bodegapuenteajuda.com/-/media/client/global/images/blogs/content-images/2024/january/bam/table1.png?h=200&w=809&la=en&rev=ba7e0b74ad154b43b57d31c8deb5c4bb&hash=21FEFCA7C1132313BA788E6EE18B77F1)
Table 1: GIDS Training Time for IGB-Heterogenous Full Dataset - 100 Iterations
The first part of the training is graph sampling done by the GPU and by accessing the graph structure data within system memory (seen in blue). This value varies little across the different test configurations because the structure stored in system memory does not change between these tests.
Another part is the actual training time (seen at the far right in green). This part is highly dependent on the GPU, and we can see that this does not change much between the multiple test configurations as expected.
The most important section, where we see the largest difference, is feature aggregation (shown in gold). As the feature data is stored on the Micron 9400 SSDs for this system, we see that scaling from 1 to 4 Micron 9400 SSDs drastically improves (reduces) the feature aggregation processing time. Feature aggregation improves by 3.68x as we scale from 1 SSD to 4 SSDs.
We also included a baseline calculation, which uses a memory map abstraction and the Deep Graph Library (DGL) data loader to access the feature data. Because this method of accessing the feature data requires the use of the CPU software stack instead of direct access by the GPU, we can see how inefficient the CPU software stack is at keeping the GPU saturated during training. The feature abstraction improvement versus baseline is 35.76x for 1 Micron 9400 NVMe SSD using GIDS and 131.87x on 4 Micron 9400 NVMe SSDs. Another view of this data can be seen in Figure 2 and Table 2, which shows the effective bandwidth and IOPs during these tests.
![](http://media-2q.bodegapuenteajuda.com/-/media/client/global/images/blogs/content-images/2024/january/bam/mazziebam-blog_figure-2.png?h=1081&w=2493&la=en&rev=14ef7ddf595b4f0b854c5a92c9746d3c&hash=A0F96BE1FD72BD9EFD2DB0A1BA9185CB)
Figure 2: Effective Bandwidth and IOPS of GIDS Training vs Baseline
![](http://media-2q.bodegapuenteajuda.com/-/media/client/global/images/blogs/content-images/2024/january/bam/table2.png?h=125&w=809&la=en&rev=e41a9a76573544db8fd596dc9c1ffc40&hash=F837F82EC60AF1CB8B3A58AE8A67B7C0)
Table 2: Effective Bandwidth and IOPS of GIDS Training vs Baseline
As datasets continue to grow, we can see the need for a shift in paradigm in order to train these models in a reasonable amount of time and to take advantage of the improvements provided by leading GPUs. BaM and GIDS are a great starting point, and we look forward to working with more of these types of systems in the future.
Test System
Component |
Details |
Server |
Supermicro® AS 4124GS-TNR |
CPU |
|
Memory |
1 TB Micron DDR4-3200 |
GPU |
Memory Clock: 1512 MHz SM Clock: 1410 MHz |
SSDs |
4x Micron 9400 MAX 6.4TB |
OS |
Ubuntu 22.04 LTS, Kernel 5.15.0.86 |
NVIDIA Driver |
535.113.01 |
Software Stack |
CUDA 12.2, DGL 1.1.2, Pytorch 2.1 running in NVIDIA Docker container |
Reference Links
Big Accelerator Memory Paper & GitHub
GPU-Initiated On-Demand High-Throughput Storage Access in the BaM System Architecture (arxiv.org)
GitHub - ZaidQureshi/bam
GPU Initiated Direct Storage Paper and GitHub
Accelerating Sampling and Aggregation Operations in GNN Frameworks with GPU Initiated Direct Storage Accesses (arxiv.org)
GitHub - jeongminpark417/GIDS
GitHub - IllinoisGraphBenchmark/IGB-Datasets: Largest real-world open-source graph dataset - Worked done under IBM-Illinois Discovery Accelerator Institute and Amazon Research Awards and in collaboration with NVIDIA Research.
*Note: NVIDIA Big Accelerator Memory (BaM) and NVIDIA GPU Initiated Direct Storage (GIDS) dataloader are prototype projects from NVIDIA Research and are not intended for general release.