GPU
3 of 4 posts
More performance checks of NVIDIA PyTorch on DGX Spark
Following up with fp32, fp16 and bf16 across GEMM, convolution and TorchBench: where the NGC container actually wins, and where the DGX Spark is simply bandwidth-bound.
FindingGEMM gains hold; convolution unchanged; bandwidth-bound
Is it worth using the official NVIDIA PyTorch image?
An 8192² bfloat16 matmul run 100 times, inside NVIDIA's NGC PyTorch container and against a stock pip install, measured on the same DGX Spark.
Finding+50% TFLOPS · 8192² bf16 matmul
Oh no, 128 GB is not enough for 7B parameters!
A 7B model OOMs on a 128 GB machine. The memory is not missing — the OS page cache is holding it, and unified memory hides that from the usual GPU tooling.
FindingOS page cache, not the model, causes the OOM