Deployment & Scaling · Practitioner
GPU Selection and Memory Sizing: Choosing the Right Card on Purpose
Most GPU purchases and rentals are decided by rumour. Learn to compute the memory a workload needs, read bandwidth and precision support, and produce a defensible one-page recommendation instead of 'we should get H100s'.
- 1.2 hours lesson
- intermediate level
- NVIDIA AI Track
- Narrated presentation
What you’ll learn
- Weight memory = parameters x bytes per parameter; precision changes the answer by 4x
- KV cache scales with batch size and context length and often dominates at high concurrency
- Training memory is several times inference memory, hence LoRA and QLoRA
- Decode throughput is bounded by memory bandwidth, not peak FLOPS
- L40S/L4/A10 as inference workhorses; A100/H100/H200 for training and large models
- MIG partitioning on datacentre cards versus no partitioning on consumer RTX
Inside this lesson
A narrated, slide-by-slide presentation with 1 prediction exercise, 1 knowledge check, 2 videos. You answer as you go, and the platform checks your work.