EDGE OSDEFINITION
VRAM (video random access memory) is the memory built into a GPU. In AI it holds the model weights, the KV cache and working data during inference or training. A model runs at full speed only when everything it needs fits in VRAM. Data center GPUs use high-bandwidth memory (HBM), with capacities from tens to hundreds of gigabytes per GPU.
Planning hardware for AI on your own premises? Work out the GPU memory a model needs in our VRAM calculator, or let us size it with you.
Get a free sizing checkWhy VRAM Matters
- Model size: Sets the largest model a GPU or server can run.
- Speed: Memory bandwidth largely decides how fast tokens are generated.
- Users: VRAM left after loading the weights decides how many people can be served at once.
- Cost: Is usually the main factor when choosing GPU hardware for AI.
Use Cases
- Choosing GPUs for on-premise AI servers.
- Deciding model size and quantization level.
- Planning for concurrent users and long context.
- Estimating the hardware needed for fine-tuning.
See it running on your own hardware
Vanik Edge OS runs AI models, agents and apps inside your building, so your data stays there.