EDGE OSOPENAI · 21B PARAMETERS
gpt-oss-20b hardware and VRAM requirements
OpenAI’s smaller open-weight reasoning model, built to run on a single 16 GB-class GPU. gpt-oss-20b is a mixture-of-experts model with 3.6B of its 21B parameters work on each token. At MXFP4 it needs about 15 GB of GPU memory for one user, and about 18 GB to serve a team of 16 with 8K context each.
Total parameters
21B
Active per token
3.6B
Context length
128K
Released as
MXFP4 (native)
Minimum VRAM
15 GB at MXFP4
For a team of 16
18 GB at MXFP4
VRAM needed to run gpt-oss-20b
GPU memory by precision, number of concurrent users and context length. Each user needs their own KV cache, so memory grows with users and context.
| Precision | Weights only | 1 user, 8K | 16 users, 8K | 16 users, 32K | LoRA fine-tune |
|---|---|---|---|---|---|
| MXFP4 (native) Released | 13 GB | 15 GB | 18 GB | 28 GB | 18 GB |
Full fine-tuning in mixed precision needs about 339 GB: weights, gradients and optimizer states at roughly 16 bytes per parameter. LoRA trains small adapter layers on a frozen copy of the model instead, so it needs far less.
GPUs that can run gpt-oss-20b on-premise
For a team of 16 using gpt-oss-20b at MXFP4 with 8K context each (about 18 GB), smallest fit first.
NVIDIA L40S
48 GB per GPU · 25 GB to spare.
NVIDIA H100 NVL
94 GB per GPU · 67 GB to spare.
NVIDIA RTX PRO 6000 Blackwell
96 GB per GPU · 68 GB to spare.
NVIDIA DGX Spark
128 GB per system · 91 GB to spare.
NVIDIA H200 NVL
141 GB per GPU · 109 GB to spare.
AMD Instinct MI300X
192 GB per GPU · 155 GB to spare.
AMD Instinct MI325X
256 GB per GPU · 212 GB to spare.
NVIDIA DGX B200
1.44 TB per system · 1.28 TB to spare.
Weighing two of these? Compare AI hardware side by side, for example H100 vs H200, RTX PRO 6000 vs RTX 5090 or DGX Spark vs Mac Studio.
Running gpt-oss-20b for your team?
We will size the GPU server for your users and data, and set up gpt-oss-20b on it inside your own building. Free, no obligation.
gpt-oss-20b at a glance
Compare gpt-oss-20b with other models
FROM SIZING TO RUNNING
Sizing the hardware is the easy part.
Getting several GPUs to work as one system for a whole company is where most projects slow down. That is the part Vanik Edge OS does for you.
STEP 01 · THIS TOOL
Size it
Work out the GPU memory your models need and which GPUs or servers can hold them.
STEP 02 · VANIK EDGE OS
Cluster it
Edge OS joins your GPU servers into one cluster, splits a large model across them and keeps it running if a server fails. NVIDIA and AMD servers work side by side.
STEP 03 · VANIK EDGE OS
Run it
Your team gets AI apps with single sign-on and an audit trail, managed from one console, with your data staying inside your building.
gpt-oss-20b GPU questions
What are the gpt-oss-20b requirements?
To run gpt-oss-20b locally you need about 15 GB of GPU memory at MXFP4 for one user (18 GB for a team of 16), system RAM of at least 13 GB so the weights can be loaded, around 13 GB of fast NVMe storage for the model files, and an inference engine such as vLLM or SGLang. A team of 16 fits on NVIDIA L40S.
How much VRAM to run gpt-oss-20b?
gpt-oss-20b needs about 15 GB of GPU memory at MXFP4 for one user with 8K context: 13 GB for the weights plus the KV cache and runtime overhead. Serving 16 people at once with 8K context each needs about 18 GB.
Can gpt-oss-20b run on an RTX 4090?
Yes. An RTX 4090 has 24 GB of VRAM, and gpt-oss-20b needs about 15 GB at MXFP4 for one user, leaving room for longer context. A 10 GB card such as the RTX 3080 is too small to hold the whole model.
How do I fine-tune gpt-oss-20b?
LoRA fine-tuning at MXFP4 needs roughly 18 GB with 4K sequences; it trains small adapter layers on a frozen copy of the model. Full fine-tuning needs about 339 GB, which calls for a multi-GPU training cluster.
Why does gpt-oss-20b need so much memory if only 3.6B parameters are active?
It is a mixture-of-experts model. Only 3.6B parameters work on each token, which keeps it fast, but a different set of experts is picked for every token, so all 21B parameters must stay loaded in GPU memory.
How are these numbers calculated?
Weights are parameters × bytes per parameter. The KV cache uses 24.0 KB per token per user. We add about 5% of the weights plus 1 GB for runtime overhead and keep 10% of each GPU free. Figures are planning estimates; real usage depends on the inference engine and settings.
VRAM for similar models
Specifications from OpenAI: gpt-oss model card. Memory and bandwidth are manufacturer figures; GPU counts are rounded to 1, 2, 4 or 8 per server. All figures are planning estimates. Last checked November 10, 2026.
Run gpt-oss-20b inside your own building
Vanik Edge OS runs open-weight models on NVIDIA DGX, AMD Instinct and other GPU servers on your premises, so your data never leaves.