EDGE OSFREE TOOL
Open-weight LLM comparisonSpecs, VRAM and the hardware to run them
Compare the open-weight models people are choosing between for private, on-premise AI. Every comparison shows parameters, context, license, the GPU memory each model needs and which hardware can run it.
Popular open-weight model comparisons
Not sure which model fits your team?
Tell us your use case, users and data. We will recommend a model and the hardware to run it inside your own building with Vanik Edge OS. Free, no obligation.
Models at a glance
Memory for one user with 8K context at the lowest released precision. Open a model for VRAM by precision, team sizes and fine-tuning.
| Model | Parameters | Active | Context | Memory, 1 user | Smallest setup for 16 users |
|---|---|---|---|---|---|
| gpt-oss-20b OpenAI | 21B | 3.6B | 128K | About 15 GB | 1 NVIDIA L40S |
| Qwen3.6-27B Qwen | 27.78B | 27.78B (dense) | 262K native | About 20 GB | 1 NVIDIA L40S |
| Gemma 4 31B | 30.7B | 30.7B (dense) | 256K tokens | About 20 GB | 1 NVIDIA L40S |
| Qwen3.6-35B-A3B Qwen | 35B | 3B | 262K native | About 24 GB | 1 NVIDIA L40S |
| gpt-oss-120b OpenAI | 117B | 5.1B | 128K | About 70 GB | 1 NVIDIA H100 NVL |
| MiniMax-M2.5 MiniMax | 229B | 229B (dense) | Not stated on the model card | About 135 GB | 2 × NVIDIA H100 NVL |
| DeepSeek-V4-Flash DeepSeek | 284B | 13B | 1M tokens | About 177 GB | 2 × NVIDIA DGX Spark |
| MiMo-V2.5 Xiaomi | 310B | 15B | 1M tokens | About 190 GB | 2 × NVIDIA DGX Spark |
| GLM-5.3-Flash Z.ai | 320B | 18B | Up to 1M input tokens | About 192 GB | 2 × NVIDIA DGX Spark |
| MiniMax-M3 MiniMax | 427B | 23B | 1M tokens | About 277 GB | 4 × NVIDIA H100 NVL |
| DeepSeek-V3.2 DeepSeek | 685B | 37B | 128K | About 419 GB | 2 × AMD Instinct MI325X |
| GLM-5.2 Z.ai | 744B | 40B | 1M tokens | About 455 GB | 4 × NVIDIA H200 NVL |
| GLM-5.3 Z.ai | 753B | 40B | 1M tokens | About 460 GB | 4 × NVIDIA H200 NVL |
| GLM-5.1 Z.ai | 754B | 40B | Up to 200K tokens | About 461 GB | 4 × NVIDIA H200 NVL |
| Kimi K2.6 Moonshot AI | 1T | 32B | 256K tokens | About 626 GB | 8 × NVIDIA H100 NVL |
| Kimi K2.7 Code Moonshot AI | 1T | 32B | 256K tokens | About 626 GB | 8 × NVIDIA H100 NVL |
| Kimi K2.5 Moonshot AI | 1T | 32B | 256K tokens | About 626 GB | 8 × NVIDIA H100 NVL |
| DeepSeek-V4-Pro DeepSeek | 1.6T | 49B | 1M tokens | About 976 GB | 8 × NVIDIA H200 NVL |
| Kimi K3 Moonshot AI | 2.8T | 104B | 1M tokens | About 1.62 TB | 8 × AMD Instinct MI325X |
Most of these models need several GPUs working as one. Buying the GPUs is the easy part. Vanik Edge OS splits the model across them, keeps it running if a server fails and manages it all from one console. See how Edge OS clusters GPU servers
FROM SIZING TO RUNNING
Sizing the hardware is the easy part.
Getting several GPUs to work as one system for a whole company is where most projects slow down. That is the part Vanik Edge OS does for you.
STEP 01 · THIS TOOL
Size it
Work out the GPU memory your models need and which GPUs or servers can hold them.
STEP 02 · VANIK EDGE OS
Cluster it
Edge OS joins your GPU servers into one cluster, splits a large model across them and keeps it running if a server fails. NVIDIA and AMD servers work side by side.
STEP 03 · VANIK EDGE OS
Run it
Your team gets AI apps with single sign-on and an audit trail, managed from one console, with your data staying inside your building.
Open-weight LLM questions
What is an open-weight LLM?
An open-weight model is one whose trained weights you can download and run on your own hardware, such as DeepSeek, Kimi, GLM, Qwen, gpt-oss and Gemma. The license decides what you may do with it; most of the models here allow commercial use.
Which open-weight LLM needs the least GPU memory?
Of the models compared here, gpt-oss-20b needs the least: about 15 GB for one user with 8K context. Qwen3.6-35B-A3B and Gemma 4 31B also fit on a single workstation GPU at 4-bit.
Which open-weight LLM is the largest?
Kimi K3, with 2.8 trillion parameters and 104B active per token. It needs about 1.62 TB of GPU memory for one user, so it needs an 8-GPU AMD MI325X server or a multi-server NVIDIA cluster.
Why do mixture-of-experts models need so much memory if few parameters are active?
Every expert has to be loaded in GPU memory because any of them may be picked for the next token. Active parameters decide speed; total parameters decide memory.
How are these numbers calculated?
Weights at the lowest released precision, plus the KV cache for each user’s context, plus about 5 to 10% runtime overhead, keeping 10% of each GPU free. These are planning estimates; use the VRAM calculator for your own settings.