OPEN-WEIGHT MODELS · UPDATED NOVEMBER 2026
Best local LLMsThe best open-source models to run on your own hardware
The best local LLM is the strongest one your GPUs can hold. These are our picks for each size of hardware, from one workstation GPU to a multi-server cluster, with the GPU memory each model needs, the smallest setup that runs it and its license.
Best local LLM for each size of hardware
ONE GPU · 32 TO 94 GB
Qwen3.8-27B
Qwen · About 24 GB · Apache 2.0
About 24 GB at NVFP4, Apache 2.0, and ahead of Qwen3.6-27B on every test on its model card. It reads images and video as well as text. Nothing larger that fits one GPU clearly beats it, so spend spare memory on more users and longer context.
Qwen3.8-27B hardware requirements →ALSO CONSIDER
- Qwen3.6-27B: about 20 GB, so it fits a 24 GB card
- Gemma 4 31B: Google’s model, with image input
- Nemotron 3.5 Super VL: about 80 GB, with a 1M-token context
TWO GPUS · ABOUT 190 GB
MiMo-V2.6-Flash
Xiaomi · About 188 GB · MIT
87.6% on Terminal-Bench 2.1 and 73.6% on Toolathlon Verified, level with models more than twice its size. MIT license, with image, video and audio input.
ALSO CONSIDER
- MiniMax-M2.5: about 135 GB, at 80.2% on SWE-bench Verified
- DeepSeek-V4-Flash: about 177 GB, the strongest reasoning scores at this size
- GLM-5.3-Flash: MIT license, with image and video input
ONE 8-GPU SERVER · 450 TO 630 GB
DeepSeek-V4.1-Flash-1017
DeepSeek · About 537 GB · MIT
77.0% on DeepSWE, the highest coding score of any open-weight model listed here, under the MIT license.
ALSO CONSIDER
- MiMo-V2.6-Pro: the highest tool-use score, with audio input
- GLM-5.3: about 460 GB, so it fits four H200 GPUs
- GLM-5.2: the highest math scores, under plain MIT
MORE THAN ONE SERVER · OVER 1 TB
Kimi K3.1 Pro
Moonshot AI · About 1.64 TB · Kimi K3.1 Pro License
About 1.64 TB for one user. The highest published scores of any open-weight model on browsing, science questions and chart reading. Check its license before commercial use.
ALSO CONSIDER
- Kimi K3: the earlier release, which also takes video
- DeepSeek-V4-Pro: about 976 GB under MIT, so it still fits one 8-GPU H200 server
These are 15 of the 19 models we compare. See all 19 side by side ↓
Not sure which model fits your team?
Tell us your use case, users and data. We will recommend a model and the hardware to run it inside your own building with Vanik Edge OS. Free, no obligation.
The best open-source LLMs to run locally, compared
Memory for one user with 8K context at the lowest released precision, smallest model first. Open a model for VRAM by precision, team sizes and fine-tuning.
| Model | Best for | Parameters | Memory, 1 user | Smallest setup for 16 users | License |
|---|---|---|---|---|---|
| gpt-oss-20b OpenAI | The smallest footprint: laptops, small GPUs and many users on one card | 21B | About 15 GB | 1 NVIDIA L40S | Apache 2.0 |
| Qwen3.6-27B Qwen | The same work as Qwen3.8-27B in a little less memory | 27.78B | About 20 GB | 1 NVIDIA L40S | Apache 2.0 |
| Gemma 4 31B | Image and document understanding on a single GPU, from Google | 30.7B | About 20 GB | 1 NVIDIA L40S | Apache 2.0 |
| Qwen3.8-27B QwenOur pick | Coding, agents and documents on a single GPU, with image and video input | 27.8B | About 24 GB | 1 NVIDIA L40S | Apache 2.0 |
| Qwen3.6-35B-A3B Qwen | Fast replies for many users on a single GPU | 35B (3B active) | About 24 GB | 1 NVIDIA L40S | Apache 2.0 |
| gpt-oss-120b OpenAI | Text reasoning on one 80 GB-class GPU | 117B (5.1B active) | About 70 GB | 1 NVIDIA H100 NVL | Apache 2.0 |
| Nemotron 3.5 Super VL NVIDIA | A 1M-token context with image and video input on one GPU | 120B (12B active) | About 80 GB | 1 NVIDIA RTX PRO 6000 Blackwell | OpenMDW 1.1 |
| MiniMax-M2.5 MiniMax | Coding and agent work on two GPUs | 229B | About 135 GB | 2 × NVIDIA H100 NVL | Modified MIT |
| DeepSeek-V4-Flash DeepSeek | Reasoning with a 1M-token context under the MIT license on modest hardware | 284B (13B active) | About 177 GB | 2 × NVIDIA DGX Spark | MIT |
| MiMo-V2.6-Flash XiaomiOur pick | Coding and agents on two GPUs, with image, video and audio input | 309B (15B active) | About 188 GB | 2 × NVIDIA DGX Spark | MIT |
| GLM-5.3-Flash Z.ai | Image and video input with long documents | 320B (18B active) | About 192 GB | 2 × NVIDIA DGX Spark | MIT |
| GLM-5.2 Z.ai | A capable coding model under the plain MIT license | 744B (40B active) | About 455 GB | 4 × NVIDIA H200 NVL | MIT |
| GLM-5.3 Z.ai | A strong coding and agent model that fits four H200 GPUs | 753B (40B active) | About 460 GB | 4 × NVIDIA H200 NVL | GLM-5.3 License (MIT-like) |
| DeepSeek-V4.1-Flash-1017 DeepSeekOur pick | The strongest coding model that fits one 8-GPU server | 552B (16B active) | About 537 GB | 8 × NVIDIA H100 NVL | MIT |
| MiMo-V2.6-Pro Xiaomi | Tool use and computer use on one 8-GPU server, with image, video and audio input | 1.02T (42B active) | About 603 GB | 8 × NVIDIA H100 NVL | MIT |
| Kimi K2.6 Moonshot AI | A general assistant and agent model on one 8-GPU server | 1T (32B active) | About 626 GB | 8 × NVIDIA H100 NVL | Modified MIT |
| DeepSeek-V4-Pro DeepSeek | DeepSeek’s largest model with a 1M-token context | 1.6T (49B active) | About 976 GB | 8 × NVIDIA H200 NVL | MIT |
| Kimi K3 Moonshot AI | The earlier Kimi K3 release, with image and video input | 2.8T (104B active) | About 1.62 TB | 8 × AMD Instinct MI325X | Kimi K3 License |
| Kimi K3.1 Pro Moonshot AIOur pick | The strongest open-weight model, with image input | 2.8T (104B active) | About 1.64 TB | 8 × AMD Instinct MI325X | Kimi K3.1 Pro License |
Most of these models need several GPUs working as one. Buying the GPUs is the easy part. Vanik Edge OS splits the model across them, keeps it running if a server fails and manages it all from one console. See how Edge OS clusters GPU servers
How we picked
Every model here has weights you can download and run yourself. Picks weigh four things: the GPU memory a model needs, its license, what it accepts as input, and the benchmark figures its developer publishes on the model card. We did not run our own benchmarks, and a model your team has tested on its own work beats any ranking.
FROM SIZING TO RUNNING
Sizing the hardware is the easy part.
Getting several GPUs to work as one system for a whole company is where most projects slow down. That is the part Vanik Edge OS does for you.
STEP 01 · THIS TOOL
Size it
Work out the GPU memory your models need and which GPUs or servers can hold them.
STEP 02 · VANIK EDGE OS
Cluster it
Edge OS joins your GPU servers into one cluster, splits a large model across them and keeps it running if a server fails. NVIDIA and AMD servers work side by side.
STEP 03 · VANIK EDGE OS
Run it
Your team gets AI apps with single sign-on and an audit trail, managed from one console, with your data staying inside your building.
Local LLM questions
What is the best local LLM right now?
It depends on the GPU memory you have. On one GPU, Qwen3.8-27B is our pick. On two GPUs, MiMo-V2.6-Flash. On one 8-GPU server, DeepSeek-V4.1-Flash-1017. With more than one server, Kimi K3.1 Pro is the strongest open-weight model you can run.
What is a local LLM?
A language model whose weights you download and run on your own hardware, so prompts and documents never leave your building. Open-weight models such as Qwen, Gemma, gpt-oss, DeepSeek, GLM and Kimi can all be run this way.
How much VRAM do I need to run a local LLM?
From about 15 GB for gpt-oss-20b to about 1.64 TB for Kimi K3.1 Pro, for one user with 8K context. Every extra user needs their own KV cache, so memory grows with users and context length. The VRAM calculator works it out for your settings.
Can I use these models commercially?
Most of them, yes. Apache 2.0 and MIT allow commercial use. Modified MIT and vendor licenses add conditions, so read the license on the model card before a commercial deployment.
What happens when a model does not fit on one GPU?
It has to be split across several GPUs, and for the largest models across several servers. Buying the GPUs is the easy part; Vanik Edge OS splits the model across them and keeps it running if a server fails.
How were these models picked?
Only models with downloadable weights are listed. Picks weigh the GPU memory a model needs, its license, what it accepts as input and the benchmark figures its developer publishes. We did not run our own benchmarks.
