Choosing a local LLM? Get a free sizing check.Get a sizing check

OPEN-WEIGHT MODELS · UPDATED NOVEMBER 2026

Best local LLMsThe best open-source models to run on your own hardware

The best local LLM is the strongest one your GPUs can hold. These are our picks for each size of hardware, from one workstation GPU to a multi-server cluster, with the GPU memory each model needs, the smallest setup that runs it and its license.

Best local LLM for each size of hardware

ONE GPU · 32 TO 94 GB

Qwen3.8-27B

Qwen · About 24 GB · Apache 2.0

About 24 GB at NVFP4, Apache 2.0, and ahead of Qwen3.6-27B on every test on its model card. It reads images and video as well as text. Nothing larger that fits one GPU clearly beats it, so spend spare memory on more users and longer context.

Qwen3.8-27B hardware requirements →

ALSO CONSIDER

TWO GPUS · ABOUT 190 GB

MiMo-V2.6-Flash

Xiaomi · About 188 GB · MIT

87.6% on Terminal-Bench 2.1 and 73.6% on Toolathlon Verified, level with models more than twice its size. MIT license, with image, video and audio input.

ALSO CONSIDER

ONE 8-GPU SERVER · 450 TO 630 GB

DeepSeek-V4.1-Flash-1017

DeepSeek · About 537 GB · MIT

77.0% on DeepSWE, the highest coding score of any open-weight model listed here, under the MIT license.

ALSO CONSIDER

  • MiMo-V2.6-Pro: the highest tool-use score, with audio input
  • GLM-5.3: about 460 GB, so it fits four H200 GPUs
  • GLM-5.2: the highest math scores, under plain MIT

MORE THAN ONE SERVER · OVER 1 TB

Kimi K3.1 Pro

Moonshot AI · About 1.64 TB · Kimi K3.1 Pro License

About 1.64 TB for one user. The highest published scores of any open-weight model on browsing, science questions and chart reading. Check its license before commercial use.

ALSO CONSIDER

  • Kimi K3: the earlier release, which also takes video
  • DeepSeek-V4-Pro: about 976 GB under MIT, so it still fits one 8-GPU H200 server

These are 15 of the 19 models we compare. See all 19 side by side ↓

Not sure which model fits your team?

Tell us your use case, users and data. We will recommend a model and the hardware to run it inside your own building with Vanik Edge OS. Free, no obligation.

Get a free sizing check

The best open-source LLMs to run locally, compared

Memory for one user with 8K context at the lowest released precision, smallest model first. Open a model for VRAM by precision, team sizes and fine-tuning.

ModelBest forParametersMemory, 1 userSmallest setup for 16 usersLicense
gpt-oss-20b
OpenAI
The smallest footprint: laptops, small GPUs and many users on one card21BAbout 15 GB1 NVIDIA L40SApache 2.0
Qwen3.6-27B
Qwen
The same work as Qwen3.8-27B in a little less memory27.78BAbout 20 GB1 NVIDIA L40SApache 2.0
Gemma 4 31B
Google
Image and document understanding on a single GPU, from Google30.7BAbout 20 GB1 NVIDIA L40SApache 2.0
Qwen3.8-27B
QwenOur pick
Coding, agents and documents on a single GPU, with image and video input27.8BAbout 24 GB1 NVIDIA L40SApache 2.0
Qwen3.6-35B-A3B
Qwen
Fast replies for many users on a single GPU35B (3B active)About 24 GB1 NVIDIA L40SApache 2.0
gpt-oss-120b
OpenAI
Text reasoning on one 80 GB-class GPU117B (5.1B active)About 70 GB1 NVIDIA H100 NVLApache 2.0
Nemotron 3.5 Super VL
NVIDIA
A 1M-token context with image and video input on one GPU120B (12B active)About 80 GB1 NVIDIA RTX PRO 6000 BlackwellOpenMDW 1.1
MiniMax-M2.5
MiniMax
Coding and agent work on two GPUs229BAbout 135 GB2 × NVIDIA H100 NVLModified MIT
DeepSeek-V4-Flash
DeepSeek
Reasoning with a 1M-token context under the MIT license on modest hardware284B (13B active)About 177 GB2 × NVIDIA DGX SparkMIT
MiMo-V2.6-Flash
XiaomiOur pick
Coding and agents on two GPUs, with image, video and audio input309B (15B active)About 188 GB2 × NVIDIA DGX SparkMIT
GLM-5.3-Flash
Z.ai
Image and video input with long documents320B (18B active)About 192 GB2 × NVIDIA DGX SparkMIT
GLM-5.2
Z.ai
A capable coding model under the plain MIT license744B (40B active)About 455 GB4 × NVIDIA H200 NVLMIT
GLM-5.3
Z.ai
A strong coding and agent model that fits four H200 GPUs753B (40B active)About 460 GB4 × NVIDIA H200 NVLGLM-5.3 License (MIT-like)
DeepSeek-V4.1-Flash-1017
DeepSeekOur pick
The strongest coding model that fits one 8-GPU server552B (16B active)About 537 GB8 × NVIDIA H100 NVLMIT
MiMo-V2.6-Pro
Xiaomi
Tool use and computer use on one 8-GPU server, with image, video and audio input1.02T (42B active)About 603 GB8 × NVIDIA H100 NVLMIT
Kimi K2.6
Moonshot AI
A general assistant and agent model on one 8-GPU server1T (32B active)About 626 GB8 × NVIDIA H100 NVLModified MIT
DeepSeek-V4-Pro
DeepSeek
DeepSeek’s largest model with a 1M-token context1.6T (49B active)About 976 GB8 × NVIDIA H200 NVLMIT
Kimi K3
Moonshot AI
The earlier Kimi K3 release, with image and video input2.8T (104B active)About 1.62 TB8 × AMD Instinct MI325XKimi K3 License
Kimi K3.1 Pro
Moonshot AIOur pick
The strongest open-weight model, with image input2.8T (104B active)About 1.64 TB8 × AMD Instinct MI325XKimi K3.1 Pro License

Most of these models need several GPUs working as one. Buying the GPUs is the easy part. Vanik Edge OS splits the model across them, keeps it running if a server fails and manages it all from one console. See how Edge OS clusters GPU servers

How we picked

Every model here has weights you can download and run yourself. Picks weigh four things: the GPU memory a model needs, its license, what it accepts as input, and the benchmark figures its developer publishes on the model card. We did not run our own benchmarks, and a model your team has tested on its own work beats any ranking.

FROM SIZING TO RUNNING

Sizing the hardware is the easy part.

Getting several GPUs to work as one system for a whole company is where most projects slow down. That is the part Vanik Edge OS does for you.

  1. STEP 01 · THIS TOOL

    Size it

    Work out the GPU memory your models need and which GPUs or servers can hold them.

  2. STEP 02 · VANIK EDGE OS

    Cluster it

    Edge OS joins your GPU servers into one cluster, splits a large model across them and keeps it running if a server fails. NVIDIA and AMD servers work side by side.

  3. STEP 03 · VANIK EDGE OS

    Run it

    Your team gets AI apps with single sign-on and an audit trail, managed from one console, with your data staying inside your building.

See how Edge OS clusters GPU servers

Local LLM questions

What is the best local LLM right now?

It depends on the GPU memory you have. On one GPU, Qwen3.8-27B is our pick. On two GPUs, MiMo-V2.6-Flash. On one 8-GPU server, DeepSeek-V4.1-Flash-1017. With more than one server, Kimi K3.1 Pro is the strongest open-weight model you can run.

What is a local LLM?

A language model whose weights you download and run on your own hardware, so prompts and documents never leave your building. Open-weight models such as Qwen, Gemma, gpt-oss, DeepSeek, GLM and Kimi can all be run this way.

How much VRAM do I need to run a local LLM?

From about 15 GB for gpt-oss-20b to about 1.64 TB for Kimi K3.1 Pro, for one user with 8K context. Every extra user needs their own KV cache, so memory grows with users and context length. The VRAM calculator works it out for your settings.

Can I use these models commercially?

Most of them, yes. Apache 2.0 and MIT allow commercial use. Modified MIT and vendor licenses add conditions, so read the license on the model card before a commercial deployment.

What happens when a model does not fit on one GPU?

It has to be split across several GPUs, and for the largest models across several servers. Buying the GPUs is the easy part; Vanik Edge OS splits the model across them and keeps it running if a server fails.

How were these models picked?

Only models with downloadable weights are listed. Picks weigh the GPU memory a model needs, its license, what it accepts as input and the benchmark figures its developer publishes. We did not run our own benchmarks.