Choosing a local LLM for AI agents? Get a free sizing check.Get a sizing check

OPEN-WEIGHT MODELS · UPDATED NOVEMBER 2026

Best local LLMs for AI agentsPicks by GPU size, with scores, VRAM and license

An agent model has to call tools reliably, browse, and keep going over many steps. These are our picks for each size of hardware, with the agent scores each developer publishes, the GPU memory each model needs and its license.

Our picks by hardware size

ONE GPU · 32 TO 94 GB

Qwen3.8-27B

Qwen · About 24 GB · Apache 2.0

84.3% on OSWorld-Verified, which measures operating a desktop, and 64.8% on WebArena-Verified, in about 24 GB of memory. Well ahead of Qwen3.6-27B (63.9% and 48.8%), under Apache 2.0.

Qwen3.8-27B hardware requirements →

TWO GPUS · ABOUT 190 GB

MiMo-V2.6-Flash

Xiaomi · About 188 GB · MIT

73.6% on Toolathlon Verified and 80.8% on OSWorld-Verified, level with GLM-5.3 on tool use in well under half the memory. MIT license, and it reads screens, video and audio.

ONE 8-GPU SERVER · ABOUT 603 GB

MiMo-V2.6-Pro

Xiaomi · About 603 GB · MIT

76.9% on Toolathlon Verified, the highest of any open-weight model listed here, and 82.0% on OSWorld-Verified. MIT license. GLM-5.3 (73.0%) is the text-only alternative that fits four H200 GPUs.

MORE THAN ONE SERVER · ABOUT 1.64 TB

Kimi K3.1 Pro

Moonshot AI · About 1.64 TB · Kimi K3.1 Pro License

92.1% on BrowseComp, the highest open-weight score listed here, and 79.6% on the newer OSWorld 2.1. Check its license before commercial use.

Choosing a model for your team?

Tell us your users, your work and your data. We will recommend a model and the hardware to run it inside your own building with Vanik Edge OS. Free, no obligation.

Get a free sizing check

Scores and hardware, model by model

Scores are the ones each developer reports on its own model card. Where a card lists several reasoning settings, the highest is shown. Developers use different test harnesses, so compare models within a column loosely. A dash means the developer has not published that figure.

ModelBrowseCompToolathlonMCP-AtlasOther agent testMemory, 1 userSmallest setup for 16 usersLicense
Qwen3.6-27B
Qwen model card
———Claw-Eval 72.4%About 20 GB1 NVIDIA L40SApache 2.0
Gemma 4 31B
Google model card
———τ²-bench 76.9%About 20 GB1 NVIDIA L40SApache 2.0
Qwen3.8-27B
Qwen model card
———OSWorld-Verified 84.3%About 24 GB1 NVIDIA L40SApache 2.0
Qwen3.6-35B-A3B
Qwen model card
———Claw-Eval 68.7%About 24 GB1 NVIDIA L40SApache 2.0
Nemotron 3.5 Super VL
NVIDIA model card
77.5%——OSWorld-Verified 52.3%About 80 GB1 NVIDIA RTX PRO 6000 BlackwellOpenMDW 1.1
MiniMax-M2.5
MiniMax model card
76.3%———About 135 GB2 × NVIDIA H100 NVLModified MIT
DeepSeek-V4-Flash
DeepSeek model card
73.2%47.8%69.0%—About 177 GB2 × NVIDIA DGX SparkMIT
MiMo-V2.6-Flash
Xiaomi model card
—73.6% (Verified)—OSWorld-Verified 80.8%About 188 GB2 × NVIDIA DGX SparkMIT
GLM-5.2
Z.ai model card
—59.9% (Verified)76.8%—About 455 GB4 × NVIDIA H200 NVLMIT
GLM-5.3
Z.ai model card
—73.0% (Verified)——About 460 GB4 × NVIDIA H200 NVLGLM-5.3 License (MIT-like)
MiMo-V2.6-Pro
Xiaomi model card
—76.9% (Verified)—OSWorld-Verified 82.0%About 603 GB8 × NVIDIA H100 NVLMIT
Kimi K2.6
Moonshot AI model card
83.2%50.0%—OSWorld-Verified 73.1%About 626 GB8 × NVIDIA H100 NVLModified MIT
DeepSeek-V4-Pro
DeepSeek model card
83.4%51.8%73.6%—About 976 GB8 × NVIDIA H200 NVLMIT
Kimi K3
Moonshot AI model card
91.2%76.5% (Verified)84.2%OSWorld-Verified 84.8%About 1.62 TB8 × AMD Instinct MI325XKimi K3 License
Kimi K3.1 Pro
Moonshot AI model card
92.1%——OSWorld 2.1 79.6%About 1.64 TB8 × AMD Instinct MI325XKimi K3.1 Pro License

Most of these models need several GPUs working as one. Buying the GPUs is the easy part. Vanik Edge OS splits the model across them, keeps it running if a server fails and manages it all from one console. See how Edge OS clusters GPU servers

What the benchmarks measure

BrowseComp. Finding hard-to-locate facts by browsing the web over many steps. A test of search and persistence.

Toolathlon. Long tasks that need many tools across real applications. Toolathlon Verified is a re-checked version, and the two are not directly comparable.

MCP-Atlas. Using real MCP servers and their tools to complete multi-step tasks.

Other agent tests. OSWorld-Verified measures operating a real desktop by looking at the screen. τ²-bench measures customer-service conversations that need tools and policies. Claw-Eval is an agent test Qwen reports.

FROM SIZING TO RUNNING

Sizing the hardware is the easy part.

Getting several GPUs to work as one system for a whole company is where most projects slow down. That is the part Vanik Edge OS does for you.

  1. STEP 01 · THIS TOOL

    Size it

    Work out the GPU memory your models need and which GPUs or servers can hold them.

  2. STEP 02 · VANIK EDGE OS

    Cluster it

    Edge OS joins your GPU servers into one cluster, splits a large model across them and keeps it running if a server fails. NVIDIA and AMD servers work side by side.

  3. STEP 03 · VANIK EDGE OS

    Run it

    Your team gets AI apps with single sign-on and an audit trail, managed from one console, with your data staying inside your building.

See how Edge OS clusters GPU servers

Best local LLMs for AI agents: questions

What is the best local LLM for AI agents?

On a single GPU, Qwen3.8-27B. On two GPUs, MiMo-V2.6-Flash at 73.6% on Toolathlon Verified. On one 8-GPU server, MiMo-V2.6-Pro at 76.9%. With more than one server, Kimi K3.1 Pro.

Which local LLM is best for tool calling?

Of the models with published tool-use scores, MiMo-V2.6-Pro leads on Toolathlon Verified (76.9%), just ahead of Kimi K3 (76.5%), which also leads on MCP-Atlas (84.2%). MiMo-V2.6-Flash scores 73.6% in about 188 GB.

Can a local model operate a computer?

Yes, if it accepts images. On OSWorld-Verified, which measures operating a desktop from screenshots, Kimi K3 scores 84.8%, Qwen3.8-27B 84.3%, MiMo-V2.6-Pro 82.0% and MiMo-V2.6-Flash 80.8%. Text-only models such as GLM-5.3 cannot read a screen.

How much VRAM does an agent model need?

More than a chat model of the same size. Agents carry long histories of tool calls and results, so each running agent needs a large KV cache. Plan memory for the context your agents will really use and for how many run at once.

Why run agents on your own hardware?

Agents read internal systems, files and customer records, and act on them. Running the model inside your own building keeps that data and those actions off third-party servers.

Where do these scores come from?

Each developer’s own model card, linked in the table. We did not run the benchmarks ourselves, and developers use different test harnesses, so small differences between models are not meaningful.