OPEN-WEIGHT MODELS · UPDATED NOVEMBER 2026
Best local LLMs for AI agentsPicks by GPU size, with scores, VRAM and license
An agent model has to call tools reliably, browse, and keep going over many steps. These are our picks for each size of hardware, with the agent scores each developer publishes, the GPU memory each model needs and its license.
Our picks by hardware size
ONE GPU · 32 TO 94 GB
Qwen3.8-27B
Qwen · About 24 GB · Apache 2.0
84.3% on OSWorld-Verified, which measures operating a desktop, and 64.8% on WebArena-Verified, in about 24 GB of memory. Well ahead of Qwen3.6-27B (63.9% and 48.8%), under Apache 2.0.
Qwen3.8-27B hardware requirements →TWO GPUS · ABOUT 190 GB
MiMo-V2.6-Flash
Xiaomi · About 188 GB · MIT
73.6% on Toolathlon Verified and 80.8% on OSWorld-Verified, level with GLM-5.3 on tool use in well under half the memory. MIT license, and it reads screens, video and audio.
ONE 8-GPU SERVER · ABOUT 603 GB
MiMo-V2.6-Pro
Xiaomi · About 603 GB · MIT
76.9% on Toolathlon Verified, the highest of any open-weight model listed here, and 82.0% on OSWorld-Verified. MIT license. GLM-5.3 (73.0%) is the text-only alternative that fits four H200 GPUs.
MORE THAN ONE SERVER · ABOUT 1.64 TB
Kimi K3.1 Pro
Moonshot AI · About 1.64 TB · Kimi K3.1 Pro License
92.1% on BrowseComp, the highest open-weight score listed here, and 79.6% on the newer OSWorld 2.1. Check its license before commercial use.
Choosing a model for your team?
Tell us your users, your work and your data. We will recommend a model and the hardware to run it inside your own building with Vanik Edge OS. Free, no obligation.
Scores and hardware, model by model
Scores are the ones each developer reports on its own model card. Where a card lists several reasoning settings, the highest is shown. Developers use different test harnesses, so compare models within a column loosely. A dash means the developer has not published that figure.
| Model | BrowseComp | Toolathlon | MCP-Atlas | Other agent test | Memory, 1 user | Smallest setup for 16 users | License |
|---|---|---|---|---|---|---|---|
| Qwen3.6-27B Qwen model card | — | — | — | Claw-Eval 72.4% | About 20 GB | 1 NVIDIA L40S | Apache 2.0 |
| Gemma 4 31B Google model card | — | — | — | τ²-bench 76.9% | About 20 GB | 1 NVIDIA L40S | Apache 2.0 |
| Qwen3.8-27B Qwen model card | — | — | — | OSWorld-Verified 84.3% | About 24 GB | 1 NVIDIA L40S | Apache 2.0 |
| Qwen3.6-35B-A3B Qwen model card | — | — | — | Claw-Eval 68.7% | About 24 GB | 1 NVIDIA L40S | Apache 2.0 |
| Nemotron 3.5 Super VL NVIDIA model card | 77.5% | — | — | OSWorld-Verified 52.3% | About 80 GB | 1 NVIDIA RTX PRO 6000 Blackwell | OpenMDW 1.1 |
| MiniMax-M2.5 MiniMax model card | 76.3% | — | — | — | About 135 GB | 2 × NVIDIA H100 NVL | Modified MIT |
| DeepSeek-V4-Flash DeepSeek model card | 73.2% | 47.8% | 69.0% | — | About 177 GB | 2 × NVIDIA DGX Spark | MIT |
| MiMo-V2.6-Flash Xiaomi model card | — | 73.6% (Verified) | — | OSWorld-Verified 80.8% | About 188 GB | 2 × NVIDIA DGX Spark | MIT |
| GLM-5.2 Z.ai model card | — | 59.9% (Verified) | 76.8% | — | About 455 GB | 4 × NVIDIA H200 NVL | MIT |
| GLM-5.3 Z.ai model card | — | 73.0% (Verified) | — | — | About 460 GB | 4 × NVIDIA H200 NVL | GLM-5.3 License (MIT-like) |
| MiMo-V2.6-Pro Xiaomi model card | — | 76.9% (Verified) | — | OSWorld-Verified 82.0% | About 603 GB | 8 × NVIDIA H100 NVL | MIT |
| Kimi K2.6 Moonshot AI model card | 83.2% | 50.0% | — | OSWorld-Verified 73.1% | About 626 GB | 8 × NVIDIA H100 NVL | Modified MIT |
| DeepSeek-V4-Pro DeepSeek model card | 83.4% | 51.8% | 73.6% | — | About 976 GB | 8 × NVIDIA H200 NVL | MIT |
| Kimi K3 Moonshot AI model card | 91.2% | 76.5% (Verified) | 84.2% | OSWorld-Verified 84.8% | About 1.62 TB | 8 × AMD Instinct MI325X | Kimi K3 License |
| Kimi K3.1 Pro Moonshot AI model card | 92.1% | — | — | OSWorld 2.1 79.6% | About 1.64 TB | 8 × AMD Instinct MI325X | Kimi K3.1 Pro License |
Most of these models need several GPUs working as one. Buying the GPUs is the easy part. Vanik Edge OS splits the model across them, keeps it running if a server fails and manages it all from one console. See how Edge OS clusters GPU servers
What the benchmarks measure
BrowseComp. Finding hard-to-locate facts by browsing the web over many steps. A test of search and persistence.
Toolathlon. Long tasks that need many tools across real applications. Toolathlon Verified is a re-checked version, and the two are not directly comparable.
MCP-Atlas. Using real MCP servers and their tools to complete multi-step tasks.
Other agent tests. OSWorld-Verified measures operating a real desktop by looking at the screen. τ²-bench measures customer-service conversations that need tools and policies. Claw-Eval is an agent test Qwen reports.
FROM SIZING TO RUNNING
Sizing the hardware is the easy part.
Getting several GPUs to work as one system for a whole company is where most projects slow down. That is the part Vanik Edge OS does for you.
STEP 01 · THIS TOOL
Size it
Work out the GPU memory your models need and which GPUs or servers can hold them.
STEP 02 · VANIK EDGE OS
Cluster it
Edge OS joins your GPU servers into one cluster, splits a large model across them and keeps it running if a server fails. NVIDIA and AMD servers work side by side.
STEP 03 · VANIK EDGE OS
Run it
Your team gets AI apps with single sign-on and an audit trail, managed from one console, with your data staying inside your building.
Best local LLMs for AI agents: questions
What is the best local LLM for AI agents?
On a single GPU, Qwen3.8-27B. On two GPUs, MiMo-V2.6-Flash at 73.6% on Toolathlon Verified. On one 8-GPU server, MiMo-V2.6-Pro at 76.9%. With more than one server, Kimi K3.1 Pro.
Which local LLM is best for tool calling?
Of the models with published tool-use scores, MiMo-V2.6-Pro leads on Toolathlon Verified (76.9%), just ahead of Kimi K3 (76.5%), which also leads on MCP-Atlas (84.2%). MiMo-V2.6-Flash scores 73.6% in about 188 GB.
Can a local model operate a computer?
Yes, if it accepts images. On OSWorld-Verified, which measures operating a desktop from screenshots, Kimi K3 scores 84.8%, Qwen3.8-27B 84.3%, MiMo-V2.6-Pro 82.0% and MiMo-V2.6-Flash 80.8%. Text-only models such as GLM-5.3 cannot read a screen.
How much VRAM does an agent model need?
More than a chat model of the same size. Agents carry long histories of tool calls and results, so each running agent needs a large KV cache. Plan memory for the context your agents will really use and for how many run at once.
Why run agents on your own hardware?
Agents read internal systems, files and customer records, and act on them. Running the model inside your own building keeps that data and those actions off third-party servers.
Where do these scores come from?
Each developer’s own model card, linked in the table. We did not run the benchmarks ourselves, and developers use different test harnesses, so small differences between models are not meaningful.
