OPEN-WEIGHT MODELS · UPDATED NOVEMBER 2026
Best local LLMs for document AIPicks by GPU size, with scores, VRAM and license
Document AI means reading PDFs, scans, tables, forms and charts and turning them into text and answers. The best open-weight models for it fit on a single GPU. These are our picks, with the document scores each developer publishes, the GPU memory each model needs and its license.
Our picks by hardware size
ONE GPU · 32 TO 94 GB
Qwen3.8-27B
Qwen · About 24 GB · Apache 2.0
91.1% on OmniDocBench 1.5 and 83.7% on CharXiv chart reasoning in about 24 GB of memory, under Apache 2.0. Its document score is level with the figure published for Kimi K3, in a small fraction of the memory.
Qwen3.8-27B hardware requirements →ONE GPU, MANY DOCUMENTS AT ONCE · ABOUT 24 GB
Qwen3.6-35B-A3B
Qwen · About 24 GB · Apache 2.0
89.9% on OmniDocBench 1.5, 81.9% on CC-OCR and 90.0% on OCRBench. Only 3B parameters are active per token, so it works through large batches of pages faster.
Qwen3.6-35B-A3B hardware requirements →MORE THAN ONE SERVER · ABOUT 1.64 TB
Kimi K3.1 Pro
Moonshot AI · About 1.64 TB · Kimi K3.1 Pro License
67.4% on OfficeQA Pro, which tests questions across office files, and 89.0% on CharXiv, with a 1M-token context. Worth it only when the model must reason across very long documents; for reading pages, the small models match the largest.
Choosing a model for your team?
Tell us your users, your work and your data. We will recommend a model and the hardware to run it inside your own building with Vanik Edge OS. Free, no obligation.
Scores and hardware, model by model
Scores are the ones each developer reports on its own model card, without extra tools. Developers use different test settings, so compare models within a column loosely. A dash means the developer has not published that figure.
| Model | OmniDocBench | CC-OCR | OCRBench | CharXiv (RQ) | Memory, 1 user | Smallest setup for 16 users | License |
|---|---|---|---|---|---|---|---|
| Qwen3.6-27B Qwen model card | 89.4% (v1.5) | 81.2% | 89.4% | 78.4% | About 20 GB | 1 NVIDIA L40S | Apache 2.0 |
| Gemma 4 31B Google model card | 0.131 edit distance (v1.5) | — | — | — | About 20 GB | 1 NVIDIA L40S | Apache 2.0 |
| Qwen3.8-27B Qwen model card | 91.1% (v1.5) | — | — | 83.7% | About 24 GB | 1 NVIDIA L40S | Apache 2.0 |
| Qwen3.6-35B-A3B Qwen model card | 89.9% (v1.5) | 81.9% | 90.0% | 78.0% | About 24 GB | 1 NVIDIA L40S | Apache 2.0 |
| Nemotron 3.5 Super VL NVIDIA model card | — | — | 68.3% (v2, English) | 79.9% | About 80 GB | 1 NVIDIA RTX PRO 6000 Blackwell | OpenMDW 1.1 |
| Kimi K2.6 Moonshot AI model card | — | — | — | 80.4% | About 626 GB | 8 × NVIDIA H100 NVL | Modified MIT |
| Kimi K3 Moonshot AI model card | 91.1% | — | — | 84.8% | About 1.62 TB | 8 × AMD Instinct MI325X | Kimi K3 License |
| Kimi K3.1 Pro Moonshot AI model card | — | — | — | 89.0% | About 1.64 TB | 8 × AMD Instinct MI325X | Kimi K3.1 Pro License |
Most of these models need several GPUs working as one. Buying the GPUs is the easy part. Vanik Edge OS splits the model across them, keeps it running if a server fails and manages it all from one console. See how Edge OS clusters GPU servers
What the benchmarks measure
OmniDocBench. Turning PDF pages into text, tables, formulas and the right reading order. Google reports it as an edit distance, where lower is better, so Gemma’s figure cannot be compared with the percentages.
CC-OCR. Reading text in varied settings: street scenes, multilingual text, document parsing and pulling named fields out of forms.
OCRBench. Recognizing and understanding text in images, including handwriting and formulas.
CharXiv (RQ). Reasoning questions about charts taken from research papers.
FROM SIZING TO RUNNING
Sizing the hardware is the easy part.
Getting several GPUs to work as one system for a whole company is where most projects slow down. That is the part Vanik Edge OS does for you.
STEP 01 · THIS TOOL
Size it
Work out the GPU memory your models need and which GPUs or servers can hold them.
STEP 02 · VANIK EDGE OS
Cluster it
Edge OS joins your GPU servers into one cluster, splits a large model across them and keeps it running if a server fails. NVIDIA and AMD servers work side by side.
STEP 03 · VANIK EDGE OS
Run it
Your team gets AI apps with single sign-on and an audit trail, managed from one console, with your data staying inside your building.
Best local LLMs for document AI: questions
What is the best local LLM for document AI?
Qwen3.8-27B: 91.1% on OmniDocBench 1.5 in about 24 GB of GPU memory, under Apache 2.0. For large batches of pages, Qwen3.6-35B-A3B is faster and scores 89.9%.
Which local LLM is best for OCR?
Of the models with published OCR scores, Qwen3.6-35B-A3B leads on both CC-OCR (81.9%) and OCRBench (90.0%), just ahead of Qwen3.6-27B. Qwen3.8-27B has the highest document-parsing score but its card does not publish these two tests.
Do I need a large model to read documents?
No. On OmniDocBench, Qwen3.8-27B in about 24 GB matches the figure Kimi K3 publishes in about 1.62 TB. A large model only helps when it must reason across hundreds of pages at once or do other work as well.
How much VRAM does document processing need?
More than the model’s base figure. Each page image becomes hundreds to thousands of tokens, so a 50-page PDF can fill a long context and a large KV cache. Plan memory for pages per request and for how many requests run at once.
Why run document AI on your own hardware?
Contracts, invoices, medical records and identity documents are the files companies least want to send to a third party. A model running inside your own building reads them without the pages leaving your network.
Where do these scores come from?
Each developer’s own model card, linked in the table. We did not run the benchmarks ourselves, and developers use different test settings, so small differences between models are not meaningful.
