Choosing a local LLM for document AI? Get a free sizing check.Get a sizing check

OPEN-WEIGHT MODELS · UPDATED NOVEMBER 2026

Best local LLMs for document AIPicks by GPU size, with scores, VRAM and license

Document AI means reading PDFs, scans, tables, forms and charts and turning them into text and answers. The best open-weight models for it fit on a single GPU. These are our picks, with the document scores each developer publishes, the GPU memory each model needs and its license.

Our picks by hardware size

ONE GPU · 32 TO 94 GB

Qwen3.8-27B

Qwen · About 24 GB · Apache 2.0

91.1% on OmniDocBench 1.5 and 83.7% on CharXiv chart reasoning in about 24 GB of memory, under Apache 2.0. Its document score is level with the figure published for Kimi K3, in a small fraction of the memory.

Qwen3.8-27B hardware requirements →

ONE GPU, MANY DOCUMENTS AT ONCE · ABOUT 24 GB

Qwen3.6-35B-A3B

Qwen · About 24 GB · Apache 2.0

89.9% on OmniDocBench 1.5, 81.9% on CC-OCR and 90.0% on OCRBench. Only 3B parameters are active per token, so it works through large batches of pages faster.

Qwen3.6-35B-A3B hardware requirements →

MORE THAN ONE SERVER · ABOUT 1.64 TB

Kimi K3.1 Pro

Moonshot AI · About 1.64 TB · Kimi K3.1 Pro License

67.4% on OfficeQA Pro, which tests questions across office files, and 89.0% on CharXiv, with a 1M-token context. Worth it only when the model must reason across very long documents; for reading pages, the small models match the largest.

Choosing a model for your team?

Tell us your users, your work and your data. We will recommend a model and the hardware to run it inside your own building with Vanik Edge OS. Free, no obligation.

Get a free sizing check

Scores and hardware, model by model

Scores are the ones each developer reports on its own model card, without extra tools. Developers use different test settings, so compare models within a column loosely. A dash means the developer has not published that figure.

ModelOmniDocBenchCC-OCROCRBenchCharXiv (RQ)Memory, 1 userSmallest setup for 16 usersLicense
Qwen3.6-27B
Qwen model card
89.4% (v1.5)81.2%89.4%78.4%About 20 GB1 NVIDIA L40SApache 2.0
Gemma 4 31B
Google model card
0.131 edit distance (v1.5)———About 20 GB1 NVIDIA L40SApache 2.0
Qwen3.8-27B
Qwen model card
91.1% (v1.5)——83.7%About 24 GB1 NVIDIA L40SApache 2.0
Qwen3.6-35B-A3B
Qwen model card
89.9% (v1.5)81.9%90.0%78.0%About 24 GB1 NVIDIA L40SApache 2.0
Nemotron 3.5 Super VL
NVIDIA model card
——68.3% (v2, English)79.9%About 80 GB1 NVIDIA RTX PRO 6000 BlackwellOpenMDW 1.1
Kimi K2.6
Moonshot AI model card
———80.4%About 626 GB8 × NVIDIA H100 NVLModified MIT
Kimi K3
Moonshot AI model card
91.1%——84.8%About 1.62 TB8 × AMD Instinct MI325XKimi K3 License
Kimi K3.1 Pro
Moonshot AI model card
———89.0%About 1.64 TB8 × AMD Instinct MI325XKimi K3.1 Pro License

Most of these models need several GPUs working as one. Buying the GPUs is the easy part. Vanik Edge OS splits the model across them, keeps it running if a server fails and manages it all from one console. See how Edge OS clusters GPU servers

What the benchmarks measure

OmniDocBench. Turning PDF pages into text, tables, formulas and the right reading order. Google reports it as an edit distance, where lower is better, so Gemma’s figure cannot be compared with the percentages.

CC-OCR. Reading text in varied settings: street scenes, multilingual text, document parsing and pulling named fields out of forms.

OCRBench. Recognizing and understanding text in images, including handwriting and formulas.

CharXiv (RQ). Reasoning questions about charts taken from research papers.

FROM SIZING TO RUNNING

Sizing the hardware is the easy part.

Getting several GPUs to work as one system for a whole company is where most projects slow down. That is the part Vanik Edge OS does for you.

  1. STEP 01 · THIS TOOL

    Size it

    Work out the GPU memory your models need and which GPUs or servers can hold them.

  2. STEP 02 · VANIK EDGE OS

    Cluster it

    Edge OS joins your GPU servers into one cluster, splits a large model across them and keeps it running if a server fails. NVIDIA and AMD servers work side by side.

  3. STEP 03 · VANIK EDGE OS

    Run it

    Your team gets AI apps with single sign-on and an audit trail, managed from one console, with your data staying inside your building.

See how Edge OS clusters GPU servers

Best local LLMs for document AI: questions

What is the best local LLM for document AI?

Qwen3.8-27B: 91.1% on OmniDocBench 1.5 in about 24 GB of GPU memory, under Apache 2.0. For large batches of pages, Qwen3.6-35B-A3B is faster and scores 89.9%.

Which local LLM is best for OCR?

Of the models with published OCR scores, Qwen3.6-35B-A3B leads on both CC-OCR (81.9%) and OCRBench (90.0%), just ahead of Qwen3.6-27B. Qwen3.8-27B has the highest document-parsing score but its card does not publish these two tests.

Do I need a large model to read documents?

No. On OmniDocBench, Qwen3.8-27B in about 24 GB matches the figure Kimi K3 publishes in about 1.62 TB. A large model only helps when it must reason across hundreds of pages at once or do other work as well.

How much VRAM does document processing need?

More than the model’s base figure. Each page image becomes hundreds to thousands of tokens, so a 50-page PDF can fill a long context and a large KV cache. Plan memory for pages per request and for how many requests run at once.

Why run document AI on your own hardware?

Contracts, invoices, medical records and identity documents are the files companies least want to send to a third party. A model running inside your own building reads them without the pages leaving your network.

Where do these scores come from?

Each developer’s own model card, linked in the table. We did not run the benchmarks ourselves, and developers use different test settings, so small differences between models are not meaningful.