EDGE OSMOONSHOT AI · 2.8T PARAMETERS
Kimi K3 hardware and VRAM requirements
Moonshot AI’s 2.8 trillion parameter model, the largest open-weight model on Hugging Face. Kimi K3 is a mixture-of-experts model with 104B of its 2.8T parameters work on each token. At MXFP4 it needs about 1.62 TB of GPU memory for one user, and about 1.63 TB to serve a team of 16 with 8K context each.
Total parameters
2.8T
Active per token
104B
Context length
1M tokens
Released as
MXFP4 (native)
Minimum VRAM
1.62 TB at MXFP4
For a team of 16
1.63 TB at MXFP4
VRAM needed to run Kimi K3
GPU memory by precision, number of concurrent users and context length. Each user needs their own KV cache, so memory grows with users and context.
| Precision | Weights only | 1 user, 8K | 16 users, 8K | 16 users, 32K | LoRA fine-tune |
|---|---|---|---|---|---|
| MXFP4 (native) Only released format | 1.54 TB | 1.62 TB | 1.63 TB | 1.64 TB | 2.07 TB |
Full fine-tuning in mixed precision needs about 45.09 TB: weights, gradients and optimizer states at roughly 16 bytes per parameter. LoRA trains small adapter layers on a frozen copy of the model instead, so it needs far less.
GPUs that can run Kimi K3 on-premise
For a team of 16 using Kimi K3 at MXFP4 with 8K context each (about 1.63 TB), smallest fit first.
AMD Instinct MI325X
256 GB per GPU · Split across 8 GPUs in one server, 218 GB to spare.
NVIDIA DGX B200
1.44 TB per system · Needs 2 linked systems, 967 GB to spare.
NVIDIA DGX Spark
128 GB per system · Needs more than 2 linked systems.
NVIDIA L40S
48 GB per GPU · Needs more than one 8-GPU server.
NVIDIA H100 NVL
94 GB per GPU · Needs more than one 8-GPU server.
NVIDIA H200 NVL
141 GB per GPU · Needs more than one 8-GPU server.
NVIDIA RTX PRO 6000 Blackwell
96 GB per GPU · Needs more than one 8-GPU server.
AMD Instinct MI300X
192 GB per GPU · Needs more than one 8-GPU server.
Every setup for Kimi K3 needs several GPUs working as one. Buying the GPUs is the easy part. Vanik Edge OS splits the model across them, keeps it running if a server fails and manages it all from one console. See how Edge OS clusters GPU servers
Weighing two of these? Compare AI hardware side by side, for example H100 vs H200, RTX PRO 6000 vs RTX 5090 or DGX Spark vs Mac Studio.
Running Kimi K3 for your team?
We will size the GPU server for your users and data, and set up Kimi K3 on it inside your own building. Free, no obligation.
Kimi K3 at a glance
Compare Kimi K3 with other models
FROM SIZING TO RUNNING
Sizing the hardware is the easy part.
Getting several GPUs to work as one system for a whole company is where most projects slow down. That is the part Vanik Edge OS does for you.
STEP 01 · THIS TOOL
Size it
Work out the GPU memory your models need and which GPUs or servers can hold them.
STEP 02 · VANIK EDGE OS
Cluster it
Edge OS joins your GPU servers into one cluster, splits a large model across them and keeps it running if a server fails. NVIDIA and AMD servers work side by side.
STEP 03 · VANIK EDGE OS
Run it
Your team gets AI apps with single sign-on and an audit trail, managed from one console, with your data staying inside your building.
Kimi K3 GPU questions
How much VRAM to run Kimi K3?
Kimi K3 needs about 1.62 TB of GPU memory at MXFP4 for one user with 8K context: 1.54 TB for the weights plus the KV cache and runtime overhead. Serving 16 people at once with 8K context each needs about 1.63 TB.
Is Kimi K3 open source?
Kimi K3’s weights are published openly on Hugging Face (moonshotai/Kimi-K3), so you can download and self-host it. Check Moonshot AI’s license on the model page for commercial terms.
What do you need to run Kimi K3 locally?
To run Kimi K3 locally you need about 1.62 TB of GPU memory at MXFP4 for one user (1.63 TB for a team of 16), system RAM of at least 1.54 TB so the weights can be loaded, around 1.54 TB of fast NVMe storage for the model files, and an inference engine such as vLLM or SGLang. A team of 16 fits on AMD Instinct MI325X (8 GPUs).
Can you run Kimi K3 locally?
Yes, on data-center GPUs. At MXFP4 it needs about 1.62 TB of GPU memory for one user, more than consumer graphics cards or laptops hold. A team of 16 fits on AMD Instinct MI325X (8 GPUs).
How big is Kimi K3?
Kimi K3 has 2.8T parameters in total, of which 104B work on each token. The weights take about 1.54 TB at MXFP4.
Is Kimi K3 multimodal?
Yes. Kimi K3 accepts text and images, and Moonshot AI says it also understands video. It produces text.
Why does Kimi K3 need so much memory if only 104B parameters are active?
It is a mixture-of-experts model. Only 104B parameters work on each token, which keeps it fast, but a different set of experts is picked for every token, so all 2.8T parameters must stay loaded in GPU memory.
Can I fine-tune Kimi K3?
LoRA fine-tuning at MXFP4 needs roughly 2.07 TB with 4K sequences; it trains small adapter layers on a frozen copy of the model. Full fine-tuning needs about 45.09 TB, which calls for a multi-GPU training cluster.
How are these numbers calculated?
Weights are parameters × bytes per parameter. The KV cache uses 27.0 KB per token per user. We add about 5% of the weights plus 1 GB for runtime overhead and keep 10% of each GPU free. Figures are planning estimates; real usage depends on the inference engine and settings.
VRAM for similar models
Specifications from Hugging Face: moonshotai/Kimi-K3. Memory and bandwidth are manufacturer figures; GPU counts are rounded to 1, 2, 4 or 8 per server. All figures are planning estimates. Last checked November 10, 2026.
Run Kimi K3 inside your own building
Vanik Edge OS runs open-weight models on NVIDIA DGX, AMD Instinct and other GPU servers on your premises, so your data never leaves.