EDGE OSDEFINITION
A context window is the maximum amount of text a language model can consider at once, measured in tokens. It includes the instructions, the conversation so far, any documents provided and the response being generated. A token is roughly three-quarters of an English word, so an 8K context window holds about 6,000 words. Anything beyond the window is not seen by the model.
Planning hardware for AI on your own premises? Work out the GPU memory a model needs in our VRAM calculator, or let us size it with you.
Get a free sizing checkWhy the Context Window Matters
- Document size: Sets how much of a contract, report or codebase the model can read in one go.
- Memory: Longer context needs a larger KV cache, and so more GPU memory for every user.
- Speed: Very long prompts take longer to process before the first token appears.
- Accuracy: Models can overlook details buried in the middle of very long inputs.
Use Cases
- Summarizing long reports, contracts and policies.
- Assistants that remember earlier parts of a conversation.
- Coding assistants that read many files at once.
- Deciding when to use RAG and chunking instead of pasting whole documents.
See it running on your own hardware
Vanik Edge OS runs AI models, agents and apps inside your building, so your data stays there.