AI Support & Agents

Context Window

Definition

The context window is the maximum amount of text — measured in tokens — that a language model can consider at once, including both the input you give it and the output it generates.

Also known as: context lengthcontext sizetoken limit

How it works

Everything the model "sees" for a request — the system prompt, the conversation history, any retrieved documents, and the reply being written — has to fit inside the context window, counted in tokens. Exceed it and older content is truncated or dropped, so the model effectively forgets the earliest parts.

Window sizes vary by model, ranging from a few thousand tokens to well over a million. A larger window lets the model reason over more history or documents at once, but processing more tokens also costs more and can slow responses.

Why it matters for support

The context window sets the practical limit on how much of a long ticket thread, knowledge base, or customer history an agent can hold in mind at once. This is a core reason RAG exists: rather than stuffing an entire knowledge base into the window, you retrieve just the relevant passages and fit them in — keeping answers grounded and costs controlled.

Frequently asked

What happens when the context window is exceeded?

Content beyond the limit is truncated or dropped, so the model may lose earlier parts of the conversation or documents. Techniques like retrieval and summarization keep only the most relevant text within the window.

Is a bigger context window always better?

Not always. A larger window handles more history at once, but processing more tokens increases cost and latency, and models can still overlook details buried in a very long context. Good retrieval often beats simply cramming in more text.

Related terms

Tokens (LLM)

Tokens are the small chunks of text — words, parts of words, or characters — that a language model reads and generates; they're the fundamental unit models use to measure input, output, context limits, and usually billing..

Large Language Model (LLM)

A large language model (LLM) is a neural network trained on vast amounts of text to predict and generate language, enabling it to understand questions, summarize, classify, and write human-like responses..

Retrieval-Augmented Generation (RAG)

Retrieval-augmented generation (RAG) is a technique where an AI model retrieves relevant information from your own knowledge sources at query time and uses it to ground its answer, instead of relying only on what it memorized during training..

Agent Memory

Agent memory is an AI agent's ability to retain and recall information across a conversation — or across sessions — such as earlier messages, customer details, and past interactions, so it stays coherent instead of treating every turn as brand new..

Transformer Model

A transformer model is a neural network architecture that processes an entire sequence of text at once using an attention mechanism to weigh how much each word relates to every other word, and it's the foundation of nearly every modern large language model..

Put these ideas to work

Macha is an AI agent layer that sits on top of the help desk you already run — Zendesk, Freshdesk, Front, Intercom, or Gorgias.

Start Trial