AI Support & Agents

Tokens (LLM)

Definition

Tokens are the small chunks of text — words, parts of words, or characters — that a language model reads and generates; they're the fundamental unit models use to measure input, output, context limits, and usually billing.

Also known as: LLM tokenstext tokenssubword tokens

How it works

Before a model processes text, a tokenizer splits it into tokens. Common words may be a single token while longer or rarer words break into several ("tokenization" might become "token" + "ization"). As a rough rule of thumb for English, one token is about four characters, or roughly ¾ of a word — so 1,000 tokens is around 750 words.

The model reads tokens as input and produces tokens as output. Both count toward the context window, and most LLM APIs price usage per token, split between input and output rates.

Why it matters for support

Tokens are the unit of cost and capacity for any LLM-based system. Long ticket threads, large retrieved documents, and verbose prompts all consume more tokens — which is why concise prompts and targeted retrieval keep both latency and API cost down.

Frequently asked

How many words is a token?

For English, one token averages roughly ¾ of a word — about four characters. So ~1,000 tokens is around 750 words, though it varies with the exact text and tokenizer.

Why are LLMs billed per token?

Tokens are the actual unit a model processes, so token count is the most direct measure of the compute a request uses. Providers typically charge separate rates for input and output tokens.

Related terms

Context Window

The context window is the maximum amount of text — measured in tokens — that a language model can consider at once, including both the input you give it and the output it generates..

Large Language Model (LLM)

A large language model (LLM) is a neural network trained on vast amounts of text to predict and generate language, enabling it to understand questions, summarize, classify, and write human-like responses..

Transformer Model

A transformer model is a neural network architecture that processes an entire sequence of text at once using an attention mechanism to weigh how much each word relates to every other word, and it's the foundation of nearly every modern large language model..

Embeddings

Embeddings are numeric vector representations of text (or images, code, etc.) that capture meaning, so that pieces of content with similar meaning sit close together in a high-dimensional space..

Temperature (LLM)

Temperature is a setting that controls how random or deterministic a language model's output is: low temperature produces focused, predictable responses, while high temperature produces more varied and creative ones..

Put these ideas to work

Macha is an AI agent layer that sits on top of the help desk you already run — Zendesk, Freshdesk, Front, Intercom, or Gorgias.

Start Trial