Zendesk
Freshdesk
Gorgias
Front
Shopify
Stripe
Slack
Notion
Google Workspace
Confluence
Temperature (LLM)
Definition
Temperature is a setting that controls how random or deterministic a language model's output is: low temperature produces focused, predictable responses, while high temperature produces more varied and creative ones.
How it works
At each step, a model produces a probability distribution over possible next tokens. Temperature reshapes that distribution before a token is picked. Near 0, the model almost always chooses the most likely token, giving consistent, repeatable answers. Higher values (say 0.7–1.0+) flatten the distribution, making less-likely tokens more probable and the output more diverse.
It's often tuned alongside settings like top-p (nucleus sampling), which similarly narrow or widen the pool of candidate tokens.
Why it matters for support
For customer support, low temperature is usually the right call. You want accurate, consistent, on-policy answers — not creative variation — so an agent replies the same way to the same question and stays grounded in your sources. Higher temperature suits brainstorming or drafting, where variety is a feature rather than a risk.
Frequently asked
What temperature should a support agent use?
Usually a low value (near 0) so answers are consistent, factual, and on-policy. Support prioritizes reliability over creative variety, and low temperature reduces the chance of off-script or inconsistent replies.
Does temperature cause hallucinations?
Higher temperature can increase the odds of an unexpected or ungrounded response by making unlikely tokens more probable, but hallucinations stem mainly from missing or weak grounding. Low temperature plus good retrieval reduces the risk on both fronts.
Related terms
Large Language Model (LLM)
A large language model (LLM) is a neural network trained on vast amounts of text to predict and generate language, enabling it to understand questions, summarize, classify, and write human-like responses..
AI Hallucination
An AI hallucination is when a language model generates a response that is fluent and confident but factually wrong or fabricated — inventing details, policies, or sources that don't exist..
Tokens (LLM)
Tokens are the small chunks of text — words, parts of words, or characters — that a language model reads and generates; they're the fundamental unit models use to measure input, output, context limits, and usually billing..
Prompt Engineering
Prompt engineering is the practice of designing and refining the instructions given to a language model to get accurate, relevant, and reliable outputs for a specific task..
System Prompt
A system prompt is the underlying set of instructions that defines an AI model's role, behavior, tone, and boundaries for a conversation — set by the developer, not the end user, and applied to every interaction..
Put these ideas to work
Macha is an AI agent layer that sits on top of the help desk you already run — Zendesk, Freshdesk, Front, Intercom, or Gorgias.
Start Trial