AI Support & Agents

Temperature (LLM)

Définition

Temperature is a setting that controls how random or deterministic a language model's output is: low temperature produces focused, predictable responses, while high temperature produces more varied and creative ones.

Aussi connu sous : sampling temperaturerandomness setting

How it works

At each step, a model produces a probability distribution over possible next tokens. Temperature reshapes that distribution before a token is picked. Near 0, the model almost always chooses the most likely token, giving consistent, repeatable answers. Higher values (say 0.7–1.0+) flatten the distribution, making less-likely tokens more probable and the output more diverse.

It's often tuned alongside settings like top-p (nucleus sampling), which similarly narrow or widen the pool of candidate tokens.

Why it matters for support

For customer support, low temperature is usually the right call. You want accurate, consistent, on-policy answers — not creative variation — so an agent replies the same way to the same question and stays grounded in your sources. Higher temperature suits brainstorming or drafting, where variety is a feature rather than a risk.

Questions fréquentes

What temperature should a support agent use?

Usually a low value (near 0) so answers are consistent, factual, and on-policy. Support prioritizes reliability over creative variety, and low temperature reduces the chance of off-script or inconsistent replies.

Does temperature cause hallucinations?

Higher temperature can increase the odds of an unexpected or ungrounded response by making unlikely tokens more probable, but hallucinations stem mainly from missing or weak grounding. Low temperature plus good retrieval reduces the risk on both fronts.

Termes liés

Large Language Model (LLM)

A large language model (LLM) is a neural network trained on vast amounts of text to predict and generate language, enabling it to understand questions, summarize, classify, and write human-like responses..

AI Hallucination

An AI hallucination is when a language model generates a response that is fluent and confident but factually wrong or fabricated — inventing details, policies, or sources that don't exist..

Tokens (LLM)

Tokens are the small chunks of text — words, parts of words, or characters — that a language model reads and generates; they're the fundamental unit models use to measure input, output, context limits, and usually billing..

Prompt Engineering

Prompt engineering is the practice of designing and refining the instructions given to a language model to get accurate, relevant, and reliable outputs for a specific task..

System Prompt

A system prompt is the underlying set of instructions that defines an AI model's role, behavior, tone, and boundaries for a conversation — set by the developer, not the end user, and applied to every interaction..

Passez à l'action

Macha est une couche d'agents IA qui se pose sur le help desk que vous utilisez déjà, Zendesk, Freshdesk, Front, Intercom ou Gorgias.

Démarrer l'essai