Zendesk
Freshdesk
Gorgias
Front
Shopify
Stripe
Slack
Notion
Google Workspace
Confluence
Prompt Injection
Definition
Prompt injection is a security attack where malicious instructions are hidden in user input or external content to manipulate an AI system into ignoring its original instructions or performing unintended actions.
How it works
Because LLMs process instructions and data in the same text stream, an attacker can embed commands like "ignore your previous instructions" in a chat message, a support ticket, or a web page the agent reads. If the system can't distinguish trusted instructions from untrusted content, it may follow the injected command.
Indirect prompt injection is especially dangerous for AI agents that fetch external content or call tools, since the malicious instruction can arrive through retrieved data rather than directly from the user.
Why it matters for support
An AI support agent connected to real systems could be tricked into leaking data, bypassing policy, or taking harmful actions. Defenses include guardrails, input filtering, least-privilege tool permissions, and keeping the system prompt separate from and authoritative over user content.
Frequently asked
What is the difference between prompt injection and a jailbreak?
A jailbreak tricks a model into bypassing its safety rules; prompt injection more broadly overrides the system's intended instructions, often via untrusted input or external content. They overlap but injection includes indirect attacks through retrieved data.
How do you prevent prompt injection?
Separate trusted instructions from untrusted input, apply guardrails and input filtering, limit tool permissions to least privilege, and require confirmation for sensitive actions.
Related terms
AI Guardrails
AI guardrails are the rules, checks, and constraints placed around an AI system to keep its behavior safe, on-topic, and within policy — controlling what it can say, do, and access..
System Prompt
A system prompt is the underlying set of instructions that defines an AI model's role, behavior, tone, and boundaries for a conversation — set by the developer, not the end user, and applied to every interaction..
Large Language Model (LLM)
A large language model (LLM) is a neural network trained on vast amounts of text to predict and generate language, enabling it to understand questions, summarize, classify, and write human-like responses..
AI Agent
An AI agent is a software system that uses a language model to understand a request, decide what to do, and take actions — calling tools, retrieving data, and completing tasks — rather than just returning a single scripted answer..
Tool Use
Tool use is the ability of an AI model to invoke external functions, APIs, or systems — like looking up an order or issuing a refund — instead of only generating text, so it can act on real data rather than just describe it..
Put these ideas to work
Macha is an AI agent layer that sits on top of the help desk you already run — Zendesk, Freshdesk, Front, Intercom, or Gorgias.
Start Trial