Shopify
Stripe
Slack
Notion
Google Workspace
Confluence
How Do You Measure AI Agent Performance in Macha? Agent Analytics Explained
Agent Analytics, under the Analyze section of Macha, logs every run your agents make: interactive chats, chatbot sessions and autonomous trigger runs. Each row shows which agent ran, what triggered it, the model and the message count, so you can see what is working and what to fix.
Key takeaways
- Macha's Agent Analytics logs every agent run as a row with six columns: Conversation, Agent, When, Source, Model and Msgs, covering chats, chatbot sessions and autonomous runs.
- Agent Analytics has a Table view for investigating individual runs and a Report view for volume trends broken down by agent, source and model.
- A run whose message count climbs toward 50 signals a struggling or looping agent, because Macha's fair-use rule stops a ticket at 50 messages.
- The Model column in Agent Analytics is for quality decisions, since Macha bills per ticket and the model an agent runs on does not change the charge.
- A weekly Agent Analytics review filters to the last 7 days, compares volume with the prior week, and checks message counts and trigger sources.
You measure a Macha agent's performance in Agent Analytics, under the Analyze section: it logs every run as one row with six columns (Conversation, Agent, When, Source, Model, Msgs), and a five-minute weekly check of volume, message counts and trigger sources catches most problems.
The log covers interactive chats, chatbot sessions and autonomous trigger runs alike, so nothing your agents do runs out of sight.
| Column | What it tells you | Warning sign |
|---|---|---|
| Conversation | The title of the run, such as the ticket it handled | Titles that don't match the agent's job |
| Agent | Which agent did the work | One agent doing everything while others sit idle |
| When | The timestamp, for activity over time | Gaps when tickets were arriving |
| Source | What triggered the run, such as ticket.created | Runs from a trigger you didn't expect |
| Model | Which AI model ran | A light model on hard, multi-step work |
| Msgs | How many messages the run took | Counts climbing toward 50 |
What does Agent Analytics show you?
Open Agent Analytics (under the Analyze section) and you get a log of every conversation your agents have had. Each row gives you the essentials at a glance.
The columns:
- Conversation: the title or subject of the run (for example, the ticket it handled).
- Agent: which of your agents did the work.
- When: the timestamp, so you can see activity over time.
- Source: what triggered the run (for example,
ticket.created), so you know it fired from the right event. - Model: which AI model ran. Model choice affects answer quality and speed. It does not change what you pay: Macha bills per ticket, and the model, tool calls and conversation length leave the charge the same.
- Msgs: how many messages the run took, a quick proxy for how much work each one needed.
At the top, the total count (for example, "346 conversations") and filters for date range, agent, source and model let you slice it however you need.
When should you use the Table view and when the Report view?
There are two ways to look at the same data:
- Table: the run-by-run log above. Use it to investigate specific runs, spot-check quality and see exactly what happened.
- Report: the aggregated view. Use it for the big picture: volume over time and how it breaks down across agents, sources and models.
You'll live in Table when something looks off and in Report when you want the trend. If you'd rather ask than filter, Sidekick (the assistant built into the Macha dashboard) can read your agents' past conversations, score them against the agent's own instructions, and plot a chart such as conversations per day.
Which numbers tell you an agent is working?
The data is only useful if you know what questions to ask of it:
- Volume and trend. Is automation handling more over time? A rising run count on your support agents is the headline "it's working" signal.
- Busiest agents. Which agents carry the load? If one agent does 80% of the runs, that's where to focus tuning.
- Source breakdown. Are runs firing from the triggers you expect (
ticket.created, a schedule, a webhook)? An agent firing from the wrong source, or far more often than it should, is a misconfiguration to catch. - Message counts. A run that takes a normal number of messages is healthy. An unusually high count can mean the agent is struggling or looping on a ticket. A ticket runs up to 50 messages under Macha's fair-use rule; at 50 the agent stops and the person is asked to start a new ticket. (If you see runaway message counts, see how to stop an agent from infinitely looping.)
- Model usage. Which models are running, and where? Because the charge is per ticket whatever the model, pick each agent's model for the job: a stronger model where answers are weak on multi-step work, a faster one where the task is simple.
What does a five-minute weekly review look like?
A short habit that keeps automation healthy:
- Filter to the last 7 days.
- Check total volume against the prior week. Up is good.
- Scan for abnormal message counts (possible loops or confusion).
- Confirm runs are firing from the expected sources.
- Spot-check a few Table rows to eyeball answer quality.
Anything odd points you straight at the agent to fix.
How do you tell healthy numbers from warning signs?
A quick reference for what the log is telling you:
| What you see | Likely meaning | What to do |
|---|---|---|
| Steady or rising run volume | Automation is taking load off the team | Keep going; widen scope |
| One agent doing almost everything | Your workhorse, or others aren't triggering | Check the quiet agents are firing |
| A run with an unusually high message count | The agent is struggling or looping | Tighten instructions; add a loop guard |
| Runs firing from an unexpected source | Wrong or extra trigger wired up | Fix the trigger on that agent |
| Weak answers on complex runs | The model is too light for the task | Try a stronger model on that agent |
| A sudden spike on one topic | A release, outage or pricing change hit | Investigate; maybe a temporary agent |
None of this needs a data team. It's all visible from a glance at the log and the filters.
What habits keep agent performance on track?
- Review weekly. Trends and problems show up early when you look regularly.
- Watch message counts as your loop and early-warning signal.
- Match models to tasks using the Model column, judged on answer quality rather than cost.
- Tie it back to outcomes. Pair the volume here with your help desk's CSAT and resolution metrics for the full picture.
Frequently asked questions
What is Agent Analytics? A reporting view that logs every run your Macha agents make (chat, chatbot and autonomous) with the agent, trigger source, model and message count, filterable by date, agent, source and model.
Does it cover autonomous runs, not just chats? Yes. Every conversation, including trigger-fired autonomous runs, appears in the log.
How do I tell if an agent is misbehaving? Watch for abnormally high message counts (a sign of looping or struggle) and runs firing from unexpected sources or far more often than expected.
Can I see costs here? Cost follows ticket count, not model. Macha bills per ticket on one plan priced by monthly volume (from $299 a month for up to 750 tickets), so the run volume in this log is the number to watch. The Model column is for quality decisions.
Where is it? Under the Analyze section in the left nav.
Can I pull the same data over the API? Yes. Macha's public API has a per-agent analytics endpoint (scope analytics:read) that defaults to the last 30 days. Its response shape is marked beta, so read the keys defensively.
See what your agents are doing: open Agent Analytics and review your last 7 days. $50 of free usage, no credit card required. Start free.
