AI Agent Performance
Overview
The AI Agent Performance Dashboard is a powerful, centralized analytics tool designed to surface key insights on how your AI agent is performing. It brings clarity to user behavior, engagement trends, and content opportunities—making it easier than ever for teams to optimize experiences and prove value.
This dashboard gives you access to top-level KPIs, new engagement insights, and deeper topic-level reporting—all with no setup required.
Key Capabilities
Below is a breakdown of each tile included in the AI Agent Performance Dashboard, along with a description of its functionality.
AI Chat Volume
Analyze your customers’ usage to see your AI Agent’s impact on your business.
Displays the total number of distinct messages by end users. Shows all messages from end users to your AI agent.
Includes a calculated “Messages per Conversation” ratio to provide insight into engagement depth.
Shows the total number of distinct conversations. A conversation is defined as all messages from a given end user on a given day.
Includes a “Without Escalation” percentage to indicate how many interactions were handled without human intervention.
Calculates the estimated number of work hours saved, based on the number of AI-only conversations (assumed 5 minutes saved per conversation).
Displays the percentage of interactions that occurred outside of traditional business hours (9AM - 5PM Local timezone).
All weekend hours are included in "outside business hours"
Hours Saved quantifies AI-driven efficiency and highlights how much support is being delivered after hours.
Reports on the total number of link clicks generated by the AI Agent. Also shows how many times a user clicked onto an external link from the chat experience, helping you better track how your AI Agent guides the user to the most relevant pages on your website.
Click on the big number for a deep dive into exactly which links are generating the most engagement.
For Marketing teams: Test different call-to-action (CTA) placements by running an A/B campaign. Use the Analytics Dashboard to compare performance and see which positioning drives more engagement.
Engagement Over Time
Identify peak and off-hour usage to align your support coverage with user demand.
Displays daily volume of both conversations and messages across the selected time period using an area chart. Trends are plotted to show fluctuations, spikes, or dips in usage over time.
Provides visibility into usage trends over time, helping identify seasonal surges, campaign impact, or anomalies in engagement.
Bar chart showing message volume distributed across each hour of the day.
Helps you identify when users are most active so support staffing, content delivery, and automation strategies can be aligned with demand.
Stacked horizontal bar chart comparing the number of messages sent during business hours versus outside business hours, broken down by device type (e.g. desktop, mobile)
Reveals how usage patterns vary across platforms and helps assess the AI Agent’s role in after-hours support coverage.
Local to the client’s timezone
Inside business hours: 9am - 5pm, Monday to Friday
Segmentation Analysis
Analyze how users interact with your AI agent by message type, device channel, and response style.
Message type donut chart categoeizes end-user inputs by type: free text, quick replies, and structured message elements like buttons.
Offers insights into how users prefer to interact with the AI and whether the experience is driven more by open-ended inputs or predefined flows.
Device channel donut chart displays the distribution of messages across device types, such as Web, App, etc
Highlights where users are engaging with the AI the most, enabling teams to optimize design, messaging, and content for the most-used channels.
Generative vs. Prewritten donut chart shows the breakdown of generative AI responses versus pre-written scripted replies, specifically for end-user text inputs. Helps quantify how much the AI relies on generative models versus traditional scripted content.
We recommend a 70/30 ratio—with 70% of responses being generative—because generative responses are more personalized, flexible, and better at mirroring how people naturally communicate. They allow your AI agent to respond with more relevance and nuance, especially when handling open-ended or complex queries.
User Topics
Table listing the most common user topics based on message volume. Helps you analyze what guests are asking your AI agent, based on message volume.
Surfaces what users are asking the AI agent the most, enabling prioritization of content updates and validation of coverage in high-impact areas.
Drill down on a specific topic to see the sub-topics associated.
Drill down on a specific sub-topic to see the intents associated.
Topic Content Gaps
Topic Content Gaps is a table identifying user topics where the AI agent's answer was labeled as “No Response”. Includes counts of uncovered messages per topic.
Pinpoints weak spots in the AI’s training data or content coverage, helping you focus improvement efforts where they’re needed most.
Drill down on a specific topic to see the sub-topics associated.
Drill down on a specific sub-topic to see the intents associated.
Last updated
Was this helpful?