Your conversational AI agents are up and running. But then someone asks, “Are they actually helping our users?” We built Pendo Agent Analytics to provide clarity into value agents deliver using real-time, reliable data—for our customers and ourselves.
With Agent Analytics, you can see:
- What users ask your agent (categorized by most common use cases)
- What users do before and after agent interactions
- Which conversations drive business outcomes or create friction
Any time we build a new Pendo product or feature, we’re eager to test it ourselves. And as we’re launching more AI agents in Pendo—like Listen Explore and Agent Mode—Agent Analytics is quickly becoming a fan favorite amongst our Product leaders.
All your questions about agent impact, answered
Alejandro Dao, a lead product manager at Pendo, is using Agent Analytics to get immediate insights into how teams are using the new AI guide-creation feature—a new addition and part of our Autumn Release. This feature lets users turn a plain-language prompt into a polished, branded in-app guide in seconds. (Learn more about the AI Guide feature here.)
During the build process, Dao consistently used Agent Analytics to gather insights as internal teams and Design Partners were using it. Here’s what he learned:
Because Agent Analytics can pull conversational elements from an agent and deliver them in a report, Dao leveraged these reports to see who was using the feature, who used it most frequently, and most importantly—what the use cases were.
As he consistently monitored the most common things users asked the guide agent to do, Dao uncovered a desired use case not initially planned for—the ability to add a video into the guide. While the agent doesn’t have this capability yet, the data-backed insight allowed the team to prioritize it in future roadmap decisions, with confidence. In the meantime, the team has better-communicated what is and is not possible (yet) with the feature so that users don’t become frustrated and abandon their efforts.
Identifying unreported user frustrations with agent insights
Dao made another key discovery about AI guide creation by pairing Agent Analytics with Session Replay.
Using the latter to get visual context on dead clicks happening in the AI-led guide creation process, Dao saw repeated attempts by users to edit the in-line text. Dao had suspected this could be an issue, but it wasn’t a priority in the product roadmap, nor had any Design Partner raised it as an issue. Seeing this use case over and over firsthand gave Dao the confidence and authority he needed to make it a more-immediate engineering priority.
Without the data from Agent Analytics, Pendo’s product team would be relying on occasional, voluntary user feedback and guesswork. Instead, teams like Dao’s have detailed insights in seconds. No extra report-building, and no scheduling of countless customer calls to understand the voice of their users. As Dao puts it, “Agent Analytics doesn’t replace good product fundamentals. But it does act as an accelerator or magnifier of your own skills.”
Agents are talking, are you listening?
The critical step with AI initiatives now is measuring efficacy. There’s a wealth of user intent data, frustration, and risk, just sitting within your agents. Are you ready to optimize for better user outcomes?
Take a tour of Agent Analytics and see what’s possible.
