Diagnosing a Break in the Flow of Our New Beta Product

For the last few months, my team has been working on a brand new guide-building designer experience. While today’s guide-building has a ton of flexibility and power, there were some major areas of opportunity in which we wanted to invest. Hence, the new designer.

Part of the new designer experience is actually ‘launching’ a user from our Pendo interface (app.pendo.io) into the designer, which lives over the user’s application (new tab, new URL). We always knew that this launch would be finicky — communicating between two different applications in two different browser tabs always leaves room for some information to get lost in translation — so we decided to start off with a simple implementation and see (1) what breaks and (2) how often. The MVP consisted of five simple steps:

1. User chooses to “Manage [guide] in her app”
2. Pendo asks the user for the link to her application
3. Pendo opens her application in a new tab
4. Pendo appends a query parameter onto the end of her application URL to kick off our guide designer
5. User proceeds to design her guide

We shipped our beta toward the end of August, and as users started to try out the product, we started to learn the types of problems that arose. Some sites drop parameters, other sites have tons of URL redirects, etc. However, as the beta grew larger and larger (which was intentional), our customers became less and less forthcoming about whether or not the launch occurred successfully. So, we did what any reasonable PM at Pendo would do — tagged the features and created a funnel.

Button tagged as “P2 Launch Designer Button” in Pendo

View tagged as “P2 Action Bar Page” in Pendo

Pendo funnel from “P2 Launch Designer Button”(app.pendo.io) to “P2 Action Bar” (designer)

There were our findings! Approximately 29% of all users attempting to launch the designer hadn’t completed the task. This was definitely a large enough percentage to pursue technical alternatives for the launching process. And since funnels also provide us with a list of users who complete each step, we were also able to follow up with those users who weren’t successfully launching the designer. We used guides with open text polls and some face-to-face troubleshooting to reach out and understand the main reasons they weren’t successful.

After this research was complete, we went back to the drawing board with data to justify putting design and engineering resources back on this project to help find alternatives. I’m looking forward to shipping the next iteration of the launch and am confident that we’ll be able to measure its success when the time comes!