Building NPS Cross Product Usage, Pt. 2

Net Promoter has become a universal way of polling and benchmarking customer happiness. Despite the high adoption rates, we are still lacking data about what a “good” or “bad” NPS looks like. The Net Promoter System was developed to help companies gauge the success of their user base as a whole – this means it is critical to define what that success entails.

NPS is a both a relevant and popular topic at Pendo. We not only use and track NPS furiously, we also offer it as part of our product experience platform. For a while, we’ve been digging deeper into this metric to understand the potential that NPS has for benchmarking purposes.

As Pendo’s data scientist, I started this evaluation alongside our product and success teams to reveal some surprising insights on how NPS is being utilized by many companies today. About a month ago, Pendo Product Manager, Julie Harrow, wrote about the possibilities that NPS holds in regards to benchmarking. This post is a continuation of the series to present our early findings and insights.

The analysis covered sixteen months of data, starting January 2016 through April 2017. We focused on three segments of company size in order to break out the NPS results. Each company had over three months of consecutive NPS results for a more accurate comparison.

結果

The data across the companies show an average NPS of 8.24, with the 75th percentile at 26.85.

A key takeaway from our initial evaluation revolved around how companies were displaying their NPS survey, specifically the segments they used. For accuracy on an individual score basis, a consistent method of segmentation should be followed. Below are seven ways we observed segmentation being used with NPS.

1. Product User vs. End User
It is important to separately consider the NPS responses of product users and end users. The two different populations of users have different priorities and goals, so their responses may vary greatly.

2. Customer Role
Customers with different product access privileges have different product experiences, therefore their NPS scores should be considered separately.

3. Customer Lifetime in Product
The amount of time a customer has spent in the product can provide a better understanding of customer experience for both new and experienced users. New users can be presented NPS polls after a threshold number of days in the product to get an idea of their product experience; similarly for experienced users.

4. Sample Groups
Polls can be presented at regular intervals using random assignment to ensure a sampling of customers at pre-determined frequencies.

5. Feature or Page Usage
Customers can be presented NPS upon landing on a certain page or using a particular feature to gather more targeted feedback.

6. Paid vs. Unpaid Accounts
Some companies segment polls based on account type including payment level. This is especially important if the paid and unpaid product versions are different including features or level of access.

7. Language
NPS can be tested across different languages for companies operating across the World. This will allow for a better understanding of customer success across cultures.

NPS Early Learnings

This is only the beginning of our research, but the observations we have made thus far can be impactful for any company looking to start (or revise their strategy around) NPS. Since the goal of NPS is to provide accurate data on customer success, the quality of the data is important. Five things we learned with the initial evaluation:

1. Run a single campaign for a target segment
Running a single campaign and editing your poll time period accordingly can help ensure consistent sampling for a given target segment.

2. Experiment with NPS to understand response rate and target audience appropriately
To determine the response rate (and thereby not overwhelm your users or, alternatively, under sample your user population), present the poll to a random segment of 10% of your users. This will provide an idea of the response rate before you decide your segment criteria and poll period.

3. Implement random sampling when there is a sufficient sample size
Creating a randomized sample will ensure that you are not introducing bias into your NPS calculations.

4. Name campaigns appropriately
If you have multiple live campaigns, it can be hard to keep track of them. This can hinder your ability to draw conclusions. An easily identifiable naming convention can aid in campaign management and collaboration. For example, “2017 Q1 NPS for Admins” and “2017 Q2 NPS for Restricted Access Users” can ensure that all users are able to easily interpret the poll results.

5. Run campaigns for at least 3 intervals to get an accurate NPS measurement
If you report on or collect NPS scores at regular intervals, consider the summary of at least 3 if possible. For example, if you report on NPS each month, collect a summary of three months before drawing any conclusions. This will allow you to capture a more accurate picture of your NPS trends.