NPS for Entire Industries

In my previous post, I wrote about how the Portland software community was engaged through successive surveys, each smarter and more focused than the last. Okay, now they’re on board. But can you really measure the likelihood that an industry will succeed? Sure.

We use NPS for customer satisfaction analysis when we work for companies. It occurred to me – why not use it for entire industries? If people are passionate about their cluster, whether it’s the South Bend Tourism board or the California semiconductor industry, they’ll thrive, and NPS can measure that passion.

For our survey of the Portland software cluster, we asked if people would recommend Portland to an industry colleague as a place to do business. When we started, we had as many detractors as promoters – an NPS of 0%. That meant that if you talked to someone about the city, there was a good chance they would trash it. No wonder the software sector wasn’t going anywhere.

Six months later, the NPS is 23%. Our goal is to bring that to 40% over next 2 years, and we’ll make it. When we reach the goal, people will be fighting to put out their shingle here. Portland already has the talented people. Once they’re all brand evangelists for the city, it’s a done deal.

Misusing Recommendations

Our clients often expect us to include a recommendation question in a survey, to get an NPS score. However, Valeria Maltoni over at Conversation Agent says that the NPS (Net Promoter Score) doesn’t tell you enough. She’s right.

Some history: looking for a tool that would best predict corporate success, Fred Reichheld examined an assortment of survey questions, and found that a positive answer to the question “Would you recommend our company to a friend or colleague?” had the highest correlation with long term revenue growth.

To help improve corporate performance, Reichheld recommended that the question be asked soon after a transaction, to quickly assess the experience, and keep employees aware of their crucial role as the company’s public face.

It was also a good way to tell which agents or group of agents were leaving the best impression.

That was it. End of story.

Except that companies immediately started misusing it, asking it annually (or more often) of all customers regardless of whether or not they’d done a transaction. At i-OP one of our own vendors sent us a survey with the NPS question twice a year, like clockwork. No reasons why, nothing qualitative.

Problem is, B2B relationships are complex – a lot of factors affect our experience, and you need to understand which of those have the highest correlation with the customer’s overall assessment. To do this analysis you need to use some form of regression-like analysis (we actually use something called Bayesian Inference) to find the drivers behind the experience.

If you just ask the question, and don’t delve into the reasons, you’re diminishing the relationship, not building it. And if you don’t understand what the drivers are behind that experience, you don’t know how to focus your efforts to improve it.

We dropped that vendor. It’s not enough to ask how you’re doing – you need to ask why, and listen deeply to the answers.