Measuring Employee Relationship Quality

Written by

Measuring Employee Relationship Quality

Employee Relationship Quality

The Death of the Old Social Contract

We often talk about employees as human resources. We need to drop this terminology. Employees are not resources to be harvested or managed. They are people who turn up to work every day willing and able to do a good job for the customer. And very often we – the employers – prevent them from doing their job the best they can.

Sometimes we micro-manage them. Other times, we fail to give them the encouragement and motivation they need in order to succeed. We also expect them to achieve a proper work-life balance but fail to provide them with the flexibility to make it happen. We allowed them – actually forced them – to work from home during the COVID years. When they started to enjoy remote working, we forced them back into the office.

Even worse, we use phrases like “our staff are our most valuable asset” while secretly planning to cut 10% of the global workforce in response to AI, offshoring or other shadowy market forces. No wonder employees have become disillusioned and disengaged. 

It wasn’t always this way. In the 1960s and 1970s, there was a very clear and strong social contract between employer and employee. Baby Boomers enjoyed decent pay, job security, pension rights, regular and formal training, and so on. The contract worked both ways: employees were not just asked to work hard and do a good job for a good wage; they were expected to be loyal and stay for decades.

Over the decades, Baby Boomers retired and gave way to Generation X, Millennials, and now Generation Z. That social contract of old has all but disappeared for Gen Z employees. Today, the concept of a ‘job for life’ is – to quote Charles Dickens – as dead as a doornail. The nature of the employer/ employee relationship has changed utterly. Today’s job is no more secure than that of an English Premiership football manager.

A New Social Contract is Needed

If you think the previous paragraph is mere hyperbole, check the statistics.

The average tenure of Gen Z and Millennial employees is actually significantly less than that of an English Premiership football manager (currently 2.4 years). According to Randstad, a HR company, Gen Z’s average tenure in the first five years of their career is just 1.1 years compared to 1.8 years for Millennials. Millennials and Gen Z make up the majority of the workforce today and are far much more likely to ‘job hop’ than earlier generations.

The job market is different today – for both employees and employers. Employees may not have the job security of olden days but top companies have to battle to persuade the most talented people to join them. They also have to work much harder to make them stay. Remote working has further eroded the camaraderie between younger recruits, a key element in building bonds with colleagues and loyalty to the organisation.

Employee relationship quality has taken a major hit in the 21st century. And that’s a problem. A little employee churn is good. It allows new blood and new thinking to come into an organisation. A lot of churn is definitely not good, particularly if it takes 6-9 months to train a ‘newbie’ and get them fully up to speed, only to see them hop off to another company a few months later.

So how do we improve the quality of employee relationships with their employers? It seems to me that we need a new social contract. In many ways, Millennials and Gen Z are looking for the same things as Baby Boomers did. They want to feel that they are contributing, that they are constantly learning and progressing. Those elements are still at the heart of employee relationship quality. If we can build a new social contract with those elements at the heart of the offer, we have a good chance of keeping employees engaged. Engaged employees have a stronger bond with their employers. They stay longer. They perform better.

Measuring Employee Relationship Quality (ERQ)

We need to understand the quality of the relationships that our employees have with us in this new world. That means we need to measure it. We also need to ask the right questions for today’s new work environment.

Most companies with 1,000 or more employees already have robust systems in place for capturing the voices of both employees and customers. When it comes to mid-market companies, specifically those with 100-500 employees, there’s a much bigger deficit in understanding of what employees and customers think of them.

The lack of knowledge about employees is all the more surprising when you consider that companies today have access to a multitude of tools to capture their views. There really is no excuse today for not having a good employee feedback system in place. Think about it. Companies have email addresses for all employees to support employee surveys – something that often has to be gathered manually for customers, even when there is a customer relationship management (CRM) system in place.

The tools are available to capture the voice of the employee. Leadership teams just need to use them.

ERQ model_withCustFoc

So how do we measure Employee Relationship Quality?

Let’s start with a few basics:

  • Confidentiality. An effective employee feedback programme must be confidential so that employees can feel safe to speak openly and honestly. At the same time, it must deliver the level of detail and clarity required for leadership to drive meaningful change.
  • Methodology. We need to measure the right things for today’s employees. Today’s workplace is different – more remote working, less job security, greater tendency to ‘job hop’ and so on – and we need to reflect these changes in the work environment in the questions that we ask.
  • Frequency. All of our research suggests that the big annual employee survey is no longer enough. At a minimum, get feedback twice a year, or supplement an annual survey with more frequent pulse surveys on specific topics.

Deep-Insight has been capturing the voice of the employee for years but is now launching a new offering aimed specifically at this 100-500 employee market: Employee Relationship Quality (ERQ™).

ERQ is an independent and confidential way of getting employee insights that help you drive customer experience (CX) in your organisation. The ERQ model is based on eight drivers and takes into consideration the challenges of today’s world and aims to understand if your teams are set up for success. For example, it takes a close look at employees’ Wellbeing by measuring if they can maintain a sustainable work-life balance and if they feel a sense of job security. It also adds includes questions around Success and Support.

These eight drivers are all critical components of the current-day employee relationship.

Happy Employees = Happy Customers

For years, we have known intuitively that happy employees equals happy customers, which in turn leads to more growth and higher profits.

Back in 2002, Anne Mulcahy, the former CEO of Xerox – and the woman credited with turning around the ailing company – talked about what it was like to take the helm at a company with five consecutive quarters of losses, huge debts, and dismal employee morale. She talked about her philosophy of focusing on customers and employees:

“What do you really believe makes a difference in the company? For me it’s really clear. It’s about customers and employees. If you take care of your customers and you have motivated employees, everything else follows.”

More recently, Anne Mulcahy made a similar comment in an interview with Forbes magazine:

“Employees who believe that management is concerned about them as a whole person, not just an employee, are more productive, more satisfied, more fulfilled. Satisfied employees mean satisfied customers, which leads to profitability.”

Satisfied employees mean satisfied customers, which leads to profitability.”

Anne Mulcahy, former CEO of Xerox

 

Colm O’Neill is a Partner at KPMG. He previously ran the Corporate and Public Sector division at BT and talks about doing so with three simple metrics: employee engagement, customer relationship quality, and financial results.

In that order. Financial results are simply the outcome of happy employees and happy customers. Colm intuitively understood that point. Happy customers stayed with you for the long haul and didn’t churn. They also increased their spend with you over time, and were more profitable. For Colm, the starting point for long-lasting client relationships was the employee. To paraphrase Anne Mulcahy, if you get the employee part right, everything else follows.

If you don’t currently measure employee relationship quality, make sure you put a ‘Voice of the Employee’ system in place this year. Make that your #1 priority in 2026. You don’t have to use Deep-Insight’s ERQ measurement system. There are plenty of other tools out there. But if you don’t already have a robust system in place and want to have a chat about the best way to start, do get in touch with us.

Win/ Loss Reviews

Written by

Win/ Loss Reviews


The Importance of Win/ Loss Reviews

Here’s an uncomfortable truth: most B2B organisations don’t really know why they win or lose deals. They think they do. They have dashboards, CRM reports, pipeline analytics, and conversion ratios sliced and diced every which way.

But those metrics only answer one question: What happened. They do almost nothing to answer the far more important question: Why did it happen? 

And without understanding the “why,” all that impressive data is little more than expensive guesswork.

The Illusion of Insight

Most sales directors can recite their sales funnel numbers:

  • How many leads become qualified opportunities
  • What percentage reach proposal stage
  • What proportion of bids convert to a sale

Track these numbers over time and you can diagnose whether the issue is demand generation, qualification discipline, or closing effectiveness. CRM systems make this much easier today than in the past. In theory, at least.

In practice, data quality is uneven, sales teams prioritise revenue over administration (in other words, don’t count on the sales team to keep your CRM system up to date) and many organisations rely on Sales Operations or Revenue Operations functions to maintain order.

Today, sales leaders are quite good at measuring performance. They are far less equipped to explain it. They can tell you the company’s win rate dropped from 32% to 27%. They usually cannot tell you why.

A Discipline Hiding in Plain Sight

What makes this particularly frustrating is that the solution is not new. The solution — Win/Loss Reviews — have existed for over fifty years. Back in the 1970s, defence contractors and other large B2B firms involved in massive government bids began conducting rigorous post-bid reviews. They understood something many modern businesses still overlook:

A tiny improvement in win rate can deliver enormous revenue impact over time.

They didn’t call it “Win/Loss analysis” in those days — that terminology only became common in the 1990s — but they had already grasped the principle: If you want to improve outcomes, you must systematically learn from them. 

Yet despite decades of awareness, most organisations still treat Win/Loss Reviews as an occasional, informal exercise rather than a core business discipline.

Win/ Loss Reviews are Rare

One of the most influential book on the topic remains Win/Loss Reviews: A New Knowledge Model for Competitive Intelligence by Rick Marcet.

When Marcet, an American sales veteran with three decades of experience in technology sales, wrote the book in 2011, his central finding at the time was somewhat sobering:

Fewer than 10% of sales opportunities are ever reviewed with real rigour.

Even more striking, he observed that fewer than 5% of companies had any formal win/loss discipline at all — and those that did typically focused only on losses.

A decade and a half later, the numbers have improved slightly — but not dramatically. It’s hard to get definitive figures but the research suggests that even in large B2B companies – we’re talking enterprise organisations with annual turnovers of €1bn or more – as few as 10-25% of deals are reviewed rigorously.

That’s low. And it’s even lower for mid-market and SME companies, as the table below illustrates.

Factor

Large Enterprise

Mid-Market/ SME

Dedicated function

Yes (Sales Ops / Revenue Ops)

Rare

Formal process

Common

Limited

Data infrastructure

Advanced

Basic CRM

Cultural maturity

More analytical

More relationship-driven

% deals reviewed rigorously

10–25%

3–10%

In other words, even among large, sophisticated organisations, systematic learning from sales outcomes remains surprisingly rare. One of the biggest barriers to a wider adoption of systematic Win/Loss Reviews is cultural. We’ll discuss that shortly.

Relationships: The Reason Many Bids are Won or Lost

B2B organisations pride themselves on relationships. And rightly so — trust and credibility remain central to complex sales. Building strong relationships is probably still the best way to win bids. It’s certainly the cornerstone of renewing contracts, as I’ve discussed in previous blogs.

But Marcet offered a provocative warning that still resonates:

“An overreliance on relationships may actually mask mediocrity and underperformance in other areas.”

This is where many organisations get into trouble. Yes, strong relationships can win business — but they can also create dangerous blind spots. Teams assume they understand customers because they know them personally. They attribute losses to price pressure or procurement tactics instead of deeper issues.

The truth is that deals are almost never won or lost for a single reason.

What Really Drives Win/Loss Decisions

Proper Win/Loss reviews consistently reveal a far more complex picture. Relationships are critical but there are other factors at play too:

  • Product fit — The proposed solution may not fully address the client’s real priorities.
  • Perceived value — Price is rarely the sole issue; buyers weigh risk, confidence, and expected outcomes.
  • Competitive positioning — Reputation, differentiation, and credibility often matter more than features.
  • Service capability — Past delivery performance frequently determines future buying decisions.
  • External factors — Budget changes, mergers, strategy shifts, or regulation can override everything else.

Without direct customer feedback and proper analysis, these drivers remain largely invisible.

The Myth about Learning from Losses

Another persistent myth is that organisations learn more from losing than winning.

In reality, wins often provide richer insights. Understanding why a customer chose you — what tipped the decision, what risks they perceived, what nearly derailed the deal — provides powerful guidance for future success.

Both wins and losses are fertile ground for learning.

Ironically, organisations often focus almost entirely on losses — missing half the available insight.

Fewer than 5% of companies apply any formal win/loss review discipline to their sales outcomes, and those that do usually focus only on the losses.”

Rick Marcet


Interviews versus Surveys: Depth vs Scale

One reason why companies avoid rigorous Win/Loss Reviews is cost.

Independent buyer interviews can cost anywhere from €700 to €1,500 each. They require skilled moderation, recruitment effort, and structured analysis.

But the depth of insight they generate is unmatched. Seasoned sales professionals like Cian McLoughlin at Trinity are experts at conducting these in-depth interviews. As Cian says:

“For win-loss analysis to be truly effective, you’ve got to venture beyond surface-level feedback, to the hard truths that lie beneath. Customers will rarely volunteer this information, unless they believe you’re serious about taking action.“

Surveys offer a cheaper alternative and allow broader coverage — but typically suffer from lower response rates and more superficial feedback.

The most effective Win/Loss programmes will often combine both: surveys for scale; interviews for depth.


Why are Win/ Loss Review so Rare (and Where to Start)

The biggest barriers are not technical — they are cultural.

  • Sales teams fear blame.
  • Leadership attention moves quickly to the next quarter.
  • Ownership of insights is often unclear.
  • Learning from outcomes requires discipline and persistence.
  • As a result, Win/Loss Reviews remain ad hoc rather than institutionalised.

The good news is that organisations don’t need a large budget or complex infrastructure to begin.

Start by interviewing a small number of recent customers — both wins and losses. Even a handful of structured conversations can reveal patterns that internal teams never see.

Over time, these insights can influence strategy, messaging, product development, and sales effectiveness. In competitive B2B markets, even a modest increase in win rates can generate significant revenue gains.

 

One Final Thought

We know Win/Loss Reviews are not new. We know they are not complicated. Their value is not debated. Yet most organisations still fail to do them properly. Why?

Perhaps the real reason is this: They force businesses to confront uncomfortable truths about how they are perceived by customers. And that can be harder than building another dashboard.

But for organisations serious about improving performance, there are few more powerful — or more underutilised — sources of insight.

Contact us at Deep-Insight if you want to find out more about setting up Win/Loss Review programmes.

The B2B Blindspot: Why NPS Is Not Enough

Written by

The B2B Blindspot: Why NPS Is Not Enough


Why do so many B2B CX programmes fail?

Later this year, Bert Paesbrugghe will be hosting a LinkedIn webinar called The B2B Customer Success Blindspot: Why NPS Isn’t Enough. It sounds like it will be a good session and I have cheekily borrowed his title for this blog as it got me thinking about some of the reasons why B2B companies set up customer experience (CX) or Net Promoter Score (NPS) programmes in the first place.

More important, it’s worth reflecting on why these CX and NPS endeavours often fail to deliver on their initial promise. And that’s the sad truth – many of these programmes fail to improve the service delivered to customers. They don’t succeed for a variety of reasons. One of these is the belief that Net Promoter Score is a silver bullet for solving all manner of customer woes.

It’s not. That’s the blindspot. NPS is not enough for B2B companies.

B2B is different

The first thing to mention is that the Business-to-Business (B2B) world is VERY different to its Business-to-Consumer (B2C) counterpart.

The consumer world is all about the 4Ps: Product, Price, Place and Promotion. Marketing guru Philip Kotler popularised the 4Ps back in the 1960s. They were a core part of his Marketing Management book that many of us still have on our shelves today.

My only real problem with the 4Ps model is that it’s essentially a B2C concept. It doesn’t cover the subtleties of the B2B world where very often a service provider is delivering a very complex service across multiple locations – often in different countries. This is a world away from selling and marketing consumer products such as Mars Bars or Mercedes cars.

The 4Ps also don’t take into account the need for key/ global account management or the associated challenges of building and maintaining relationships with multiple decision makers and influencers across large global organisations.


NPS is one-dimensional

Net Promoter Score has proven to be one of the most durable metrics in management, ever since its invention by academic and business consultant Fred Reichheld more that two decades ago. Reichheld’s basic premise was that you only need to ask one question in order to understand if a customer is going to stay loyal to you or not. The question is: “How likely are you to recommend us to a friend or colleague?”

Fred, an excellent marketeer, promoted the benefits of his Net Promoter Score (NPS) concept in publications like the Harvard Business Review. He then proclaimed its merits in his 2006 book The Ultimate Question. Since then, NPS has became a hugely popular metric for customer loyalty and customer experience.

I’ve written about NPS before and, in general, I’m a fan of the metric for both its simplicity and its popularity. Sure, it’s not perfect, as Professor Nick Lee points out. But then again, is there a perfect KPI for anything? Let’s agree that Net Promoter Score has its place and is worth measuring even if it is a little one-dimensional. 

So NPS is good, but much more is required, particularly in the B2B world with all of its complexities, peculiarities and challenges.


Why NPS is not enough (in B2B)

Let’s go back to basics here. B2B IS different. So let’s recap on what some of those differences are:

  • Customer Base. Consumer brands like Mars Bars and Mercedes cars are sold to millions of individuals. Three million sold every single day, in the case of Mars Bars. In contrast, we work with B2B clients that generate annual revenues of more than €1bn from fewer than 100 clients.
  • Value. A Mars Bar costs around €1.60 at the time of writing (let me know if you can source them cheaper!) while an outsourced IT contract can be worth €100m. Admittedly, a Mars Bar can be consumed in less than five minutes while a €100m contract might take five years to consume. But you get the picture: price and Value For Money are very different in the B2B and B2C worlds.
  • Marketing Strategy. We talked earlier about Kotler’s 4Ps. While the consumer world is all about Product, the B2B world is more around Service and Relationships. Even in today’s AI-enabled world, those services are still delivered by people. Relationship-building is a critical component of the marketing mix the B2B world.
  • Sales Focus. In the consumer world, merchandising and point-of-sale advertising are key. In the B2B world, far more emphasis is placed on educating the customer about features, benefits, return on investment, and so on. This is still mainly done through personal contact and relationships.
  • What to Maximise? The consumer world is about the transaction – promoting those Mars Bar at the point of sale, for example. Customer lifetime value (CLV) is rarely if ever mentioned in the consumer world. CLV is arguably the most important thing to maximise in the B2B world as it typically takes 2-5 years to recover the initial sales cost of a major multi-year contract win.
  • Buying Process. In a supermarket, buying a Mars Bar is a split-second decision. Even for a Mercedes, the decision can be quick. Clinching that 5-year outsourcing deal can and does take years from beginning to end. It also involves multiple decision-makers and influencers.
  • Buying Decision. In the consumer world, decisions are often made on emotion – hence the importance of brand and image. In the B2B world, we like to think decisions are made on rational grounds, based on cleary-defined evaluation criteria.


What else is needed?

Let’s assume we have just sold a 5-year outsourcing deal to a client and we are now in the onboarding or delivery stage of that contract. Yes, it’s useful to know if our client would recommend us to a friend or colleague. That’s the Net Promoter question, but is it enough?

Not really. Ideally, we need to know much more. For example, do our clients trust us now that we have started working for them? Are they committed to us for the long term? Are they happy with the service that they are now receiving? 

These are just some of the questions that we need to ask our B2B clients in a systematic way. We need answers at an aggregate level but we also need feedback at an account level. Contract A may be going swimmingly. Contract B may already be on the rocks (to continue the theme) but we might not know that if we are only getting aggregated client feedback.


Eliminating the blindspot: Customer Relationship Quality (CRQ)

An alternative to asking the one-dimensional NPS question is to view the customer relationship more holistically. That’s where Customer Relationship Quality (CRQ) fits in.

CRQ can be visualised as a pyramid comprised of three different levels.

  1. The first and most fundamental is the Relationship level. Do your clients trust you, are they committed to a long-term relationship with you, and are they satisfied with that relationship?
  2. The second is the Uniqueness level. Do your clients view the experience of working with you, and the solutions you offer, as truly differentiated and unique? Do they see us as good value for money?
  3. At the top of the pyramid is the Service level. Are you seen as reliable, responsive and caring? Get this wrong and you will never be seen as Unique and you will struggle to build a long-term relationship with that client.

Interestingly, CRQ and NPS scores are highly correlated. If you score well on all six elements of this Customer Relationship Quality (CRQ) model, your clients will act as Ambassadors, generating a high NPS result for you. However, CRQ gives you so much more information to act upon, and that’s far more important.


The most important part: Action

The CRQ model above was specifically designed for the B2B world. That said, it really doesn’t matter what questions you ask your clients if you fail to do anything with their feedback.

The most important part of any NPS, CRQ, CX or client listening programme is the ‘Action’ piece. The reason that many  customer programmes fail to deliver is that they are run by the Marketing department (sorry guys and gals!) while the members of the company’s Senior Management Team have collectively washed their hands of any responsibility for acting on that client feedback.

In most B2B organisations, key client relationships are owned by Sales. In some cases where delivery is an ongoing function, it’s the Service or Operations functions that have most of the day-to-day client contact. It’s rarely, if ever, somebody from Marketing. The Sales Director (or Service/ Operations Director) needs to own the ‘Close The Loop’ element of the programme. It’s unfair to expect Marketing to take responsibility for it.

Put it another way: it’s madness to think that Marketing can effect change on its own. That’s a Leadership function. I’ve never seen a successful NPS ar CX programme that has not been driven from the top. So regardless of what you think of NPS as a B2B metric, don’t assume that NPS or any other set of survey questions is going to improve your top line or your profitability. It won’t, unless there’s follow-up action. That action needs to be managed systematically, and it needs to be driven by the  SMT or Executive Team.

Finally, do remember that it’s not about the score. It’s about using that valuable client feedback to take action and become more customer-centric. That’s how you generate more revenues and boost profits.

What Does Net Promoter Score Actually Measure?

Written by

What Does Net Promoter Score Actually Measure?

Interview with Professor Nick Lee

Some weeks ago, I met Nick Lee, Professor of Marketing at Warwick Business School to discuss his views about Net Promoter Score (NPS). I specifically wanted to get Nick’s views on NPS as a measurement tool. Does it work? Is it linked to sales growth? What does Net Promoter Score even measure?

Professor Nick Lee

Nick has an impressive background. In fact, The Academy of Marketing has already honored him with Life Membership in recognition of his outstanding lifetime research achievements and contribution to marketing scholarship. He is currently the Editor-in-Chief of the Journal of Personal Selling and Sales Management (JPSSM), which is the premier journal for research in professional selling. He is the first UK academic to hold this position, and only the second ever from outside the US. Nick was also the Editor in Chief of the European Journal of Marketing from 2008-2018. 

Nick is more than just an academic. He holds strategic advisor positions for a number of innovative sales and leadership development companies, and he was part of the All Party Parliamentary Group inquiry into professional sales in 2019. His work has been featured in The Times, the Financial Times and Forbes, and he has appeared on BBC Radio 4, BBC Radio 5Live, and BBC Breakfast television. 


Net Promoter Score: 20th Birthday

What prompted me to interview Nick Lee was a recent academic paper he was involved in called “The use of Net Promoter Score (NPS) to predict sales growth” which was published in the Journal of the Academy of Marketing Sciences (JAMS) in 2022.

Our discussion was very timely as Net Promoter Score is a metric that was invented by Fred Reichheld, a partner at Bain & Company, 20 years ago this year. 

Over the past two decades, NPS has divided opinions. While it has been embraced enthusiastically by many businesses, it has been shunned by others. The academic world has questioned what Net Promoter Score actually measures.

I think you’ll find Nick’s comments on Net Promoter Score and what it really measures quite fascinating because he doesn’t hold back from his criticism of NPS but also points out that the flaws don’t invalidate its usefulness as a measurement system, as long as it’s used in the right way: tracking changes over time, rather than simply chasing a number. 


The Interview

John:

Good morning Nick. To start, could I ask you to tell me a little about your own academic background. What was your first interest in the field of marketing?

Nick:

Well, I began my academic career as a doctoral student in marketing strategy. It seemed to me that the connection between sales and psychology was quite important. And there was a lot of work in management that was related to psychology, but very little of that research had been focused on the sales force.

A lot of sales research is actually about things like incentive structures and territory design. I call that ‘technical management’ but what I was more interested in was not so much the decisions that managers made, but how they implemented those decisions. Sales management is more about psychology than a mathematical or technical thing. More recently, we’ve seen how digital transformation has led to a a merging of the ‘technical’ things with the more psychological things, and that’s really the space I operate in now.

Is NPS a Fundamentally Flawed Metric?

John:

So let’s talk about the psychology of Net Promoter Score. It’s clearly a sales and marketing concept. It’s also a performance metric. When did you start getting involved with Net Promoter Score and is it a good sales and marketing measurement tool?

Nick:

My interest in NPS really came from Sven Bähre and that paper we wrote called The use of Net Promoter Score (NPS) to predict sales growth. Sven drove that project while my role was to use that project to address something that was important to the marketing literature. And I think it is very interesting that academia’s gone down one road with Net Promoter Score, the very simple road which says “NPS is useless and a load of rubbish”.

At the same time, business practice has completely ignored that academic view. Net Promoter Score has become the dominant customer metric in business. It feels like someone has to be right and someone has to be wrong here. But the interesting thing is it turns out that both sides are right. They’re just talking about different things. And that’s what fascinated me.

John:

So tell me about those different things. When I looked at Net Promoter Score many years ago, a guy called Tim Keiningham – one of the people you refer to in your paper – was very critical about NPS as a metric. At the time he worked at Ipsos so I wondered if he was bringing his own biases to the table. But at the same time, he was saying that the data did not show any link between NPS and sales growth.

Nick:

Oh, that’s interesting about Keiningham, I didn’t know that he was at Ipsos then. So there are a couple of issues that lead to this disconnect. One is that we generally don’t like it when a publication like the Harvard Business Review tells us there’s a single number that every company needs to look at. That automatically gets people’s interest and it actually didn’t make sense to me. 

The other issue is, and I have every sympathy with this view, is that Net Promoter Score doesn’t really measure what it claims it measures. There are so many potential flaws in the idea that this one number could be a valid measurement of anything real. I spend a lot of time trying to develop measures around attitudes and psychological concepts. And this is a classic example of a metric that doesn’t seem to actually ‘measure’ anything. So on that basis, it is quite flawed.

When you add in the idea that you have to subtract the bottom scores (Detractors) from the top scores (Promoters), you're torturing the measure to within an inch of its life.


What does NPS Actually Measure?

John:

Surely Net Promoter Score is a measure of advocacy, if nothing else?

Nick:

To some extent it taps into advocacy, sure. However, it’s a number in response to a single question that doesn’t take account of all kinds of other factors that might be relevant. And then you have this weird calculation for subtracting ‘Detractors’ from ‘Promoters’. As a mathematical construct, that’s not great. But the real issue is that advocacy is a much more complicated idea and can’t really be accurately captured by a response to a single question.

So there’s no real evidence that that answer to the NPS question is a reliable measure of advocacy. And then when you add in the idea that you have to subtract the bottom scores (Detractors) from the top scores (Promoters), you’re torturing the measure to within an inch of its life. At that point, it ceases to become a measure anymore, even if it was at the beginning. It becomes a number which is divorced from the underlying concept.

John:

I’m with you. But does that invalidate it completely as a measure?

Nick:

Well, here we get to the bigger question: rather than “is NPS a measure”, we need to ask “is NPS actually useful?” In academia. we’ve said nothing about NPS for the last 20 years apart from “it’s crap”. But when I see a whole bunch of senior executives in large companies saying “well, I’m finding a use in it”, then academia needs to look at that.

What’s my conclusion? I think that is a problem for academia insofar as we tend to talk past each other in a lot of areas. We have to provide some insight into what practitioners are doing in this field. Of course, it’s our sole driving force as a discipline to find out what practice is doing and we study that. But at the same time, it is worth studying if the entire business community is using something that 20 years ago we in academia said was wrong. 

John:

Have you come to any conclusions as to why senior leaders use Net Promoter Score, or how they can use it more effectively?

How Should NPS be Used?

Nick:

One reason why it’s used so much is partly a self-fulfilling prophecy. It’s used because everyone uses it and therefore nobody wants to not have that information. That’s an important factor.

But then the other aspect is that it’s used because it’s simple. It’s easy to collect and it’s simple to use. Whether it’s easy to interpret is actually a more challenging question. I don’t think it is that easy to interpret. For starters, what does the NPS number actually mean at any given point in time?

Now when you start tracking NPS over time, those questions fall away because what you’re looking at is trend data. What was it last year? Is it up or is it down? You have to operationalise NPS in the right way – by tracking the change in NPS score from one period to the next, not the absolute score. That’s important.

Net Promoter Score is also influenced by a lot of transient factors. For example, it’s very easy to manipulate and there’s a big selection bias. Who is asked to complete the NPS survey? Also, every surveyor cannot help but lead the customer on towards a higher NPS score. So at any given point in time, the net promoter score doesn’t mean much because of that selection bias. But if you assume that those forces are broadly the same over time, you can extract that little bit of signal from that noise with the time series a little bit more effectively than at a single point in time.

John:

I’m with you. But in our experience, the level of bias can increase over time. So you need to have a governance structure to ensure consistency. Or indeed, you may need to break the system apart and start again if the ‘gaming’ gets too deeply ingrained.

We should track trends, not individual time points. And the more data we have, the better. More bad data isn't better than less good data. But more flawed data is probably better than less flawed data.

Nick:

Yeah, I think you’ve got it right there. It is important not to be naïve that over time there might be an ‘instrument effect’ or a ‘history effect’ where people learn how to better game the system. I think with something like Net Promoter Score, that’s less of a thing because it’s pretty easy to know how to game the system straight away. And the only thing you can do really is try to say to your customers: “this is really important to me, can you please leave me a good score?” And there’s only so convincing you can be there. It’s not like you’re going to get better at doing that after a certain point in time. So I would be less worried about that.

But of course there are always ways to game the system. But the point is we should track trends and not individual time points. And the more data we have the better. Of course more bad data isn’t better than less good data. But more flawed data is probably better than less flawed data. So given we assume the data is flawed all the time, the most important thing is to know how that data is flawed. And while you can never perfectly extract the signal from the noise, the signal is there if you have enough data points gathered over a sufficiently long period of time.

Where Do We Go From Here?

John:

So where do we go from here, and where should academics be focusing their efforts?

Nick:

A few things for us to work on. First is international comparability. Big multinationals use Net Promoter Score across their different national areas. And I would imagine they’re comparing EMEA with America with Australasia. Is NPS really able to support that comparison? That’s a challenge so that’s the first thing I would look at.

Second is to move away from a single question. We really need multiple measures in order to compare them statistically across different cultures. So Net Promoter Score is one item. You would like it to be three or four items. And then you could compare those items across countries.

A third area is to get a wider industry perspective. We looked at NPS in a branded consumer goods context: sportswear. Is it equally useful across all kinds of different industry sectors? Particularly if you look at the service sector and front line services, which are linked to business to business (B2B) personal selling. Is NPS a useful metric for these interpersonal interactions? How well does it work in a B2B setting?

John:

Nick, I really appreciate your time today. I’m looking forward to seeing more research into Net Promoter Score. From a selfish perspective, I’d particularly like to see some B2B research done as there’s very little out there that I can find on the topic. Thanks again, Nick.