In modern organizations, data drives decisions, but when it comes to employees, the real challenge may not be the decision, but the data used (or not used) to make it.
If it seems like labor trends are changing faster than the weather, it’s because they are. In one breath, leaders are focused on attracting talent, and in another, how to right size when the economy turns. The only constant is talent strategy whiplash where people leaders get caught reacting to one issue after another.
For years, organizations have been relying on engagement surveys as a key indicator of employee sentiment and productivity. According to McKinsey research, “employee disengagement could cost a median-size S&P 500 company between $228 million and $355 million per year in lost productivity.”
The lost productivity fear has driven organizations to measure engagement at record rates, making it an almost billion dollar industry in 2024. But do these measures actually make a difference? Researchers have been asking this question for years, and yet leaders still struggle with what to measure.
The concept of an annual employee engagement survey is antiquated,” says Anna Lyons, Chief Talent Officer for Alegeus, a Boston-based fintech company. “Employee sentiment changes by the minute, not by the year. For employee engagement surveys to be valuable, they need to provide continuous, real-time data and, more importantly, insights.”
Lyons is not alone in her assessment. Over the past few years, HR technology companies have been responding with in-the-moment pulse survey options adding to the mountain of data and dashboards companies already have to make sense of on a regular basis.
Is the answer more data, or as Lyons suggests, the insights? HR and company leaders are drowning in data – from HRIS demographics to dashboards tracking candidate career site clicks and pulse survey responses to employee sentiment. As leaders add to their stacks there is no shortage of automated and AI-driven promises for real-time data in just a click.
But more data doesn’t solve business problems. And that may mean simply answering the question ‘are my employees engaged?’ may be all wrong.
Here are five questions your organization should be asking instead:
1) Are we getting the right data?
Most organizations use a survey approach because it’s easy to get critical mass, and report back in numerical fashion. Percentages make for great sound bytes and may offer more immediate intelligence. But they don’t tell the whole story.
“The concept of a survey being a substitute for a conversation is ridiculous and should never be the case,” says Jeffrey Lackey, CEO of JKL Advisors. Lackey, the former Vice President of Talent Acquisition for CVS Health, is an advocate for a holistic data set. “Leaders should take the time to have listening tours and focus groups as a regular part of their management routine.”
Beyond just the comments on a survey, qualitative methods like focus groups and diaries allow leaders to understand why employees feel a certain way. Just like academic researchers dive deeper to understand the root cause, so should people leaders.
2) Are we measuring the right thing?
Engagement surveys can solicit a vast amount of data, but they’re confused in purpose: measuring employee sentiment or manager performance? This can be very confusing to leaders who are left with determining how performance management and engagement relate.
Additionally, most organizations have distinct performance management processes so managers get measured in multiple, incongruent ways, and may not be consistently evaluated or rewarded as a result. Engagement scores are often broken down by function, department or team, and managers can be rewarded or penalized for things they can’t control or impact.
“They are the same questions year after year so that managers can get a “grade” and encourage their leaders to ‘hit their numbers,’ says Lackey who says this is the wrong attitude for leaders.
If you’re measuring engagement, you should focus on the relationship employees have with work and how the manager relationship impacts that, rather than how the manager is performing vis a vis engagement.
3) Are we accounting for business reality?
In consumer market research, customer sentiment is measured constantly. Companies like Apple are collecting data on user experience all the time. There’s a reason for this – the world, and our reaction to it – are constantly changing.
Jesper Sørensen, Senior Associate Dean at Stanford talks about this challenge of research versus reality: “one of the hallmarks of research is trying to isolate particular mechanisms through various kinds of control. Scientists live in the world where it’s a vacuum: and so we just watch the leaf fall and we can then time it and then tell you what the answer is. And managers live in a world where the wind is blowing and there’s all these kinds of forces getting in the way.”
Employment is a relationship. How engaged or happy employees are can change every day based on what’s happening around them and annual measures can be outdated in the weeks and months after they are collected. Because of this, the antidote to the annual survey as a lagging indicator became real time feedback in the form of pulse surveys for many firms.
But fast and frequent methods of data collection have their downfall too. Quick data sets provide real-time, regular outputs but don’t have room to ask employees why they feel a certain way and then probe into their answers. They don’t allow for qualitative insights and they leave the ‘why’ up to leaders to assume.
Imagine if marketing leaders didn’t find out why customers weren’t buying their latest product, or worse, didn’t have a way to triangulate that data with what’s happening in the world around their customers.
4) Are we relying too much on technology?
Those real time outputs and the accompanying dashboards are really attractive to leaders looking to be more nimble and efficient. Technology is more advanced than ever in its ability to collect data at lighting speed and display it in ways we could never do ourselves. And artificial intelligence (AI) and machine learning (ML) are game-changing for leaders seeking real-time data and insights.
Like many leaders, Anna Lyons of Alegeus sees the potential. “AI /ML is positioned to derive insights, connect the dots, and allow leaders to respond with speed to what matters.” says Lyons.
But are our tech stacks too bloated with the promises of business-changing technology? Beyond extensive cost, platform integrations, and implementation time, it’s important to consider what we’re really missing in understanding employees when we rely so heavily on digital intelligence: feeling.
“Humans tend to be superior to AI in contexts and at tasks that require empathy,” according to Michael Bennett at The Institute for Experiential Artificial Intelligence at Northeastern University. “Human intelligence encompasses the ability to understand and relate to the feelings of fellow humans, a capacity that AI systems struggle to emulate.”
So while faster data and the promise of insights on demand seems attractive, we may be missing a crucial element to the employment relationship: feeling. Understanding the relationship between engagement and productivity requires an ability to relate to the challenges or obstacles that may get in the way of employees’ success. And even for leaders who started on the front line, they’re not there now and can’t relate to employees’ current feelings in the current climate.
5) Are we making any business impact?
In all of the discussion and debate around the value of measuring employee engagement, one thing is clear: we have to be able to drive change in our businesses.
At Virgin Voyages, Head of People, Michael Peterman is looking for just that. Peterman also feels like engagement surveys are too focused on a high or low score.
“Leaders can applaud or cry over their results, but rarely does it drive a company toward a true culture of engagement,” he says. “While understanding sentiment is helpful, I need more actionable guidance. What are the most impactful systemic changes I can implement in our organization to significantly improve our relationship with crew members?”
This kind of improvement only comes with a nuanced or custom approach to measurement. And that starts with going beyond engagement to measure the whole of the employment relationship. This means collecting real-time insights, but also tracking those over time to see how what happens in the business changes the strength of that relationship.
The custom piece is also crucial. When we measure towards arbitrary benchmarks or ask employees the same questions as every other company (because a survey platform requires us to do so), it strays from a company’s individual business strategy and needs.
Finally, let’s not forget that even in the age of AI, we still need humans to work. And understanding what makes them engaged and productive, requires the empathy that only comes from a deep dive into how employees feel and why, understanding what has changed their perspective or behavior now, and how those feelings may change their decision to stay or leave the organization.
The best part? This approach doesn’t require extensive technology, a year-long plan for survey deployment, or a massive investment. It just requires a new mental model for how we think about employee engagement.
After all, if your colleagues in marketing are measuring the customer relationship this way, why shouldn’t we be doing it with our people?