Your Instinct Still Matters — Why Data Alone Isn’t Enough to Drive Decisions

Netflix’s Reed Hastings leans on it. So does SoftBank founder Masayoshi Son. As did Howard Schultz when he ran Starbucks. It has been well-documented that these renowned business leaders have relied on one thing to drive their business decisions: gut instinct.

They’re not alone. Oprah Winfrey once said mistakes happen when she doesn’t listen to that “still, small voice” within her. Richard Branson, who became the first (but amazingly not the only) billionaire to venture into space, has admitted to relying more on gut instinct than research and statistics. 

But there’s also no question that businesses increasingly rely on data to make their most important decisions, with the rapid proliferation of artificial intelligence (AI) spurring analytics and algorithms, making data-driven thinking that much easier to scale. How, then, do we reconcile this apparent contradiction?

“Data versus intuition is a false choice. You need both,” said Wilfried Schobeiri, chief technology officer at Banyan, a New Jersey–based data marketplace. “The best decisions tend to be those where data supports or enables instinct and intuition to come into play – and are then followed up to measure the impact and success of that decision.”

More executives invest in AI

Just a few years ago, only a little more than one-third of executives trusted their organization’s use of data and analytics, and about 67% had ignored computer-generated data because it contradicted their own intuition, according to a global CEO survey from KPMG. 

But a survey by Cognizant and ESI ThoughtLab highlighted the need for more data-driven AI because of the digital transformation that exploded throughout the global coronavirus pandemic. That survey’s results showed two-thirds of executives believe AI plays a considerable role in the future of their businesses. For executives who have already implemented AI in their business, faith in the technology skyrockets to 98%.

Executives are not buying into the hype for nothing. AI is able to process information and predict trends with levels of accuracy that a single human can only dream of. The truth is, one person, even with the most refined executive intuition, can only process so much information at a given moment. AI allows for infinite combinations to be assessed at once.

While AI infinitely speeds up the process, is there a cost of losing the human touch when it comes to important decision making? 

Data represents people

It’s often too easy to read datasets, see that data as a bunch of numbers, and divorce those numbers from the flesh-and-blood customers they stand for on the page. That’s why, when thinking about your business, it’s important to remember not to make decisions based on growth and making money alone. You need to consider how your decisions and processes affect actual people, whether your customers, employees, or the larger public. Insights derive from how algorithms analyze the data, so weighing a human element in your decision making becomes equally important.

“If you’re only leaning on data, you’re not taking into account human condition, experiences, and feedback,” said Melissa Doman, a Denver-based organizational psychologist and former clinical mental health therapist. “I suggest leaders take in all the data and then slow down, ask how these decisions are informed by the data, how they’ll land with people. What is the action you want them to take? What is the new thought you want them to have? If you don’t pause to consider how that data will drive behavior, the data will fall flat.”

We can look at this from a medical perspective. More and more, healthcare professionals rely on data to help drive decisions. A doctor can use diagnostic and prescriptive data to help determine a patient’s illness or future health risks based on personal, psychographic, and demographic data. But when it comes time to put together a plan to tackle an illness or make recommendations, shouldn’t a doctor use her own experience and knowledge to help guide the patient? 

“Although we are blessed with rich data,” said Tracy Allison Altman, founder and executive director of the Museum of AI, “a crucial leadership skill is developing instincts for incorporating facts into decisions.”

Where bias plays a role 

Some leaders wrongly assume that if they simply grab more numbers and continue to follow the data, everything else in their business will magically fall into place. However, this approach is lazy, especially when the data grab isn’t followed by a hard look at what the numbers represent and how they shape society. Most of all, the constant chase for more data is a form of procrastination, giving leaders a legitimate excuse to buy more time and weigh the options. 

“There are people who can’t get enough data, and they just keep exploring the data,” said Anil Cheriyan, executive vice president of strategy and technology at Cognizant, a business consulting firm. “As a result, they don’t make any decisions.”

For the data chasers, more numbers and figures means less of a chance they’ll make a regrettable or subjective decision. Yet while a leader’s personal read of a situation may be riddled with biases and misconceptions, so may be seemingly objective datasets.

In theory, while AI gets more accurate and smarter the more it does its job, someone has to initially create the program that AI runs on. Despite our best efforts, underlying bias is often built into the AI algorithms, which results in discriminating against whole datasets – which, as we said, represent people. Even on results not riddled with bias, people tend to project their own preconceived notions.

“Oftentimes, people don’t listen. They hear instead what they’re expecting to hear,” Cheriyan added. “The patterns they’re seeing are likely prejudiced based on what they are used to seeing.” 

With time, however, business leaders can start to better understand the data and the patterns and integrate their own experience into the fold to properly make the right decisions. 

“Mature decision making requires defining a repeatable process,” Altman said. “The process should bring in meaningful data and also the intangible, anecdotal, and qualitative information influencing a decision. It’s not really gut instinct; it’s a highly sophisticated cognitive exercise that should not be underestimated.”

Many times, it’s not a decision that one person should singularly make. 

Lean on your trusted advisers 

Does the buck stop with the CEO or team leader? Yes, but getting to that place usually requires the confidence gained by having consulted with close advisers. That way, you’re reviewing data but also taking into account different subjective perspectives, helping to ensure blind spots get accounted for. Once you have data you can trust, discuss it. Come at it from different perspectives, and come to a result based on the conversation. 

“Seeking out data and analyzing it from a place of neutrality or intellectual honesty can lend credence to intuition or act as a nudge away from what may end up being a poor business decision,” Schobeiri said. “Intuition is also highly personal. What may seem obvious or self evident to me may seem without merit to you. How do we value my intuition against yours? In most environments, the answer is usually consensus.”

If you try to answer all the questions on your own, you’ll either get buried in data or give yourself an ulcer taking on all the responsibilities. This is where having trusted people around you will help in making those tough decisions.

“Different people have different strengths that need to exist in concert with other people’s strengths,” Doman adds. “There is a reason diversity of thought is the foundation of successful teams.”

If you think of data and intuition as different people on the same team, the results will come much more easily – and effectively. And when it comes to making the big decisions, you know you will have weighed everything equally.