The Underestimated Importance of Soft Skills in Data Science

I once read that Elon Musk succeeded with soft skills. At first, I thought it was hyperbole, but then I figured that you probably wouldn't want to be the first one to take humanity to Mars if you weren't resilient, you wouldn't be able to attract the best talent and partners without inspiring storytelling.

But do data scientists really need soft skills? Turns out, they do.

Soft Skills Are Essential

Although the absolute majority (according to some studies as much as over 90%) of employers say that soft skills are very important or essential, job applicants rarely include them in their CVs. So either employers are not clearly indicating the soft skills they require, or workers do not understand the value of them.

Back in 1918, Harvard University, Carnegie Foundation and Stanford Research Center concluded that 85% of “job success” for engineers depends on soft skills. Who are we to question such data, but it's not always the gold standard. What constitutes “job success” is entirely subjective and depends on which industry you are talking about.

If your idea of success means to be the best Python programmer in the world, maybe a smaller percentage of soft skills would do the job. If you want that "junior-mid-senior-lead-director-VP-CEO" career, 85% sounds quite realistic.

Soft Skills for Data Scientists, and Why They Need Them

When it comes to data scientists, in most articles on the internet you will find the following mentioned as crucial attributes: excellent communication, critical thinking, storytelling, the ability to work in a team, adaptability, knowledge of your brand, and an enduring sense of curiosity.

When you think of it, these qualities apply to almost any professional, the same way analytical skills help in any role. Who among us has a problem with critical thinking these days? Or storytelling in the era of storytellers? Or the ability to work in a team, assuming you have not yet gone full Henry David Thoreau and live alone in the woods.

The soft skills needed for data scientists are almost the same as many other specialties. Why? As Deividas Skiparis, ​​Turing College instructor and Data Scientist at Vinted, says, just look at what a typical day for a data scientist entails.

"Breakfast —> standup —> a/b test analysis and presentation to the team —> new features analysis/research and discussions —> table football match —> data requests from other teams —> work prioritization, because everyone needs your help —> eat/ sleep/repeat... Almost all steps require those soft skills and technical/analytical work probably takes less than half of the working time. Soft skills are no less necessary for data people than the ability to analyze data", says Skiparis.

However, as Skiparis says, one of the soft skills that is rarely pointed out is listening.

"How often do data scientists start thinking about a solution after hearing the first few sentences about a problem? I do that and then I punish myself for not listening fully, for not understanding the details, for having to go back and ask what was discussed while I was constructing my SQL query in my head... Consultants are taught not to jump too fast into solutions, first clarify the problem and only then look for solutions, in other words – to be able to listen. This should also be learned by every data person", says Skiparis.

Skiparis also distinguishes attention to detail as a key skill. According to him, this also applies to any other specialty, but is particularly necessary for data scientists. "In my opinion, it is the small things that matter: can you notice any anomalies in the data? Are there any gaps in the storytelling? Are the color palettes consistent? Are the visuals neat? Is the code easy to read? This can greatly affect the quality of the work and influence further steps," says Skiparis.

Since data scientists are constantly faced with situations where it is necessary to make a decision that will determine the trajectory of the project, decision making is another extremely fundamental skill.

"It's very easy to find yourself in situations where to make a trivial decision you could spend another 2 weeks analyzing and thinking, but you only have 2 hours. For example, in data modeling, you need to choose the best features for your model. It seems trivial enough: we all know how to create a model and look at feature importance with N different tools. But when it comes time to clearly communicate to the developers, I often start asking myself if I really did everything to find the most optimal solution? Will it really be good...? And if not...?", Skiparis explains.

According to Skiparis, when headhunting prospective employees, he takes into account general soft skills, which are necessary for most specialists. During a job interview, it is difficult to assess, for example, an individual’s ability to work in a team, their adaptability or brand knowledge, but certain questions make it easy to notice that some skills are clearly absent, which is already a major red flag.

"But this, in my experience, happens quite rarely," he adds.

The term "soft skills" is dynamic, meaning that it evolves and is reappraised quite often. A different skill set is needed now, for example, to that of 20 years ago. According to Skiparis, the art of negotiation will be increasingly appreciated in the future.

"I think that it is time for people to learn how to negotiate: from getting the conditions from your job offer, to working with competing demands, to compromising for the simplest and best solution in a project will be (and maybe already is) one of the more important skills," he says.

Can You Learn Soft Skills?

It's true that soft skills can certainly be harder to learn, but our brains are capable of acquiring cognitive, emotional and social intelligence. And we have a lot of potential! Neuroplasticity suggests that we can learn even the most complex things, but there are no shortcuts for that – it simply takes a lot of time and a lot of work. Malcolm Gladwell had a great point suggesting his 10,000 hour rule.

There is growing scientific evidence that you can learn all kinds of soft skills. Let’s take communication for example. Our brains are wired to hear what we expect to hear. However, there is already data that suggests we can train our listening (and hence our communication) by paying close attention and actively listening. Programs designed to improve teamwork and collaboration show positive outcomes, at least in the short term. Even empathy, which would seem to be a really intangible quality that everyone has a different understanding of, can be improved.

Attention to detail can also be learned – although the internet is full of advice that’s not backed up by evidence, even a simple habit such as giving up multitasking (sorry, our brains can’t multitask) will noticeably improve our attention.

According to Skiparis, we learn certain soft skills at work, and others by communicating with our significant other, friends, parents, children.

"However, unlike tech skills, soft skills do not have a standard – every manager is different, every story teller tells differently, everyone's communication is different. I think that when starting to learn something so abstract, it is much more important to learn how not to be at the beginning." So first, learn what kind of a leader/employee/parent/spouse you don't want to be. And, over time, supplement this skill set with life experience and personality traits.

Learning does not take a year or two, but a lifetime. So a 10-hour Coursera course will definitely not be enough – but it doesn't mean you shouldn't sign up for it. And while you’re here, you can also join us at Turing College because our courses are heavily based on mentorship and the development of soft skills.

Author: Goda Raibyte