For the past few years, IT leaders at a US financial services company have been struggling to hire data scientists to harness the increasing flood of incoming data that, if used properly, could improve customer experience and drive new products. To deal with this hiring problem, they’ve had to get creative.
They’re not just offering remote work and flexible hours. Oh, no. They’ve also created a relationship with universities, setting up a pipeline of emerging technology-focused interns, who work at the company, gain experience in data science, and then can potentially be hired after they graduate.
“Even though we’ve seen a huge proliferation of data, the supply for analysts does not meet the demand,” says Bess Healy, senior vice president and CIO at Stamford, Conn.-based Synchrony, a financial company with more than 18,000 employees and $62 billion in deposits. “We try to be data-driven in our decisions so we have a great need for analytics skill sets. … We have a very focused approach around building that pipeline.”
Synchrony isn’t the only company dealing with a dearth of data scientists to perform increasingly critical work in the enterprise.
Companies are struggling to hire true data scientists — the ones trained and experienced enough to work on complex and difficult problems that might have never been solved before. And hiring them becomes far more difficult if the company isn’t the biggest brand or the biggest name. Finding and retaining IT workers, in general, has been difficult for a while now. It’s exponentially harder when it comes to data scientists.
Recent research from industry analyst firm IDC showed that there are 210,000 data science jobs listed on LinkedIn. The research report also noted that top enterprises, such as Deloitte, Amazon and Microsoft, are looking to fill a wide spectrum of technical jobs but data science far outweighs all other roles.
That is backed up by a 2021 survey by industry analysts at Forrester, which showed that, of 2,329 data and analytics decision-makers worldwide, 55% want to hire data scientists. Another 62% said they plan to hire data engineers, and 37% are looking for machine learning engineers — data analytics team members who could support data scientists.
“Data scientists have been in high demand because they have the alchemy to turn data into insights,” says Brandon Purcell, vice president and principal analyst at Forrester. “It’s very mystical to the layperson. They have a very specialized skill set. There’s been a trickle of people entering this profession. It’s more than it was before. But it’s still not much.”
The makeup of an enterprise’s data science team also has been changing.
Data scientists have extensive academic backgrounds — often in computer science, statistics, and mathematics. They specialize in building powerful algorithms, and analyzing, processing, and modeling data so they can then interpret the results to create actionable plans. But for years these specialists also found themselves in jobs where they were expected to do engineering work, such as building pipelines and embedding models into operations systems. This has left data scientists not only bored but also frustrated that they weren’t focusing on the core work they have been trained to do.
Now companies are catching on to this frustration, and in an attempt to attract new data scientists and retain the ones already on their teams, they are expanding the roles on the analytics teams.
Enterprises increasingly are bringing onboard data engineers, who can handle work such as building ETL pipelines, preparing data, and making it available for data scientists to analyze. And machine learning engineers are being hired to design and build automated predictive models. These people are making up a data science support system.
“What was called a data scientist three years ago is now [split up between] a data scientist, data engineer, and machine learning engineer,” Purcell says.
