Data science was termed the sexiest job of the 21st century by Harvard, but in a surprising turn of events, data Engineering may overtake this status. “The dearth of data engineers will be felt even more in 2022”, noted a survey by AIM on the top AI and Data Science trends for 2022. This is owing to the increase in digital transformation after the pandemic and the explosion of data following it.
Historically, data engineers have only dealt with distributed systems and Java programming, but now they have to leverage AI, ML and BI to manage data.
“There has not been enough emphasis on data engineering in the past. Data science has been considered sexier and more aspirational while data engineering has been regarded as the dirty part,” said Nidhi Pratapneni, SVP, Product, analytics & Modelling at Wells Fargo. “But the complexity of sources of data leads it to move back and forth the value chain, but there is more potential to drive growth through it. The better you engineer the data, the more efficiency it will drive; otherwise, you keep going in circles and hinder the streamline of delivery of analytics solutions.”
Termed the nerve centre of Digital Strategy by Sriram Narasimhan, Head of Data, Analytics and AI at Cognizant, data engineers’ demand can be seen through recent market statistics.
In 2020, the Dice Tech Job Report stated data engineering to be the fastest-growing job in technology with a predicted 50% year-over-year growth in the number of open positions. Global employment platform, Monster has published two recent trends and employment reports showing that ‘engineer’ was the number one job search over the month of WHAT. Additionally, engineering was among the top sectors likely to expand in 2022 by 57%.
Further, the annual salary study conducted by AIM Research in June 2021 showed that data engineers commanded a median salary greater than big-data scientists or AI engineers, indicating the growth in importance for the position. The demand for this can be seen with big tech companies like Google, IBM, Cloudera and SAS that have initiated data engineering certification programs to skill and upskill employees. However, they don’t have certifications in data science or AI engineering. “With the market for artificial intelligence and Machine Learning-powered solutions projected to grow to $1.2B by 2023, it’s important to consider business needs now and in the future. To address the new skills data engineers now need, we updated our Data Engineering on Google Cloud learning path,” Google stated.
