Data engineering combines elements of software engineering and data science and is one of the fastest-growing roles in IT. According to Indeed.com, data engineers develop and maintain the architecture used in data science projects. They are responsible for ensuring that data flows between servers and applications uninterrupted.
Data engineers must be familiar with a range of operating systems and databases and able to write and program software. They are experienced with data warehousing and data analysis and must possess excellent critical thinking and communication skills. Data engineers may learn their skills through a combination of education, on-the-job training, and ongoing certificates. Indeed notes that acquiring a certification is an excellent way to showcase abilities and move ahead in the field.
To find out what’s involved in becoming a data engineer, we spoke with Lance Miles, a data engineer at unitQ.
Miles earned a Bachelor of Science degree in neuroscience from the University of California, Santa Cruz, in 2013; a certification in data science from the University of Washington in 2017; and a Master of Information and Data Science degree from the University of California, Berkeley, in 2020.
“When I reflect on my career and the steps I have taken, one particular experience had a great deal of influence on me,” Miles says. “In the final quarter of college, a Python course, Programming for Biologists, set the groundwork for a new passion."
Miles spent every day writing code to extract information from massive sequencing data sets, developing methods to calculate the physical-chemical properties of protein sequences, identify the length and location of genes, and characterize viral DNA.
“The ability to distill unwieldy data sets into concise results highlighted the power of pairing programming with biology,” Miles says. “This course challenged me in new ways, and I found myself completely hooked. At a personal level, the simple act of coding brought me happiness and gratification.”
Although Miles had always been interested in technology, he started his career in the healthcare sector at Gilead Sciences, a pharmaceuticals company.
“My journey to becoming a data engineer has been far from straightforward, but what has connected it all together has no doubt been my interest in utilizing data to change how teams and companies look at their work and the impact it has,” Miles says.
At Gilead Sciences, Miles worked as a senior research associate in in-vitro biology, identifying clinically translatable biomarkers indicative of cardiovascular health. Each experiment he worked on yielded thousands of data points, but the data analysis was time consuming.
“I saw an opportunity to streamline the analysis, creating Excel macros that efficiently parsed the data and extracted vital information,” Miles says. “This allowed the team to focus on digesting the results and deciding on the subsequent experiments. Seeing the impact of my work in identifying effective biomarkers, I sought to focus on projects where the impact on patients was clear and immediate.”
Upon completing hist pre-clinical projects, Miles transferred to the clinical pharmacology group as a bioanalytical operations lead for antiviral clinical studies. “Tapping into my data analytics roots, I worked with clinical enrollment information to forecast when we would have pharmacokinetic data available for drug submissions,” he says. “In addition, I had opportunities to work with clinical data, where I compiled, cleaned, and analyzed patient data across multiple clinical studies to assess data quality.”
With the encouragement and help of senior leadership, Miles enrolled in a data science certificate program through the University of Washington. “This was my first exposure to machine learning and what ultimately cemented my desire to switch careers,” he says.
