5 tactics to beat big data hiring challenges
Hiring for specialist roles in emerging areas of technology can be tough – and expensive. But you don’t need to over-pay when building a new team if you follow the right strategy.
When big data started heating up, I faced the challenge of building a group of qualified engineers in New York City when most of the available talent was located on the West Coast. We could have staffed up by paying more, but that isn’t necessarily the best use of company funds. Instead, we did the following:
These tactics can be modified to help grow any specialist technical team. Consider these five strategies to address your hiring challenges.
In 2008, when we first saw the need to develop a big data team oriented around the Hadoop ecosystem, there were very few big data engineers in NYC. That essentially left two choices: Recruit and relocate from the West Coast, or mentor and train our own team.
Given the limited availability and high costs of finding, hiring, and relocating engineers, we chose to build and train our own team. We established a hiring and training process oriented around an engineer’s intelligence, ability to learn, and interest in working on big data engineering challenges. This approach was, and continues to be, wildly successful.
The engineers we hired were ecstatic to be on the forefront of learning and establishing a new engineering discipline. Another plus: While many established West Coast big data engineers would likely be inclined to leave for a higher salary, the engineers we hired thrived in learning and overcoming challenges, and exhibited outstanding loyalty to their technology leaders and the company. In short, they appreciated the opportunity.
Big data community participation worked well for us. We founded the NY Hadoop User Group in 2008 to share knowledge and grow the big data community within the metropolitan area, and the group has substantially improved since then. We also sent key members of our team to Hadoop World and other conferences, where they rubbed elbows with leaders in the big data world. Some early influencers ended up writing seminal books around Hadoop, and some of our team members served as book reviewers. Getting your team members and company name out into the community can also be enormously helpful in recruiting.
While we didn’t find any experienced big data engineers offshore, Eastern Europe was home to many smart, experienced engineers with analytical math backgrounds who were excited to learn about big data technologies. You can either build your own remote offshore team or partner with a consulting firm to help you build one.
