One key question: Should you be hiring from the small pool of outside experts, or training up existing staff from the inside? Also, how can you break into big data from another IT path?
Many IT pros say this is just another example of a supposed IT talent shortage. While there's little dispute that PhD-level data science experts are a small pool, that isn't the only hiring need. Are companies unwilling to invest in training workers to acquire new skills such as Hadoop? Here are some interesting perspectives from David Menninger, head of business development and strategy for Pivotal, a subsidiary of EMC.
Part of his job now is sending in data science teams to do early work with companies that are simply trying to understand the value of data science projects. They might have data analysis experts but not data science or big data experts and they are just starting down the big data path. In other words, these EMC teams do early engagements with companies that EMC and Pivotal will pitch for future data analysis business.
Our conversation pointed out some interesting facts for those of you trying to break into big data as a career. For starters, how does EMC staff this effort when data science experts are in such demand?
1. Data Science is a Team Sport.
EMC has recruited experts from the outside and trained up internal resources to become this big data team's members. He estimates 20% of the team was hired from outside and 80% were trained from the inside. EMC did it using bootcamps in several locations across the country. The data science experts hired from the outside created the curriculum that was later taught at the bootcamps. EMC rolled out the bootcamp program in the second half of 2013.
Menninger's advice echoes what we heard recently from Dr. Michael Wu, chief scientist of Lithium Technologies: Hiring managers should stop trying to find mythical people who have three important kinds of big data skills and focus on building a team.
2. Data Science/Hadoop Skills.
Hiring for the data science skills is harder than hiring for the Hadoop skills right now, Menninger says. Hadoop skills are just table stakes if you want in to a big data career path, he says. If you haven't got them, you're going to hit a wall trying to get into the EMC program. This is one example where an online course or evening program isn't just a nice-to-have but a necessary first step.
3. Presentation skills matter.
This is not a back-room art, for EMC or its customers. "They [big data team members] have to be able to communicate well," Menninger says.
Big data efforts bring together people from parts of the business who might not have worked together before. They involve political struggles between the CIO and the CMO, for control over the power that will accompany running data analysis efforts moving forward. Are you the type of person that can build consensus amid discord? Demonstrate that.
4. Diverse teams matter.
Some experts advocate hiring musicians and biologists for big data teams -- people trained outside IT and skilled at identifying and dissecting patterns in varying disciplines.
Pivotal's Menninger agrees. "We don't have musicians but we do have some biologists," he says. "We believe in interdisciplinary skills." In other words, if you staff your team only with people who came up through IT or traditional analytics educations, you're missing out on important skills and perspectives.
5. Marketing and Hadoop Skills.
If you have marketing-side experience and acquire Hadoop skills, you just might be a star player. An important question for IT organizations is unanswered still: Who does the data science team report up to: the CIO or CMO?
Jeremy Burton, executive VP, product operations and marketing at EMC, says he thinks it has to be the CMO. "It's a partnership with IT though," he adds. IT will bring the experience in areas like moving data, and marketing will bring the understanding of the data sets themselves and the closeness to the customer, he says.
If you as a job applicant could walk in with both experience sets of Hadoop and marketing, you have an interesting story to tell right now. Even if you have a marketing project story to tell, emphasize it.