The world is finally acknowledging something that my fellow data scientists and I have cheerfully staked our futures on.
Gaurav Vohra
The world is finally acknowledging something that my fellow data scientists and I have cheerfully staked our futures on. As the founder of an institute that trains data analysts, I am overjoyed to see governments and businesses coming to the same conclusion that my fellow data-geeks and I did years ago—that Big Data is a big deal. Interestingly, although the demand for data professionals has grown, there has not been a correspondingly meteoric surge in supply. This widening gap between demand and supply can be attributed to specific drivers.
So, what is Big Data? It refers to large swathes of semi-structured or unstructured information that is so heterogeneous and voluminous—and generated at such a brisk pace—that it exceeds the capacities of traditional data management systems. Big Data’s defining characteristics are best summed up by four ‘Vs’—variety, volume, value and velocity. Mobile phones, social networking platforms like Facebook and Twitter, and search engines like Google and Bing (to name a few) are popular sources of Big Data. Just to give you a sense of the quantum of data that these sites and their users generate—10 million photos are uploaded on Facebook every hour, and 400 million tweets are tweeted every day.
Why Big Data is big business
As more and more people shop, read and communicate online, the more they reveal about their likes and dislikes. Companies with access to this knowledge can personalise their marketing and product development strategies. They become so intuitive, they are able to anticipate—and capitalise on—their customers’ needs and desires.
The pivotal stage of the data extraction process is the analysis phase. Big Data only means big bucks for institutions that have the expertise to cull out insights from the information they generate or collect. This is the domain of Big Data analytics, which is the process of parsing, organising and interpreting Big Data for actionable information. From start-ups to more established concerns, Big Data analytics has become a core component of companies’ business development strategies. As one financial journal put it, “The market potential … is huge for vendors of Big Data analytics services.” So potent is this potential that Pascal Clement, head of a leading flight bookings management service, sees data analysts as the ‘new superheroes’ of the information age.
In inverse proportion to growing demand is a stark shortage of data analysis services. In the US, the situation is so dire that, by 2018, experts expect demand to outstrip supply by 50-60%. The same trend seems to be playing out in India too. Aditya Narayan Mishra, president of the Indian arm of global recruiting firm Randstad, says that though “the demand for talent has been going up steadily … nothing significant has been happening on the supply side.” Experts estimate a shortfall of approximately 2 lakh data analysts in India by 2018. The extremely competitive pay scales reflect this discrepancy—in 2005, entry-level salaries were around R2-4 lakh per year; today, they fall into the R6-9 lakh range.
Drivers behind the demand for Big Data experts
Demand is climbing because both domestic and global companies have converged on India in search of talent.
Niche analysts: Niche analytics services are the primary drivers of growth in this field. With increased funding and growth, they are on the constant lookout for talented analysts.
Multinationals: GE Capital, Bank of America, JP Morgan, Amazon, eBay and Target are among a slew of multinational companies that are capitalising on India’s data science pool.
Consultancies: Deloitte, AC Nielsen, KPMG and McKinsey now place a big emphasis on conducting data-based analyses, and providing clients data-backed recommendations.
Indian IT companies: India’s IT services market is saturated. IT companies realise that, to maintain a healthy growth trajectory, they will have to build their employees’ data analysis capacities. Wipro, for example, is now pursuing and landing contracts worth over a billion dollars in analytics.
Retailers: B2C Indian companies are aware that to compete with global giants like Amazon and eBay, they need to strengthen their analytics game.
Sexiest job of 21st century
Big Data analysts are ‘data whisperers’, with the skills to talk—and listen—to data. As the venerable Harvard Business Review noted, this makes Big Data analysis the “sexiest job of the 21st century.” With data being generated at such a furious pace, I don’t see the demand for big data analysis—or analysts—slowing down at any time soon.
The author is CEO, Jigsaw Academy, an institute that trains data analysts