By Pankaj Maru.
The road to big data analytics is not easy as it might appear. It’s an exhaustive, time-consuming journey that would test the skills, understanding and knowledge of CIOs as well as the patience and perseverance of enterprises. But there is little doubt about companies across the board embracing big data solutions.
Several research reports and industry estimates point to the growth of big data analytics market globally.
According to consultant Wikibon’s projections, the sale of big data-related hardware, software and services will grow from $7.3 billion in 2011 to as much as $50.1 billion by 2017. A Forbes.com report cites IDC as predicting that the market for big data will reach $16.1 billion in 2014 — much lower than Wikibon’s projection of $28.5 but still quite significant.
Research firm Ovum’s recent study states that the banking industry will be a key driver for the big data segment. It expects the global spending on management information systems (which includes big data solutions) in the retail banking industry to reach $9.3 billion by 2018 end (up from $6.9 billion in 2013).
Quoting a recent IDC study on big data adoption covering 250 Indian enterprises, Gupta says only 15% of surveyed enterprises have adopted and deployed big data technology. “Though 15% is not a very high adoption rate, it shows that these organizations have been able to place some kind of matrix to measure ROI and gain value for business,” observes Gupta.
The study suggests that the adoption and deployment of big data by Indian enterprises is more of an experimentation at the moment.
Bracing for market opportunities
American firm Teradata, a global player in analytic data platforms, applications and related services, is highly banking on its vertical-specific approach to tap the attractive market of big data analytics.
But there are segments among traditional businesses, such as BFSI and telecom, which are embracing big data solutions.
“Today, even the traditional industries like banking and insurance are using mobile and Internet as new business channels. They are looking at big data solutions to capture the data generated through these channels and are trying to understand the business value and insights from that data,” says Vasudevan.
“To capture sensor generated data, traditional businesses need technology like big data that can help to provide insights into their business. Most of our customers are looking at the business aspect of big data analytics and not just the technology,” says Vasudevan.
In general, the awareness and interest level in big data analytics among Indian enterprises is quite encouraging. According to Santhosh D’Souza, Director – Systems Engineering, NetApp India, enterprises are demonstrating a definite interest in the application of big data analytics to their operations.
“There is widespread awareness of the insights an enterprise can obtain from all of the data residing in various applications supporting their businesses. There is also a realization that enterprises need data science skills in their IT departments to translate business questions into analytical queries which can then result in actionable insight,” says D’Souza.
The company has seen enquiries for big data solutions from several sectors, including financial services, energy, telecom, retail enterprises and even government agencies delivering citizen services.
According to D’Souza, there are two things that are making Indian enterprises turn their attention to big data. One, their application infrastructure is getting increasingly standardized, consolidated and virtualized, and they have relatively matured data warehouse implementations. And two, conventional analytics applications and data stores have been supplemented recently by in-memory computing, next generation filesystems and NoSQL databases – technologies that better facilitate processing of huge amounts of data.
Churning the existing data volumes into more meaningful information for business purposes is where the real power of big of data analytics comes into play. “In our view, big data is about putting intelligence on top of traditional data,” says Vasudevan.
Big data bottlenecks
Compared to global markets, India is still an emerging market and even the local enterprises are not at par with their global counterparts in terms of maturity levels when it comes to big data analytics adoption.
Analysts believe that cost and shortage of skilled tech professionals in India are among the factors slowing down the pace of big data analytics adoption. Besides, most suppliers and vendors are finding it hard to convince enterprises to invest in big data technology because of lack of easily quantifiable RoI. The majority of companies in India are following the ‘wait and watch’ policy on the big data analytics arena.
“There’s a lack of clarity over big data analytics. Enterprises actually want to understand the value that big data can create in a measurable way,” points out Gupta of IDC India.
Going by industry definitions, organizations with around 100 TB data or more and having an average data growth rate of 60% per year, are considered large enough to venture into big data.
Today, businesses are witnessing exponential data growth and big data analytics is invariably becoming inevitable for them. Many companies are generating sensor-based data as well, which needs technology that can help provide cross-functional value across departments.
That’s where big data scientists and business analysts can play a significant role in order to derive business insights from volumes of organizational data. Cross-functionality of data, though highly beneficial to organizations, is quite a challenge, points out Gupta.
The question of talent
Talent has a huge role to play in driving big data analytics to mainstream across verticals. Ideally, for big data analytics, the talent needs to have a mix of technology skills, domain knowledge and work experience. However, with the short supply of talent in the market, the solution to the problem lies in the creating industry partnerships and collaborations.
“Though there’s a short supply of skilled professions in the area of big data analytics, an ecosystem of resources can be developed by providing trainings and knowledge sessions through industry partnerships. It will take 3 to 5 years to reach a maturity level where industry can start to take full benefits of big data analytics,” says Gupta.
While the IT industry is looking for tech professionals in the roles of data scientists and business analysts, Vasudevan, who himself has worked as data scientist in past at the same company, says that organizations and enterprises actually need to find staff internally, which has strong fundamentals around data and business domains. “Staff can be trained with the help of CIOs and strategic business units within organization,” suggests Vasudevan.
Despite issues around cost and talent shortage, industry analysts expect a matured big data analytics ecosystem over the next 3 to 5 years time. Besides telecom, IT and ITeS, verticals such as financial services sector including banks, insurance firms, manufacturing and retail are seeing some traction in India. The big data related revenue moved up to $55.7 million in CY2013 from $ 40.7 million in CY2012, according to Gupta of IDC India.
Wide opportunities
Big data analytics brings wide opportunities for technology suppliers and service providers. However, it’s the enterprises and their CIOs that are actually going to drive those business opportunities for these technology vendors and suppliers; and would boost big data analytics to some maturity levels in the coming years.
But for CIOs, taking the big data analytics path requires putting more effort — right from planning and designing, resource allocation and technology utilization to investments and a reasonable time frame to reach a desired set of goals. However, this doesn’t mean that enterprises and their CIOs are shying away from investing and taking advantage big data analytics offers to businesses. But the decision to adopt big data analytics is highly dependent on the nature of business or industry in which it operates.
So for instance, companies operating in domains like banks, financial services, insurance, telecommunication, retail and others are more likely to invest and benefit from big data technology.
Tesco, the U.K. based retail giant with 6,600 stores globally is among the top enterprises that deals with enormous size of data. Beside data generated from more 80,000 tills (cash registers/PoS) spread across all stores located in 12 countries, the retailer also has offers online and mobile channels to customers, which too generates data of significant scale.
And that is where Tesco started to encounter big data in the classical definition of four vs. many decades ago. To address the exponentially growing data, the world’s third largest retailer with a $115 billion revenue, came up with Tesco Hindustan Service Center (HSC) at Bangalore in 2004. This key center provides all the operational and technology support to Tesco’s retail business in different countries.
Given such a huge customer base and burgeoning number of transactions across channels, the rate at which data is generated at Tesco is massive. However, it is the Bangalore center with over 6,500 staff that manages all the customer data, as well as provides key insights and foresight to Tesco, both on business and customer behavior using a mix of customized big data analytics solutions and platforms like Hadoop, ETL (extract, transform, load) and others, according to Bidarkoppa.
At Tesco, the applicability of big data analytics is largely in three key areas — operational efficiency, customer services and personalization. “All the three areas are equally important to us, and data generated in those three areas is crunched and analyzed with a mix of technology, domain knowledge and right resources with the right skills, to create meaningful information. That’s what makes Tesco a leading retailer in the world,” says Bidarkoppa.
For example, Tesco uses video analytics to understand customer behavior but at the same, uses video feeds to monitor supply and inventories at shelves in its stores. Importantly, Bidarkoppa points that customization of big data analytics solutions is necessary as per the industry and nature of business.
He is of the view that presently, many organizations are focusing more on unstructured data coming from internet, social media and mobile channels but are not taking a holistic approach, to combine their existing structured data with the unstructured. Overall, big data is a journey and enterprises need to have a congregation of domain knowledge and technical skills, says Bidarkoppa.
Like Tesco, the Chennai based Apollo Hospitals Enterprise Limited also has been using big data analytics solutions not only for clinical research but also for clinical systems.
In Sivaramakrishnan’s view, dealing with big data analytics remains a tough challenge for any CIO. “One of the biggest challenge with big data for CIOs is actually defining the outcome of big data. And in the process of defining the big data outcome, the CIO is required to have a systematic designing of the existing organizational data and try to understand it more from business perspective,” he says.
“Working on big data analytics has two main parts — technology and business. So while planning the big data outcome, it should be based on existing volumes of data and the CIO has to have a technology design aligned with the business domain and its requirements,” Sivaramakrishnan adds further.
He says that the designing of big data needs to be based on the nature of business and not looked at just from a technology point of view.
With the focus on business insights, Uflex Limited, a Noida based flexible packing company has placed a multi-layered big data strategy to drive new business initiatives, where it uses its big data architecture to get business insights, and then shares it with various business vertical heads to support the decision making processes.
“The activity is an ongoing process wherein the technical and financial analysts work in a group to build insights for functional level leaders that help to meet the company’s big data analytics objectives,” says Adurti.
It’s more of the large enterprise and organizations that have deployed big data analytics to an extent, however, striking a right chord between technology and talent is equally challenging for CIOs in India like anywhere else.
“Definitely, the skills and talent aspect is challenging for CIOs. It’s a combination of business and technology, where we need to find IT or technology people with capabilities and skills to understand and churn volumes of big data into more meaningful information,” says Sivaramakrishnan of Apollo Hospitals.
Moreover, one of the key aspects about talent in the big data analytics space in any organization, is that the IT team needs to align and augment with the business team, as it can help in the process of designing the big data solution, selecting the right technology and driving the desired outcomes.
Tackling the talent shortage
In view of Adurti, organizations need to work strategically, create new positions with required roles and responsibilities to drive this key business initiative and mitigate the manpower shortage. “Creating C-level positions for big data analytics helps add sustained focus and prepares the organization for achieving big data related strategic objectives,” says Adurti.
Organizations need to identify staff with techno-functional and analytical skills, which can work with the chief business analytical officer to drive the big data strategy, for result oriented decisions, suggests Adurti.
However, Adurti points that the shortage is mainly because of non-availability of professional
business analysts and strategic business interpreters. He adds, “The same can be managed by developing the in-house functional people with the knowledge of technology for big data analytical requirements.”
While these suggestions and ideas looks perfect, not all enterprises and their CIOs would be in a position to leverage their internal staff. Large size organizations and companies having specific business requisites, backed by strong finance, can actually leverage in-house staff through strategic investments in skill and knowledge enhancement initiatives.
Tesco HSC for instance, has instituted a comprehensive two dimensional structured training initiative for all staff, that includes technical trainings and domain knowledge. The technical trainings are combined trainings through in-house practitioners and online trainings in areas of technologies that are inline with the retail industry.
Secondly, company invites industry experts from Gartner and others agencies to be part of conferences where the staff can interact and get learning opportunities with the experts.
“Basically, our in-house practitioners are people having long working experience at Tesco coupled with sound knowledge of retail industry and business processes. Training is very essential because most freshers come with technical skills and knowledge, but lack domain understanding and work experience,” says Bidarkoppa.
Few years ago, besides the technical trainings, the Tesco management also decided to set up a Retail Certifications program based on domain curriculum specifically designed as per the company’s business processes and needs, as it operates in a multi-country and multi-channel business environment.
“We have three certifications – Brown, Silver and Gold. Every one joining Tesco by default needs to have the Brown certification in one year’s time. It provides in-depth understanding of retail industry. The Silver certification is a more detailed course with 30-odd modules over six months time, and is mandatory for staff at certain levels,” explains Bidarkoppa.
“The Gold certification is more like a mini-master degree course focused on staff with Brown and Silver certifications. Here, the certified staff are exposed to pure technologies like Hadoop cluster, ETL, data warehousing, micro strategies and others, which enables the understanding of big data analytics to drive the business of organization,” adds Bidarkoppa. He summarizes that data analytics is all about marrying technology, business processes with data and coming out with meaningful information.
Certainly, Tesco’s approach towards training staff in the area of big data analytics is quite illustrative in nature for other companies to follow, but requires good efforts and investments in developing such courses and trainings, in-house.
Serving the analytics demand
Following Tesco’s style of training would be more suitable for large or very large size organizations. But mid or small size companies with tight budgets and limited resources, need to look at other options like outsourcing data analytics services or consulting experts from the industry.
However, some industry experts differ on the whole idea of outsourcing analytics, as data is considered quite critical and they reckon that in-house staff would be better to handle that data rather then external resources.
And today, there are several data analytics service providers available in the market, that cater to local and foreign organizations. Among them, is a U.S based data analytics consultant and data management services firm LatentView founded in 2006. The company has set up a global delivery center in Chennai for its clients.
“At LatentView, we have been seeing significant traction in the market and we have been doubling our scale, year on year. The primary verticals that drive the demand for data analytics are those with a strong consumer focus-consumer packaged goods, retail, technology, etc. Also, historically we have done a significant amount of work in the BFSI sector,” says Venkat Viswanathan, Founder and Chairman, LatentView.
On the data analytics services demand, Viswananthan points that the demand has definitely grown tremendously in the last five years as companies have realized the value of data they are sitting on.
“Data is seen as an asset that needs to be utilized. Traditionally, organizations have looked at their owned internal data when running any analysis — let’s say from their ERP systems. Nowadays, companies have realized that this is no longer adequate, and they need to map external data sources onto their internal ones,” says Viswananthan.
He adds examples of external sources of data could be social media, location based data etc. This realization, coupled with the plummeting cost of data storage, has led to numerous technology innovations in the big data space.
“Banking, telecom and insurance are three verticals that have been engaged in identifying and understanding the application of data analytics initiatives within their businesses,” says Jayaraman. He points out that internet and mobile device-based retail transactions generate high volumes of data, and ask for a well-managed data service that can reduce turnaround times and enhance business efficiency.
Buoyed by the market demands, Jayaraman strongly feels that clients are highly enthusiastic and are looking to use data analytics at every stage of customer interaction. “Having said that, in the banking industry, risk remains the stronghold of data analytics, though marketing initiatives are increasingly looking to use quantitative techniques as well,” points out Jayaraman.
Big data analytics is gaining ground in the enterprise world and the industry is seeing a shift from hype to reality. Encapsulating the overall industry mood around big data analytics, Jayaraman says, “The room for growth is tremendous and the scope for analytics has never been more pronounced.”