The Big World of Analytics

Mining complex user data real-time can help companies capitalize on hitherto-untapped business opportunities

By KTP Radhika

It is no longer a hypothesis. The exponential growth in digital information has brought forth new verticals and businesses—and analytics is a key segment that sprung up in the boom. Research firm IDC projects that the digital universe will reach 40 zettabytes (ZB) by 2020. According to analysts, data volume has grown over 900% in the past five years and is expected to grow at 40% year-on-year till 2020. Today, businesses are capturing trillions of bytes of data about their customers, suppliers and operations. Data is entering the system at a rate that follows Moore’s Law, doubling every two years.

Before the concept of big data entered mainstream, companies were using simple methods to study in-house data. In the structured data space, data mining technologies existed. “Traditionally, enterprise data used to come from systems like enterprise resource planning (ERP), human resource (HR) systems, supply chain systems and customer relationship management,” says Shailendra Ravi, Senior Director, Emerging Markets, EMC India. “Traditional analytics used to focus on post-factor questions like what, when, who, etc. basically for reports about key performance indicators (KPI), metrics and transactions within an organization either by automated monitoring, alerts or dashboards.”

However, as data started flooding from a variety of sources such as customer loyalty programs, remote sensors, call centers, ATMs, social media and machine-to-machine communications, data analytical technologies also started changing. Enterprises realized that they should have the technology to decode the growing enterprise data in order to maintain a competitive edge.
“Today, both structured and unstructured data grow faster than ever and nearly 85% of the data falls in unstructured category. So, advanced BI (business intelligence) and analytics tools are essential for faster and better decision-making capabilities and for driving new business growth,” says Arindam Dutta, Head – Global Business Analytics & Asia Business Services, HP.

According to a report by IT industry body Nasscom and research firm Crisil, the big data and analytics market in India is estimated to grow to about $1 billion by 2015. The rising demand for data analysis and intelligence across industries has prompted companies to embrace BI and analytics increasingly. According to Abhay Koranne, Associate Vice President – DW BI Practice, Collabera, awareness about BI and analytics is fast catching up in India.

Many executives see analytics as a fundamental capability and resource. Companies have started looking at wider datasets. Daniel Ziv, Vice President, Voice of the Customer Analytics, Verint Systems, says, “While in the past, BI and analytics may have been viewed as nice-to-have options, they are now becoming staple technologies for many organizations.” Today, the increase in awareness for BI is driving its adoption in the country. The actionable information hunger is widespread across industry verticals. With big data in the picture, BI’s definition now has widened from just intelligence, to encompass analytics, predictive intelligence and most importantly, intelligence in real-time.

As the market is expanding, a lot of technological advancements are happening. Businesses that earlier used analytics to examine past problems are today leveraging analytics to spot trends and predict future outcomes. The same is the case with BI: it is no longer a mere reporting tool, but a strategic enabler that empowers smarter and faster decisions. All thanks to cutting-edge, user-friendly and efficient solutions.

Sudipta K Sen, Regional Director – South East Asia, and CEO & MD – SAS Institute India, points out, “Today, slicing and dicing a billion records of data is only a matter of seconds or minutes instead of hours or days. With the growing volume, velocity and variety of big data, analytics solutions are now expected to massively ramp up in terms of performance, speed and insight-deriving capabilities.”

On top of Hadoop
The technological landscape has changed with innovation in both managing and storing large data. As organizations move away from the traditional data storage systems such as file systems and databases to newer technologies such as cloud-based storage, open source software are also getting popular. To tap this market, most vendors are coming up with products to offer big data analytics and are integrating their product suite with Hadoop architecture. Explains Vikash Mehrotra, Divisional Lead, EPM BI, Oracle India. “Today, there are over 120 open source key-value databases for acquiring and storing big data, with Hadoop emerging as the primary system for organizing big data, and relational databases are expanding their reach into less structured data sets.”

According to technology research firm Gartner, Hadoop will be among the most advanced analytics products by 2015 and 65% of packaged analytic applications with advanced analytics will have Hadoop embedded in it. As technology advances, vendors are partnering with major Hadoop platforms such as Cloudera, Hortonworks, MapR, etc. and this is helping to achieve better standardization in this space. There’s more. Visualization tools are also gaining importance in this space. Plenty of new tools came up in the recent past catering to the visualization aspects of data. Notes Pankaj Rai, Director, Dell Global Analytics: “Data visualization has now started getting more importance after senior management started taking data-driven decisions. This has led to a lot of innovation in this space. Visualization tools such as Splunk are getting huge popularity.”

Says Umang Bedi, Managing Director, South Asia, Adobe Systems, “Powerful visualization allows marketers to immediately infer meaning so they can make quick business decisions that improve overall business performance.” Further, he says, with advanced analytics tools, businesses now can increase content relevance and continually refine online content to make it more relevant to customers and improve conversion rates. Adobe offers Digital Analytics and Adobe Insight in the visualization area.

Predictive, mobile, cloud-driven
Predictive analytics is another trend catching up in the big data analytics space. Predictive analytics techniques help enterprises anticipate customer needs better, understand trends and respond to them in real time and have better collaboration. Maneesh Sharma, Head – Database and Technology, SAP India, feels “BI’s definition has widened from just intelligence to encompass analytics, predictive intelligence and, most importantly, intelligence in real time. Real-time and predictive analysis has become one of the largest trends in the data analytics field.” Mathematical rigor and predictive modeling are being applied to help businesses run leaner operations, produce more with the same resources, and tailor their products and services to consumer needs and preferences.

Offering a similar view is Venkat Iyer, Director – BIM India Lead, Capgemini. “Increasing business needs are driving BI requirements to predictive analytics. It helps organizations provide a view of the lowest granular level, allowing them to see and analyze the minute details of a business, and to peep into the future.” Commercial solutions like Oracle data mining, SAS enterprise miner, IBM SPSS, SAP BusinessObjects, and open source tools like R, Rapid miner, Python, etc., are popular in this space.

As the world goes mobile and executives prefer to access insights on-the-go and take real-time decisions, mobile analytics solutions are gaining traction in the BI space. The phenomenal growth witnessed in adoption of mobile devices, such as smartphones and tablets at workplace, is driving the momentum for mobile BI. “With the concept of work from anywhere catching up, we see increasingly mobile and remote workforces. This is driving a lot of vendors to focus on delivering BI for mobile devices,” feels Arindam Mukherjee, Vice President – Sales (BFSI), Cisco Systems. Mobility is also leading organizations to provide consolidated analytics shared on a common platform, thus making BI more pervasive.

Another major trend is the cloud. According to Yogesh Sawant, Director of Partner Sales and Field Alliance Organization, India, Hitachi Data Systems, ”Cloud technology is driving the adoption of analytical tools on the cloud as it offers cost benefits and speed of deployment.” By deploying BI solutions on a cloud-based computing architecture, organizations can analyze their data and take informed decisions in a timely and cost-effective manner.

Vishnu Bhat, Vice President and Head – Cloud Services, Infosys, explains: “Leveraging constructs of extreme and big data was expensive and time-consuming earlier. But cloud technologies and distributed processing have helped address the issues of cost and time concerns. Complex data analysis can be done today at a fraction of the cost and at much higher speeds.” Even Indian SMBs have adapted to using cloud-based BI models to stay ahead of the competition.

“Cloud-based big data deployments will see a surge this year primarily driven by cheaper implementations, faster delivery of new features, and the ability to support a dynamic and mobile workforce with browser-based access from anywhere, anytime,” opines Ashok Saxena, Head, Engineering Centre, Kronos India.

 Adoption catching up
Adoption of big data analytics amongst Indian business is still in the elementary stage with only a handful of companies investing in collecting and storing enough data for decision-making. According to Rai of Dell, Indian firms are still in the standard, ad-hoc report analysis phase. “However, as the latest computing technologies such as the cloud are maturing in India, the adoption of big data analytics is also gaining momentum,” he says.

The early adopters of analytics in India were from the finance sector. Trivikram Kamath, Head – Technology, Operations & Finance, Kotak Securities, states, “BI has been used in BFSI in some form over a period of time. However, in the recent past, it is possible to analyze data easily with modern analytics tools. It no longer makes sense to keep on adding customers but to see how you can derive and extract additional value from existing customers.”

Following BFSI, the retail sector has also started using big data analytics to analyze social media and match this data against customer lists, transactions, and loyalty club memberships. Now, there is a growing demand for analytics tools among the telecom sector as well.
As more multinationals set up big bases in India, there is an increased focus on the usage of big data analytics in Indian markets. Also, as the awareness about analytics increases, most other verticals, too, will start using BI tools in a big way.

Satish Gaonkar, Vice President – Consulting Services, Blue Star Infotech, feels that at one point of time, BFSI was the largest adopter of BI and analytics. But today, in sectors like manufacturing, healthcare and logistics, analytics is becoming more popular. “Traditionally, people were focusing more on reporting tools. However, performance analytics, purchase analytics, sales analytics, etc., are now used to lower the total cost of business,” he says.
According to Arun Gupta, CIO, Cipla, big data and better analytics with almost unlimited computing through the cloud is offering quicker turnaround to complex calculations required by the pharmaceutical industry.

Commenting on the benefits of analytics in the travel industry, Bhaskar Dey, Shop Front Manager, Travelguru.com, says, “Analytics solutions are being used frequently across the travel industry today. Most industry players are using analytics to drive higher conversion, higher retention and better site optimization.”

As technology matures and the cost of deployment comes down, say industry experts, there will be an increased adoption of BI and analytics tools. “Quite a few Indian organizations have in place data generation, process management and MIS frameworks. The current wave is to derive competitive edge by turning this insight into action,” says Oracle’s Mehrotra. Government programs such as Unique Identification Authority of India (UIDAI)’s Aadhaar are providing an opportunity to showcase big data services in real-life applications.

Vendor opportunities
As the application landscape widens, leading analytics vendors are formulation strategies and developing specific solutions to address the growing opportunities in big data. Explaining Cisco’s measures, Mukherjee says, “We have joined hands with SAP to create a powerful, self-service BI appliance designed to help users locate, evaluate, and use the crucial business information while minimizing the operational cost.” Companies are also devising compelling mergers and acquisition strategies to acquire analytics firms to bolster their portfolio.
Given the huge potential, vendors have started developing complete solutions to address all the analytics needs of a customer. Says Mehrotra, “Oracle is offering complete and integrated solutions to address the full spectrum of enterprise big data requirements. Our big data strategy is that customers can evolve their current enterprise data architecture to incorporate big data and deliver business value.”

Stressing the need for integrated, real-time solutions, Sharma of SAP says, “Integrated solutions will reduce total cost of ownership, and provide a viable solution for the customer.” This is where SAP’s HANA real-time data platform comes in. “It can acquire, store, process and report big data with varying volume, variety and velocity and provide real-rime value,” says Sharma.

The increasing focus on big data is also driving a lot of acquisitions. “For instance, Oracle acquired Endeca, Terradata bought Aster Data, IBM acquired Varicent, EMC acquired a company called Pivotal Labs, and HP acquired Vertica. The number of big data acquisitions that happened in the first half of 2012 tripled compared to the same period in 2011,” states EMC’s Ravi.

IDC predicts that global enterprises will invest Rs12,000 crore by 2015 (across hardware, software and services) to capture the business impact of analytics. “This certainly presents an excellent growth opportunity for the entire vendor community,” feels Sen of SAS. With areas such as learning analytics, social analytics, social network analytics and web analytics gaining importance, many vendors have started offering tools in these domains.

Bottlenecks
While organizations are keen to have BI and analytics in place, they seldom have a well-thought-out strategy, without which their readiness and execution get impacted. With varied and voluminous data coming in, every organization now wants to go the big data way. But without the right infrastructure in place and the requisite mechanism to capture data, and in the absence of a strategy to find and use “dark” data, most organizations come up with a half-baked plan for BI and analytics. Bhavesh Gupta, Head – Business Transformation and Co-Founder, Emkor Services, opines, “Due to relative newness of the technology, successful BI implementation still remains an oxymoron. It is imperative that managers think beyond Excel if they have BI on their mind.”

The fear of not being able to achieve proper return on investment (RoI) is another dampener for many firms. “Though many CIOs plan to invest in BI going forward, most of them fear BI failure. Hence true RoI is a problem as far as BI projects are concerned. Companies make large investments during the initial stages of the project. However, the back-office analysts do not have the capabilities to use those investments. Hence, it is better to have a step-by-step approach with the right analyst, appropriate talent and suitable tools to avoid BI project failures,” explains Sanchit Vir Gogia, Chief Analyst, Founder & CEO,  Greyhound Research

Lack of expertise on the subject is another issue. A shortage of people with the skills necessary to take advantage of the insights that large datasets generate is one of the most important constraints in this field. Leading companies are already reporting challenges in hiring this type of talent.

Inferior data quality also hinders the analytical process. “To deliver accurate data analytics, organizations must first ensure that they’re acquiring the right data, since even the most painstaking analysis will be less than useful if the wrong raw data is used,” warns HP’s Dutta. Storage of big data is also becoming a problem for enterprises.

Owing to the high infrastructure costs, there is a challenge in storing large volumes of data that are being generated every day. “This problem has been rectified to an extent with technologies such as cloud, which gives unlimited compute and storage,” says Bhat of Infosys.
The ‘big data era’ has only begun. With data exploding at tremendous rate, BI and analytics will see a promising growth curve in the coming years. In the future, analytics will be the need of the hour and its significance will continue to grow as enterprises across industries start realizing the importance of leveraging big data to create big business value. This will be driven by factors such as the need for better and faster access to company data; better insights into the consumers’ buying habits and patterns; and even quicker product launches. Experts feel that data mining will grow to encompass analysis of data from different departments, including employees, finance, administration, etc., to provide a holistic picture of the quality of services and the efficiency of business.

As smart devices encourages mobile BI, experts say data analytics will become the life blood of every business. Tools that deliver power to the end users will gain immense popularity. As technology matures and adoption increases, analytics will become more integrated and real-time.