Overcoming hybrid and multi-cloud challenges in new era

By Vinod Ganesan, Regional Vice President India, Cloudera

Pandemic-induced disruptions, evolving consumer behavior, and extreme competition have made hybrid and multi-cloud environments the new normal of computing. 80% of organizations across the world have already adopted a hybrid cloud strategy, and 89% a multi-cloud strategy, according to Flexera’s 2022 State of the Cloud Report. IBM’s Transformation Index: State of the Cloud found a strong correlation between hybrid cloud adoption and progress in digital transformation, including that combining hybrid cloud with other levers of business transformation can generate up to 13 times greater benefits than cloud alone. The research also uncovered a trend of organizations moving workloads from public clouds back to some form of private infrastructure to improve performance and latency, security, and compliance.

Highly controllable, easily scalable, and cost-effective, the hybrid and multi-cloud allows organizations to customize their cloud architectures to suit evolving business requirements. The ease at which employees can share and access data with the cloud encourages a data-driven culture and helps enterprises accelerate time to insight for informed decision-making.

As big data has evolved to cloud data and now to hybrid data, so do the complexities that businesses face in accessing, using, and creating value from their data. In the Indian market where organizations are facing increasing digital regulatory compliance amidst an acute tech talent crunch, the failure to implement a holistic hybrid cloud strategy can create gaps in the organizations’ security and compliance, and cause risk across cloud environments.

Recent research from IBM revealed that while a large majority of Indian business leaders (85%) have adopted a hybrid cloud approach to help drive digital transformation, only 30% are able to manage their hybrid data cloud environments holistically.

Many businesses new to the multi and hybrid cloud often rush through setting up environments without a proper enterprise data management strategy that is organization-wide, integrated, governed, and secure, seeking to quickly enjoy efficiency gains or seize business opportunities before they disappear.

To effectively harness the full potential of the hybrid and multi-cloud requires that organizations leverage modern data architectures like data lakehouse, data fabric and data mesh that are designed to automatically handle these complexities, so IT teams do not have to. A unified data platform with portable, interoperable data analytics for the full data lifecycle and distributed data management management running on public clouds, on premises and at the edge is fundamental to making the hybrid data strategy a reality. Having secure, well-governed and accessible data on demand will fuel business growth.

Ready or not, the rise of hybrid and multi-cloud is here, bringing along numerous business opportunities and challenges that organizations must rise up to meet or risk derailing their company’s digital transformation efforts.

Challenge #1: Silos and security issues
As enterprises rush to the cloud, those uninformed of data integration challenges and disadvantages of closed-source ecosystems often find their data siloed behind multiple walled gardens. These silos hinder hybrid and multi-cloud innovation and prevent stakeholders from extracting meaningful business insights from data.
Silos and closed-source technologies also make data governance and security across hybrid and multi-cloud environments a challenge. The process of integrating technologies with proprietary data catalogs and security policies can be arduous and takes time as well as significant resources while introducing risk.
Silos and proprietary security also limit businesses’ visibility of their data lineage, preventing enterprises from establishing proper audit trails and managing risk, making compliance challenging and inviting cybersecurity threats.

Challenge #2: Cost factors
Though hybrid and multi-cloud architectures have the potential to reduce costs considerably, ironically, they can also contribute significantly to overheads when mismanaged. Silos and a lack of data governance and visibility often lead to cloud sprawl, when businesses lose track of where applications are hosted or when applications are not being run efficiently.

Some cloud services are priced based on the consumption of compute resources, and others may charge by data scanned per query basis or bandwidth. Businesses unfamiliar with the diverse variety of pricing, fee and infrastructure sizing models used by different vendors often end up racking up astronomical costs as they struggle to stop sprawl.

How can enterprises overcome the challenges?
When left unchecked, silos, data governance and security issues, and cost overruns can cripple critical services like content delivery, enterprise resource planning, and customer relationship management – causing downtime and limiting the business’s competitiveness.

This is why businesses must stop treating hybrid and multi-cloud as an afterthought and adopt an enterprise data strategy that clearly outlines how the data architecture should be designed to support business and IT goals, manage costs, and enforce data governance and security.

A recent report and survey by Vanson Bourne and Cloudera revealed that businesses with an enterprise strategy showed more resilience since the pandemic’s start. 64% of senior decision-makers surveyed believed their businesses coped very well due to having an enterprise strategy in place for more than a year.

Consider an enterprise data strategy
To break free of silos, companies should ensure that their enterprise data strategy and architecture facilitate the rapid, free flow of data between all environments. This can be done by carefully picking and integrating a best-of-breed suite of compatible technologies in an easily manageable platform.

Open-source technologies like Kubernetes and Apache Spark offer excellent compatibility, interoperability and control, allowing businesses to simultaneously store, manage and secure data across multiple cloud vendors. They also enable enterprises to avoid vendor lock-in and benefit from open source ecosystem partnerships and rapid community innovation.

To ensure proper data governance, the businesses’ enterprise data strategy should use a unified security model to enforce a consistent set of security and data governance policies across all on-premise and cloud environments.
While open source technologies help businesses gain better control over which workloads are run and what data can be accessed, a unified security model helps organizations apply fine-grained security policies with a full data lineage and audit trail crucial for ensuring data integrity and compliance no matter where data resides.

Review your existing data strategy
Cloudera’s Enterprise Data Maturity Research found that all (100%) Indian senior decision makers surveyed agreed that the way data is handled and managed has positively impacted their organizations’ performance. 72% of these respondents also reported stronger levels of resiliency from the presence of a mature data strategy.

Conducting a comprehensive data strategy review with all stakeholders will ensure that the business has covered all its bases for today’s fast-paced environment. This includes a full assessment of the multi and hybrid cloud environment’s scalability, availability and key resource metrics for running workloads – along with the associated fees and vendor contracts.

A business’ data strategy should work towards optimizing cloud workloads, but also be cost-flexible enough to encourage innovation. This keeps the business agile now and adaptable to evolving future organizational needs. By using a well-planned, well-executed enterprise data strategy to drive a company’s cloud strategy, enterprises can leverage the cloud to drive business success.