A playbook for using and deploying Generative AI: Insights from Harnath Babu, CIO, KPMG India

Generative AI has become the cornerstone of innovation, revolutionizing the way businesses operate, make decisions, and interact with data. From semantic search to reasoning-based searches, the landscape has evolved, demanding organizations to navigate this paradigm shift with precision. The stakes are high, and the potential is limitless. It’s not just about adopting cutting-edge technology but orchestrating it strategically to enhance efficiency, drive insights, and transform the very fabric of how we approach problem-solving. At one of our conferences last year, Technology Senate North, we had the pleasure of having a fireside chat with Harnath Babu, CIO, KPMG India

As the conversation is extremely relevant for CIOs, we thought of reproducing the key points from the insightful interaction we had with him at the Technology Senate North conference. We believe that the insights shared in the conversation will be extremely beneficial to every organization which is starting on its AI journey. Starting from the critical first steps in adopting generative AI, dissecting the considerations for CIOs, and exploring the nuanced decisions around public and private instances – the insights shared here aim to be a compass for organizations navigating the seas of generative AI, offering practical guidance and foresight into the potential pitfalls and soaring possibilities.

Some edited excerpts:

What should be the starting point for leveraging generative AI? What are some of the recommended key steps, especially for CIOs?
The starting point for any organization delving into generative AI is a crystal-clear business objective. Technology integration should align seamlessly with business goals, whether it’s addressing customer or employee issues or gaining deeper insights from existing data. The second critical aspect involves having the right and relevant data for training AI models. Since AI is learning and unlearning from the data provided, it often hallucinates because of the enormous information. Hence, it is crucial to train large-language models on relevant data. Caution is advised in real-time training to avoid generating false outputs. In essence, a clear business objective, well-defined use cases, and the right data are fundamental are imperative for any successful AI model.

You mentioned the importance of having the right data. Would you advise a private instance, especially for CIOs considering the various options available, such as public cloud services?
Certainly. When adopting new technology, understanding its implications is crucial. Creating a private instance allows customization to specific requirements, avoiding unnecessary complexities and costs. For instance, a school will not hire a Harvard professor to teach a kindergarten class. Similarly, a large-language model is like a Harvard professor, whose immense knowledge may not be of any value to your use case. The decision to opt for a private instance hinges on the level of customization needed. Starting with Microsoft’s Open AI or AWS Bedrock and then further fine-tuning those models to get the desired output is the right approach. Striking a delicate balance between customization and managing associated risks is key. It’s advisable to create a customized instance based on available models, fine-tuning them to respond appropriately to specific organizational needs.

How do you assess if an organization is ready for deploying generative AI models? Are there specific maturity levels or frameworks to consider? Also, what about the required skill sets?
Deploying generative AI involves assessing an organization’s readiness on multiple fronts. A clear understanding of use cases and well-defined objectives is paramount. Whether deploying for process efficiency, analytics, or customer service, objectives must be crystallized. Beyond this, having the right data is critical, and organizations should be acutely aware of potential biases in model responses, particularly in sensitive areas like healthcare. Skill sets, encompassing both technical prowess and domain expertise, play a pivotal role in ensuring accurate training and fine-tuning. Establishing a governance board to monitor usage and address security concerns is indispensable, emphasizing a methodical, step-by-step approach, starting with smaller projects.

Considering the various options available, should organizations use private instances, especially in terms of security and compliance?
When contemplating generative AI adoption, organizations must carefully weigh the pros and cons of utilizing private instances. Understanding the kind of data to feed into models and the anticipated outcomes is crucial. While public services may raise concerns about data leakage, creating a private instance offers a more controlled environment. The choice ultimately rests on specific requirements and the level of customization needed. For example, dealing with legal case laws might necessitate a custom-tailored model on a private instance. Striking a delicate balance between customization and managing associated risks is key.

Can generative AI be applied to enhance security measures?
Indeed, Generative AI stands to significantly bolster cybersecurity efforts by analysing extensive volumes of security-related data. This technology excels in providing insights and correlations between data points, enabling more informed decision-making. Integrating generative AI into Security Orchestration, Automation, and Response (SOAR) playbooks has the potential to elevate security measures by not only identifying threats but also automatically taking actions to mitigate them. While still in its early stages, the promising potential for generative AI to enhance security is evident.

Is generative AI a hype, or does it hold long-term potential?
Generative AI has undeniably generated a considerable level of hype, yet its active adoption across diverse industries suggests a sustained and impactful presence. The continuous evolution and expanding applications of this technology point towards a promising trajectory, indicating that it is more than just a passing trend. While currently experiencing the peak of the hype cycle, generative AI seems poised for a sustained and impactful journey, contributing significantly to ongoing advancements across various sectors.