India has created perhaps the biggest data repository in the world. It now faces the challenge of using this mammoth information for better governance
In modern days, data is an asset and opportunity, and often it has been referred to as a the new oil. Interestingly, India stands tall in this field, thanks to massive focus of the government on digitalization. Today, perhaps India owns the largest complex of data gathered via digitalization of records for purposes like IDs, passports and payment of subsidies. There are a number of areas where huge projects have been implemented, like Aadhaar, passports and the inception of MCA 21, a Mission Mode Project of the Corporate Affairs Ministry. All this has opened up a lot of opportunities to apply this data to improve the citizens’ service experience and government efficiency, especially in delivery of services and to boost business.
According to experts, big data analytics, which then merges into fields like deep learning, machine learning and artificial intelligence (AI), has tremendous possibilities. With internet of things (IoT) now coming onto its own, a whole new world is opening up for data. Big data is characterized by its volume, variety and speed and the analytics involves its processing in a cost-effective way in order to draw conclusions for their useful application. Fortunately, in all three key areas, there has been growth – processing power has increased by 40 per cent in last couple of years and the cost of storing data has also gone down by 500 per cent.
However India, which has created one of the biggest data repository in the world, now faces the challenge of using this information for better governance. “In India, both state and central governments are embracing this new data age. Government organizations are collecting massive amount of data every day and with Prime Minister Narendra Modi pushing Digital India and tech-driven Goods and Services Tax (GST), data collection has spurred like never before. Hence, use of data analytics tools to make sense of the mammoth amount of the data is an inevitable,” says Dinesh Tyagi, CEO, CSC E-Governance Services India.
Over the past few year, government has realized this potential, leading to efforts aimed at harnessing big data analytics. For example, recently Vasundhara Raje led BJP government in Rajasthan signed an agreement with US-based data warehouse firm Teradata to help them create a common data and analytics platform that can be used by all government departments across the state to collate and utilize data effectively.
According to the state government, this is being done to provide a big data analytics environment to analyze state level data in order to enhance citizen services and engagement as well as increase the efficiency of departmental cross functional operations. The ‘Big Data Environment project’ was created to benefit the state by “future-proofing” its IT architecture in order to build a state level integrated data platform that will become a common source of structured and unstructured data for data mining and analytics initiatives for all government departments.
The project enables a 360-degree citizens’ view by unifying multiple databases. Department of Information Technology & Communication (DoIT & C) is now able to improve the user experience of government services hosted on eMitra website by analysing the log, effectively addressing multi-lingual citizen grievances along with sentiment analysis. It will also be able to offer actionable intelligence to different departments to improve citizen services.
“Presently, government data is managed within department-level silos assisted by department-centric IT applications. Under the new ‘Big Data Environment’ project, we have developed solutions to hold integrated data being generated real time across the state to help better address analytical requirements that facilitate state-level decision-making. Implementation of big data analytics solutions will empower government departmental users with improved analytical insights and thereby enable more effective and timely decision-making as well as improve the citizen engagement processes helping the authorities to deliver better services and greater citizen satisfaction,” informs Arun Chauhan, Additional Director, Department of IT and Communication, Government of Rajasthan.
On the other hand, some of the states that have been very active on the use of technology for the governance like Andhra Pradesh, Telangana and Karnataka have a data analytics policy in place. The Andhra Pradesh government is employing big data and analytics to strengthen the decision-making process and improve governance.
Andhra Pradesh has digitized the data of every department and launched real time monitoring of the performance. The Chief Minister monitors the performance of each department through CM Office Real Time Executive (CORE) dashboard. Unlike conventional method of updating data using excel, the data is updated automatically.
“With data analytics, we are getting insights into the performance of government policies, analyze trends and predict the future behavior of people and systems so that timely corrective measures could be taken. This helps in proper insight and corrective intervention for better governance,” says J Satyanarayana, Advisor to AP Government.
Similarly, state of Odisha is banking on data analytics technology for ensuring that the least served areas get the benefit of government welfare schemes. For example, when LPG distribution centre was being opened, it was decided that centres would be opened in areas that need them the most and not where LPG companies see the most demand. This was done with the help of geo spatial analytics (GIS).
The good thing is that today, there are different sources of data that is available and data collection has become easy, this has led to innovation as a diverse data from specialized data sources drastically improves the accuracy of predictive and prescriptive analytics.
“Traditionally, these advanced analytics were generated using statistical models created on only representational samples of data. Once these models are created, a second relatively larger set of data was used to test or validate these models iteratively. Upon successful validation, this model was used to evaluate or score test of entire data set. With the advent of big data, the first pass statistical model itself is created using much larger, and at times complete data sets. This not only allows introduction of additional predictor variables but also increases forecast accuracy many folds. Therefore, it may be concluded that what was the bad situation of yesteryears due to multiple source data systems, in silos processing etc, is the welcome thing in analytics when done on big data platforms,” says Amit Sharma, Principal Technology Architect, Infosys.
According to experts, analytics will no longer be an after-thought, rather it will become an integral part of government service delivery. Because government organization needs to know about the effectiveness of their service delivery and hence, they would require the tools to measure efficiency and performance. They need to know how much work is being completed in order to determine how well it is operating.
All the good things come with some challenge, so does data analytics. It faces challenges like security and shortage of human resources. A data warehouse has huge challenge of cyber security because in one case of breach, all the data could be affected. Therefore, there are experts who say that rather having a data lake, government need to think about collaboration such as how police and Aadhar can collaborate. “We need to think how police and passport division can collaborate for verification of passport, leading to faster delivery of passport to citzizen,” says Golok Kumar Simli, Chief of Technology, Passport Seva Project, Ministry of External Affairs.
On human resource challenge, Satyanarayana comments, “We need to build a cadre of data scientists in the public sector. If they know the domain well, then they can do a good job.”