Data Lake at NHAI: Transforming the Way Major Infrastructure Projects Are Delivered
By Chandan Chowdhury
“Gyaan Kosh” Shared Learning Resources: Knowledge Bank (Initiative by the Capacity Building Commission, Government of India)
“Gyaan Kosh” Shared Learning Resources: Knowledge Bank (Initiative by the Capacity Building Commission, Government of India)
Citation
Chowdhury, Chandan. (2024). Data Lake at NHAI: Transforming the Way Major Infrastructure Projects Are Delivered “Gyaan Kosh” Shared Learning Resources: Knowledge Bank (Initiative by the Capacity Building Commission, Government of India) .
Copyright
“Gyaan Kosh” Shared Learning Resources: Knowledge Bank (Initiative by the Capacity Building Commission, Government of India), 2024
Share:
Abstract
The National Highways Authority of India (NHAI) recently achieved one of the significant milestones of the digitalization of processes by implementing Data Lake, a massive, centralized data storage and project management system. The project was managed by the Information and Technology Project Management Unit (IT-PMU) headed by Deepak Saxena, Chief General Manager- Information and Technology (CGM-IT). The team successfully digitalized NHAI's several lengthy administrative procedures and processes, which helped NHAI significantly reduce its cost and time overruns, improve road safety, bring transparency and accountability.
The digitalization journey happened in two phases: ‘Data Lake 1.0’ was developed in 2020 with the vision to digitize ongoing project-related information but also to do the same exercise for legacy project records. It had dedicated role-based logins for all NHAI offices and other stakeholders along with provisions to capture crucial project-related activity at every project stage. Implementation of Data Lake 1.0 enabled NHAI leadership to identify some of the key challenges in the digitalization of infrastructure projects in India, which led them to launch ‘Data Lake 2.0’ in 2023, which is transaction and workflow driven system. Data Lake 2.0 also adopted Artificial Intelligence (AI), Machine Learning (ML), and Geographic Information System (GIS) for advanced analytics and decision support.
Chairman, NHAI: “NHAI took a 360-degree approach to digitalize and re-engineer most of the key activities in the life cycle of a highway project, viz. planning, detailed project reports (DPR), construction, maintenance, tolling and upgradation of the highway network, to improve the system’s efficiency and ease of doing business.
For effective management of the projects at NHAI, digitalization was key for our strategy. Data Lake 1.0 was one of the first step for NHAI to completely revamp operations at NHAI. It was the beginning of a comprehensive project management system at NHAI.
Features of subsequent version i.e. Data Lake 2.0 were designed to strengthen the adherence to the various processes through digitalization. The constructive feedback from the users was crucial for us to formulate Data Lake features. It helped us make them more conducive to the end users rather than imposed. We preferred to maintain an agile approach and incorporate various types of requirements of our stakeholders so that Data Lake 2.0 felt more user–centric and user–driven.
The success of our in-house-built Data Lake 2.0 sets a benchmark for many other government authorities and ministries by showcasing what digitalization can achieve and how it can be done at such a substantial scale. Data Lake 2.0 has not only streamlined and digitalized the processes but also supports centralized planning by fostering inter-departmental collaboration across various ministries and portfolios. We have many government-to-government (G2G) applications interfaced with Data Lake 2.0 that help them use the information within it for their purpose.
In the beginning of 2023, NHAI had already started planning an even more advanced system viz. ‘Data Lake 3.0’. They aimed to enhance the system in a manner that would include automation and provide in depth data-driven insights for the users and beneficiaries, aiding in right cognitive decisions using AI and ML algorithms. There was very little time and a lot to execute. How NHAI take this project to the next level?
The digitalization journey happened in two phases: ‘Data Lake 1.0’ was developed in 2020 with the vision to digitize ongoing project-related information but also to do the same exercise for legacy project records. It had dedicated role-based logins for all NHAI offices and other stakeholders along with provisions to capture crucial project-related activity at every project stage. Implementation of Data Lake 1.0 enabled NHAI leadership to identify some of the key challenges in the digitalization of infrastructure projects in India, which led them to launch ‘Data Lake 2.0’ in 2023, which is transaction and workflow driven system. Data Lake 2.0 also adopted Artificial Intelligence (AI), Machine Learning (ML), and Geographic Information System (GIS) for advanced analytics and decision support.
Chairman, NHAI: “NHAI took a 360-degree approach to digitalize and re-engineer most of the key activities in the life cycle of a highway project, viz. planning, detailed project reports (DPR), construction, maintenance, tolling and upgradation of the highway network, to improve the system’s efficiency and ease of doing business.
For effective management of the projects at NHAI, digitalization was key for our strategy. Data Lake 1.0 was one of the first step for NHAI to completely revamp operations at NHAI. It was the beginning of a comprehensive project management system at NHAI.
Features of subsequent version i.e. Data Lake 2.0 were designed to strengthen the adherence to the various processes through digitalization. The constructive feedback from the users was crucial for us to formulate Data Lake features. It helped us make them more conducive to the end users rather than imposed. We preferred to maintain an agile approach and incorporate various types of requirements of our stakeholders so that Data Lake 2.0 felt more user–centric and user–driven.
The success of our in-house-built Data Lake 2.0 sets a benchmark for many other government authorities and ministries by showcasing what digitalization can achieve and how it can be done at such a substantial scale. Data Lake 2.0 has not only streamlined and digitalized the processes but also supports centralized planning by fostering inter-departmental collaboration across various ministries and portfolios. We have many government-to-government (G2G) applications interfaced with Data Lake 2.0 that help them use the information within it for their purpose.
In the beginning of 2023, NHAI had already started planning an even more advanced system viz. ‘Data Lake 3.0’. They aimed to enhance the system in a manner that would include automation and provide in depth data-driven insights for the users and beneficiaries, aiding in right cognitive decisions using AI and ML algorithms. There was very little time and a lot to execute. How NHAI take this project to the next level?