Deep reinforcement learning-based optimal maintenance strategy for large-scale infrastructure networks under seismic risk
Infrastructure networks, such as transportation, gas, and electricity networks, are the backbone of modern society. These networks face natural hazards that are intensifying with climate change. The reliability and operability after a disaster directly impact public safety and have significant socio-economic consequences. Despite growing risks from natural and man-made hazards, the budget allocated to maintaining these networks is insufficient to ensure their robustness. Furthermore, there is a lack of comprehensive research on the optimal trade-off between the risks and budgetary limitations. Most infrastructure network maintenance policies solely address deterioration-related failures, ignoring the risk assessment related to natural disasters. As a result, infrastructure networks are vulnerable to unforeseen disasters, which can cause catastrophic damage. To make appropriate maintenance decisions and budget allocations for sustainable infrastructure, it is imperative to not only assess network reliability accurately but also to identify optimal network-level maintenance options to minimize the potential risk of disaster.
The proposed research aims to develop methods and algorithms for optimizing network maintenance policies over their lifetimes by considering the risk of natural disasters, specifically earthquakes. The key question is how to effectively overcome combinatorial explosions while minimizing efficiency losses. This general goal translates into the following specific research objectives: (1) to develop optimal sequential decision-making for managing infrastructure networks under potential seismic risk; (2) to devise a multi-agent reinforcement learning (MARL)-based network maintenance policy with hierarchical network modeling, which is applicable to large-scale networks; and (3) to test and demonstrate the application of both developed methodologies on real-world infrastructure networks. Objective 1 is approached with advanced sampling techniques to accurately quantify seismic risk (Lee et al., 2025) for optimal policies, objective 2 by a combination of hierarchically modeled networks with MARL for high scalability with respect to network size (Lee & Song, 2023). For the hierarchical network modeling, a network simplification approach will be adopted to decrease the computational complexity to a more manageable level. Consequently, this research will contribute to a more “hazard-resilient” society by addressing risks and uncertainties in maintenance planning and streamlining civil infrastructure management budgets.
Figure 1. Schematic structure of the optimal sequential decision-making.
This project is funded by the Postdoctoral Fellowship Program (nurturing next-generation researchers), supported by National Research Foundation of Korea (NRF).
References
Journal Articles
2025
RESS
Efficient seismic reliability and fragility analysis of lifeline networks using subset simulation
Various simulation-based and analytical methods have been developed to evaluate the seismic fragilities of individual structures. However, the seismic safety and resilience of a community are substantially affected by network reliability, determined not only by component fragilities but also by network topology and commodity/information flows. However, seismic reliability analyses of networks often encounter significant challenges due to complex network topologies, interdependencies among ground motions, and low failure probabilities. This paper proposes to overcome these challenges by a variance-reduction method for network fragility analysis using subset simulation. The binary network limit-state function in the subset simulation is reformulated into more informative piecewise continuous functions. The proposed limit-state functions quantify the proximity of each sample to a potential network failure domain, thereby enabling the construction of specialized intermediate failure events, which can be utilized in subset simulation and other sequential Monte Carlo approaches. Moreover, by identifying an implicit relationship between intermediate failure events and seismic intensity, we propose a technique to obtain the entire network fragility curve with a single execution of specialized subset simulation. Numerical examples demonstrate that the proposed method can effectively evaluate system-level fragility for large-scale networks.
@article{lee2025efficient,author={Lee, Dongkyu and Wang, Ziqi and Song, Junho},title={Efficient seismic reliability and fragility analysis of lifeline networks using subset simulation},journal={Reliability Engineering \& System Safety},volume={260},pages={110947},keywords={fragility, lifeline networks, network reliability, seismic reliability, subset simulation},doi={10.1016/j.ress.2025.110947},year={2025},publisher={Elsevier},dimensions={true},}
2023
RESS
Risk-informed operation and maintenance of complex lifeline systems using parallelized multi-agent deep Q-network
Lifeline systems such as transportation and water distribution networks may deteriorate with age, raising the risk of system failure or degradation. Thus, system-level sequential decision-making is essential to address the problem cost-effectively while minimizing the potential loss. Researchers have proposed to assess the risk of lifeline systems using Markov decision processes (MDPs) to identify a risk-informed operation and maintenance (O&M) policy. In complex systems with many components, however, it is potentially intractable to find MDP solutions because the numbers of states and action spaces increase exponentially. This paper proposes a multi-agent deep reinforcement learning framework, termed parallelized multi-agent deep Q-network (PM-DQN), to overcome the curse of dimensionality. The proposed method takes a divide-and-conquer strategy, in which multiple subsystems are identified by community detection, and each agent learns to achieve the O&M policy of the corresponding subsystem. The agents establish policies to minimize the decentralized cost of the cluster unit, including the factorized cost. Such learning processes occur simultaneously in several parallel units, and the trained policies are periodically synchronized with the best ones, thereby improving the master policy. Numerical examples demonstrate that the proposed method outperforms baseline policies, including conventional maintenance schemes and the subsystem-level optimal policy.
@article{lee2023risk,author={Lee, Dongkyu and Song, Junho},title={Risk-informed operation and maintenance of complex lifeline systems using parallelized multi-agent deep Q-network},journal={Reliability Engineering \& System Safety},volume={239},pages={109512},keywords={deep reinforcement learning, lifeline systems, life-cycle cost, Markov decision process, operation & maintenance, parallel processing},doi={10.1016/j.ress.2023.109512},year={2023},publisher={Elsevier},dimensions={true},}
