Department of Civil Engineering, University of Birmingham, Birmingham, B15 2TT, UK.
Sci Rep. 2023 Feb 10;13(1):2439. doi: 10.1038/s41598-023-29526-8.
Railway maintenance is a complex and complicated task in the railway industry due to the number of its components and relationships. Ineffective railway maintenance results in excess cost, defective railway structure and components, longer possession time, poorer safety, and lower passenger comfort. Of the three main maintenance approaches, predictive maintenance is the trendy one, and is proven that it provides the highest efficiency. However, the implementation of predictive maintenance for the railway industry cannot be done without an efficient tool. Normally, railway maintenance is corrective when some things fail or preventive when maintenance is routine. A novel approach using an integration between deep reinforcement learning and digital twin is proposed in this study to improve the efficiency of railway maintenance which other techniques such as supervised and unsupervised learning cannot provide. In the study, Advantage Actor Critic (A2C) is used to develop a reinforcement learning model and agent to fulfill the need of the study. Real-world field data over four years and 30 km. is obtained and applied for developing the reinforcement learning model. Track geometry parameters, railway component defects, and maintenance activities are used as parameters to develop the reinforcement learning model. Rewards (or penalties) are calculated based on maintenance costs and occurring defects. The new breakthrough exhibits that using reinforcement learning integrated with digital twin can reduce maintenance activities by 21% and reduce the occurring defects by 68%. Novelties of the study are the use of A2C which is faster and provides better results than other traditional techniques such as Deep Q-learning (DQN), each track geometry parameter is considered without combining into a track quality index, filed data are used to develop the reinforcement learning model, and seven independent actions are included in the reinforcement learning model. This study is the world's first to contribute a new guideline for applying reinforcement learning and digital twins to improve the efficiency of railway maintenance, reduce the number of defects, reduce the maintenance cost, reduce the possession time for railway maintenance, improve the overall safety of the railway operation, and improve the passenger comfort which can be seen from its results.
铁路维护在铁路行业是一项复杂而复杂的任务，因为它的组成部分和关系数量众多。铁路维护不力会导致成本过高、铁路结构和部件缺陷、占用时间延长、安全性降低、乘客舒适度降低。在三种主要的维护方法中，预测性维护是一种流行的方法，并且已经证明它提供了最高的效率。然而，要在铁路行业实施预测性维护，就离不开一种高效的工具。通常，铁路维护是在出现故障时进行纠正性维护，或者在进行例行维护时进行预防性维护。本研究提出了一种利用深度强化学习和数字孪生技术相结合的新方法，以提高铁路维护的效率，而其他技术，如监督学习和无监督学习，无法提供这种效率。在研究中，使用优势演员评论家（A2C）来开发强化学习模型和代理，以满足研究的需要。在研究中，使用了四年和 30 公里的实际现场数据来开发强化学习模型。轨道几何参数、铁路部件缺陷和维护活动被用作开发强化学习模型的参数。奖励（或罚款）是根据维护成本和发生的缺陷来计算的。新的突破表明，使用强化学习与数字孪生相结合，可以减少 21%的维护活动，并减少 68%的缺陷发生。这项研究的新颖之处在于使用 A2C，它比其他传统技术（如深度 Q 学习（DQN））更快，提供更好的结果，每个轨道几何参数都被考虑在内，而没有组合成一个轨道质量指数，现场数据被用于开发强化学习模型，并且强化学习模型中包含了七个独立的动作。本研究首次为应用强化学习和数字孪生技术来提高铁路维护效率、减少缺陷数量、降低维护成本、缩短铁路维护占用时间、提高铁路运营整体安全性和提高乘客舒适度提供了新的指导，这可以从其结果中看出。
