Integral reinforcement learning based adaptive control of a RTG crane in twisting motion

Jialu Lv; Yongli  Zhang; Bingdong Jiang; Changli  Zhang; Aihua  Jiang

doi:10.59400/sv3820

Integral reinforcement learning based adaptive control of a RTG crane in twisting motion

Jialu Lv
Tianjin Key Laboratory of Information Sensing and Intelligent Control, School of Automation and Electrical Engineering, Tianjin University of Technology and Education, Tianjin 300222, China
Yongli Zhang
Tianjin Key Laboratory of Information Sensing and Intelligent Control, School of Automation and Electrical Engineering, Tianjin University of Technology and Education, Tianjin 300222, China
Bingdong Jiang
Guangzhou Academy of Special Equipment Inspection and Testing, Guangzhou 510510, China
Changli Zhang
Guangzhou Academy of Special Equipment Inspection and Testing, Guangzhou 510510, China
Aihua Jiang
Guangzhou Academy of Special Equipment Inspection and Testing, Guangzhou 510510, China

Article ID: 3820

DOI: https://doi.org/10.59400/sv3820

Keywords: RTG crane; time-varying; adaptive control; integral reinforcement learning; dynamic parameter updating

Abstract

The rubber-tyred gantry (RTG) crane is employed as an essential piece of equipment for container handling in port operations. The RTG crane owns time-varying characteristics and parametric uncertainties. Meanwhile, the twisting of the container during its operation has a detrimental effect on the operation efficiency. In view of this, an improved adaptive control method based on integral reinforcement learning (IRL) is proposed in this paper. The mechanism model of the RTG system is developed, and the dynamic characteristics are analysed. Then, an IRL-based adaptive controller is designed and the involved positive definite Lyapunov matrix P is optimised to improve the control performance. In contrast to classical adaptive control methods, the proposed method calculates P based on real-time state variables, thereby eliminating model reliance and guaranteeing adaptive capacity. Finally, the effectiveness of the proposed method in enhancing anti-twisting performance is verified by digital and physical experiments. In the digital experiments, compared with the classical adaptive method, the load twisting settling time is reduced by 1 s, and the maximum twisting angle is decreased by approximately 0.7 degrees. In the physical experiments, despite the influence of practical friction and disturbances, the settling time is still reduced by about 1 s. These results show that the proposed scheme possesses both theoretical effectiveness and engineering practicality.

Published

2025-12-11

How to Cite

Lv, J., Zhang, Y., Jiang, B., Zhang, C., & Jiang, A. (2025). Integral reinforcement learning based adaptive control of a RTG crane in twisting motion. Sound & Vibration, 59(6). https://doi.org/10.59400/sv3820

Download Citation

Issue

Vol. 59 No. 6 (2025)

Section

Article

This work is licensed under a Creative Commons Attribution 4.0 International License.

References

[1]Yang L, Ouyang H. Precision-positioning adaptive controller for swing elimination in three-dimensional overhead cranes with distributed mass beams. ISA Transactions. 2021; 127: 449–460.

[2]Xi H, Wu Q, Ouyang H. Nonlinear control of ship-mounted rotary crane based on adaptive dynamic programming. IEEE Access. 2024; 12: 104869–104877.

[3]Zhao B, Ouyang H, Iwasaki M. Motion trajectory tracking and sway reduction for double-pendulum overhead cranes using improved adaptive control without velocity feedback. IEEE/ASME Transactions on Mechatronics. 2022; 27(5): 3648–3659.

[4]Miao X, Zhao B, Wang L, et al. Trolley regulation and swing reduction of underactuated double-pendulum overhead cranes using fuzzy adaptive nonlinear control. Nonlinear Dynamics. 2022; 109: 837–847.

[5]Yan Y, Qin Y, Zhang L, et al. Swing suppression control in quayside crane by using fuzzy logic and improved particle swarm optimization algorithm. Iranian Journal of Science and Technology, Transactions of Mechanical Engineering. 2022; 47: 1131–1144.

[6]Zhou Q, Wang K, Xiong X, et al. Optimization of bridge crane control system using fuzzy PID control and speed control of frequency converter. Journal of Physics: Conference Series. 2021; 1802: 032007.

[7]Kim J, Kiss B, Kim D, et al. Tracking control of overhead crane using output feedback with adaptive unscented Kalman filter and condition-based selective scaling. IEEE Access. 2021; 9: 108628–108639.

[8]Lu B, Cao H, Hao Y, et al. Online anti-swing trajectory planning for a practical rubber tire container gantry crane. IEEE Transactions on Industrial Electronics. 2022; 69(6): 6193–6203.

[9]Zhang Y, Li M. Fast point-to-point transportation of the portal crane via double-input inversion-based feedforward and feedback control. Optimal Control Applications and Methods. 2025; 46(4): 1402–1416.

[10]Bandong S, Napitupulu CM, Nazaruddin YY. Optimal RTGC non-linear control system based on sliding mode controller. In: Proceedings of the 2024 SICE International Symposium on Control Systems; 25–31 March 2024; Higashi-Hiroshima, Japan.

[11]Xu M, Liu L, Wang J, et al. Research on anti-swing control system of slewing crane based on fuzzy PID. PLOS One. 2024; 19(10): e0311701.

[12]Ren Z, Huang Z, Zhao T, et al. Dynamic modelling and experimental analysis of an offshore crane payload positioning system with a parallel cable-driven method. Polish Maritime Research. 2024; 31(2): 29–45.

[13]Li G, Ma X, Li Y. Adaptive anti-swing control for 7-DOF overhead crane with double spherical pendulum and varying cable length. IEEE Transactions on Automation Science and Engineering. 2024; 21(4): 5240–5251.

[14]Zhang M, Jing X, Zhou Z, et al. Transportation for 4-DOF tower cranes: A periodic sliding mode control approach. IEEE Transactions on Intelligent Transportation Systems. 2024; 25(11): 15909–15921.

[15]Li G, Ma X, Li Y. Adaptive sliding mode control based on time-delay estimation for underactuated 7-DOF tower crane. IEEE Transactions on Systems, Man, and Cybernetics: Systems. 2025; 55: 2277–2288.

[16]Zhang C, Wang G, Yang J. Research on a hydraulic cylinder's synchronous control of lifting equipment for large prefabricated components based on IGWO-BP-PID. In: Proceedings of the 2nd International Electronic Conference on Machines and Applications Session Automation and Control Systems; 18–20 June 2024; Basel, Switzerland.

[17]Ye J, Huang J. Analytical analysis and oscillation control of payload twisting dynamics in a tower crane carrying a slender payload. Mechanical Systems and Signal Processing. 2021; 158: 107763.

[18]Ngo HQ, Hong K, Kim HK, et al. Skew control of a container crane. In: Proceedings of the 2008 International Conference on Control, Automation and Systems; 14–17 October 2008; Seoul, Korea.

[19]Tho DH, Terashima K. Robust control designs of payload's skew rotation in a boom crane system. IEEE Transactions on Control Systems Technology. 2019; 27(4): 1608–1621.

[20]Zhang M, Xu W, Gu X, et al. Model reference adaptive sliding mode control of overhead crane with uncertainties. In: Proceedings of the 2019 Chinese Control Conference; 27–30 July 2019; Guangzhou, China. pp. 405–410.

[21]Cuong HM, Dong HQ, Trieu PV, et al. Adaptive fractional-order terminal sliding mode control of rubber-tired gantry cranes with uncertainties and unknown disturbances. Mechanical Systems and Signal Processing. 2021; 154: 107601.

[22]Khadija D, Said AN. A discrete repetitive adaptive sliding mode control for DC-DC buck converter. Proceedings of the Institution of Mechanical Engineers, Part I: Journal of Systems and Control Engineering. 2021; 235: 1698–1708.

[23]Anh LT, Quang H, Pham TV. Observer-based nonlinear robust control of floating container cranes subject to output hysteresis. Journal of Dynamic Systems, Measurement, and Control. 2019; 141(11): 111002.

[24]Xu C, Hu J. Adaptive robust control of a class of motor servo system with dead zone based on neural network and extended state observer. Proceedings of the Institution of Mechanical Engineers, Part I: Journal of Systems and Control Engineering. 2022; 236: 1724–1737.

[25]Yang S, Meng D, Wang H, et al. A novel learning function for adaptive surrogate-model-based reliability evaluation. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences. 2024; 382(2264): 20220395.

[26]Meng D, Zhu SP. Multidisciplinary Design Optimization of Complex Structures Under Uncertainty. CRC Press; 2024.

[27]Liu C, Chu Z, Li Y. Online adaptive data-driven control for unknown nonlinear systems with constrained-input. In: Proceedings of the 2022 First International Conference on Cyber-Energy Systems and Intelligent Energy; 14–15 January 2023; Shenyang, China. pp. 1–6.

[28]Yan L, Liu Z, Chen PC, et al. Adaptive optimal consensus of nonlinear multi-agent systems with unknown dynamics using off-policy integral reinforcement learning. Neurocomputing. 2025. 621: 129185.

[29]Chen C, Modares H, Xie K, et al. Reinforcement learning-based adaptive optimal exponential tracking control of linear systems with unknown dynamics. IEEE Transactions on Automatic Control. 2019. 64(11): 4423–4438.

[30]Guo L, Xiong W, Zhao H, et al. A nearly optimal adaptive saturation function tuning method for quasi-sliding mode control based on integral reinforcement learning. Neurocomputing. 2025. 623: 129363.

[31]Wang X, Ye X. Optimal robust control of nonlinear uncertain system via off-policy integral reinforcement learning. In: Proceedings of the 2020 Chinese Control Conference (CCC); 27–29 July2020; Shenyang, China. pp. 1928–1933.

[32]Lv Y, Chang H, Zhao J. Online adaptive integral reinforcement learning for nonlinear multi-input system. IEEE Transactions on Circuits and Systems II: Express Briefs. 2023. 70(11): 4176–4180.

[33]Salamat B, Bencic D, Elsbacher G, et al. Investigating integral reinforcement learning to achieve asymptotic stability in underactuated mechanical systems. IEEE Robotics and Automation Letters. 2024. 9(1): 191–198.

[34]Colin M, Thomas O, Grondel S, et al. Very large amplitude vibrations of flexible structures: Experimental identification and validation of a quadratic drag damping model. Journal of Fluids and Structures. 2020. 97: 103056.

[35]Salamon R, Kamiński H, Fritzkowski P. Estimation of parameters of various damping models in planar motion of a pendulum. Meccanica. 2020. 55: 1655–1677.

[36]Dong N. Adaptive Control. Beijing Institute of Technology Press; 2009. (in Chinese)

[37]Jiang H, An T, Zhang Z, et al. Adaptive fuzzy optimal control of modular robot manipulators systems via integral reinforcement learning-based value iteration algorithm. In: Proceedings of the 2024 IEEE 14th International Conference on CYBER Technology in Automation, Control, and Intelligent Systems; 16–19 July 2024; Copenhagen, Denmark. pp. 388–393.

[38]Zhang T, Yan J, Yang X, et al. Digital twin-driven formation control of ROVs: An integral reinforcement learning-based solution. IEEE Transactions on Industrial Informatics. 2024. 20(12): 14277–14286.

[39]Mahmoud E, Mammar S, Smaïli M. Model-free optimal static output feedback control using integral reinforcement learning. In: Proceedings of the 2025 33rd Mediterranean Conference on Control and Automation; 10–13 June 2025; Tangier, Morocco; pp. 25–30.

[40]Lv Y, Zhang W, Zhao J, et al. Finite-horizon optimal control for nonlinear multi-input systems with online adaptive integral reinforcement learning. IEEE Transactions on Automation Science and Engineering. 2025. 22: 802–812.

[41]Vrabie D, Vamvoudakis KG, Lewis FL. Optimal Adaptive Control and Differential Games by Reinforcement Learning Principles. The Institution of Engineering and Technology; 2012.

This paper introduces a universal framework for understanding the vibration responses of systems subjected to harmonic excitation. By examining a simplified cylinder-spring-damper model, the study refurbishes traditional scaling methods for the excitation frequency ratio and critical damping ratio. The findings indicate that in damped systems, the maximum amplitude of vibration does not align with the natural frequency. This observation leads to the introduction of a new scaling method for reduced frequency. This new approach aligns resonance peaks at the new reduced velocity of 1.0 across different damping ratios, providing a consistent characterization of vibration behavior. A new critical damping ratio of 0.707 is identified for an excited system as opposed to the traditional damping ratio of 1.0 for an unexcited system. Key properties such as maximum amplitude, phase lag, bandwidth, and quality factor are analyzed, demonstrating that the proposed reduced frequency and critical damping ratio effectively capture the dynamics of both damped and undamped excited systems. The findings offer significant insights for practical applications in engineering and various scientific fields.

This study makes significant contributions to the field of ultrasonic testing (UT) by offering a novel approach to the identification of artificially introduced defects within Japanese cedar wood (Cryptomeria japonica). The findings are of particular relevance for the heritage conservation and construction sectors, where non-invasive defect detection is paramount. The study establishes a robust framework for assessing the structural integrity of timber by correlating ultrasonic wave velocity reductions with defect size and distribution. Big-sized defects led to more substantial decreases in wave velocity. The study establishes a robust framework for assessing the structural integrity of historical timber by correlating ultrasonic wave velocity reductions with defect size and distribution. This framework has the potential to be applicable to diverse wood species and defect types.

The control of vehicle interior noise has become a critical metric for assessing noise, vibration, and harshness (NVH) in vehicles. During the initial phases of vehicle development, accurately predicting the impact of road noise on interior noise is essential for reducing noise levels and expediting the product development cycle. In recent years, data-driven methods based on machine learning have gained significant attention due to their robust capability in navigating complex data mapping relationships. Notably, surrogate models have demonstrated exceptional performance in this domain. Numerous researchers have integrated diverse intelligent algorithms into the study of vehicle noise, leveraging advantages such as the elimination of precise modeling requirements, extensive solution space exploration, continuous learning from data, and robust algorithmic versatility. However, in NVH engineering applications, data-driven models face inherent limitations, particularly in interpretability and stability. To address these issues, this paper introduces an improved Long Short-Term Memory (LSTM) network that combines knowledge and data. Inspired by the physical information neural network concept, this approach incorporates values calculated through empirical formulas into the neural network as constraints. Comparative assessments with traditional LSTM networks highlight the advantages of this deep learning model. By integrating empirical formulas constraints, the model not only enhances interpretability but also achieves robust generalization with fewer data samples. The proposed method is validated on a specific vehicle model, showing significant improvements in prediction accuracy and efficiency.