Condition monitoring of train transmission systems based on multimodal fusion improved transformer network

Cun Shi; Shutong Zhao; Xiying Chen; Shaoping Wang; Di Liu

doi:10.59400/sv2904

Condition monitoring of train transmission systems based on multimodal fusion improved transformer network

Cun Shi School of Automation Science and Electrical Engineering, Beihang University, Beijing 100191, China
Shutong Zhao School of Automation Science and Electrical Engineering, Beihang University, Beijing 100191, China
Xiying Chen Beijing Institute of Precision Electromechanical Control, Beijing100000, China; Innovation Center for Control Actuators, Beijing 100000, China
Shaoping Wang School of Automation Science and Electrical Engineering, Beihang University, Beijing 100191, China
Di Liu School of Automation Science and Electrical Engineering, Beihang University, Beijing 100191, China

Article ID: 2904

DOI: https://doi.org/10.59400/sv2904

Keywords: condition monitoring; multi-modal fusion; transformer; self-attention; MFITN

Abstract

The train transmission system is a critical component of railway operations, playing a pivotal role in ensuring service safety and reliability. However, existing condition monitoring approaches face two major challenges: (1) the coupling of rich multimodal signals, such as vibration, acoustics, current, and rotational speed, is often overlooked, limiting monitoring accuracy; (2) the small data problem in multimodal signals adversely affects the performance of neural networks. To address these issues, this paper proposes a Multimodal Fusion Improved Transformer Network for Condition Monitoring of Train Transmission Systems. The proposed network first explores interdependencies among different modalities of signals and compresses data to reduced dimensions through correlation analysis. It then infers global dependencies through computing self-attention scores based on Q, K, and V matrices. The approach is better than traditional CNN-based models in handling single-modality constraints, with the former demonstrated to be more accurate and trustworthy on publicly available datasets.

Published

2025-04-25

How to Cite

Shi, C., Zhao, S., Chen, X., Wang, S., & Liu, D. (2025). Condition monitoring of train transmission systems based on multimodal fusion improved transformer network. Sound & Vibration, 59(2), 2904. https://doi.org/10.59400/sv2904

Download Citation

Issue

Vol. 59 No. 2 (2025)

Section

Article

This work is licensed under a Creative Commons Attribution 4.0 International License.

References

[1]World Bank. Bhutan urban policy notes: Urban resilience. Available online: https://documents1.worldbank.org/curated/en/807961559553043410/pdf/Bhutan-Urban-Policy-Notes-Urban-Resilience.pdf (accessed on 10 February 2025).

[2]Yin J, Tang T, Yang L, et al. Research and development of automatic train operation for railway transportation systems: A survey. Transportation Research Part C: Emerging Technologies. 2017; 85: 548–572. doi: 10.1016/j.trc.2017.09.009

[3]Jin Y, Chen X. Research on aerodynamic characteristics of high-velocity train bogies. Journal of Engineering and Applied Science. 2024; 71(1). doi: 10.1186/s44147-024-00542-3

[4]Gavrilovic B, Baboshin VA. Simulations of the operation of the fast light innovative regional train from “Serbian Railways” in traction and electric braking mode. Mechanical Engineering Advances. 2023; 2(1): 1214. doi: 10.59400/mea.v2i1.1214

[5]Shiau JY, Wang ST. Bogie Stability Control and Management Using Data Driven Analysis Techniques for High-Speed Trains. Applied Sciences. 2022; 12(5): 2389. doi: 10.3390/app12052389

[6]Bernal E, Spiryagin M, Cole C. Onboard Condition Monitoring Sensors, Systems and Techniques for Freight Railway Vehicles: A Review. IEEE Sensors Journal. 2019; 19(1): 4–24. doi: 10.1109/jsen.2018.2875160

[7]He Z, Guo H, Liu H, et al. A Sound Quality Evaluation Method for Vehicle Interior Noise Based on Auditory Loudness Model. Sound & Vibration. 2024; 58(1): 47–58. doi: 10.32604/sv.2024.045470

[8]Huang W, Xu J. Engineering vibration recognition using CWT-ResNet. Sound & Vibration. 2025; 59(1): 2242. doi: 10.59400/sv2242

[9]Mousavi SA, Taghipour M. Turbine vibration condition monitoring in region 3. Mechanical Engineering Advances. 2023; 1(1). doi: 10.59400/mea.v1i1.219

[10]Peng C, Cheng S, Sun M, et al. Prediction of Sound Transmission Loss of Vehicle Floor System Based on 1D-Convolutional Neural Networks. Sound & Vibration. 2024; 58(1): 25–46. doi: 10.32604/sv.2024.046940

[11]Randall RB. Vibration-based condition monitoring: Industrial, automotive and aerospace applications. John Wiley & Sons; 2021.

[12]Zhang Y, Tang X, Xu S, et al. Deep Learning-Based State-of-Health Estimation of Proton-Exchange Membrane Fuel Cells under Dynamic Operation Conditions. Sensors. 2024; 24(14): 4451. doi: 10.3390/s24144451

[13]Tsunashima H, Takikawa M. Monitoring the Condition of Railway Tracks Using a Convolutional Neural Network. Recent Advances in Wavelet Transforms and Their Applications; 2022. doi: 10.5772/intechopen.102672

[14]Zhang J, Liu M, Deng W, et al. Research on electro-mechanical actuator fault diagnosis based on ensemble learning method. International Journal of Hydromechatronics. 2024; 7(2): 113–131. doi: 10.1504/ijhm.2024.138231

[15]Shim J, Koo J, Park Y. A Methodology of Condition Monitoring System Utilizing Supervised and Semi-Supervised Learning in Railway. Sensors. 2023; 23(22): 9075. doi: 10.3390/s23229075

[16]Zou Y, Zhang Y, Mao H. Fault diagnosis on the bearing of traction motor in high-speed trains based on deep learning. Alexandria Engineering Journal. 2021; 60(1): 1209–1219. doi: 10.1016/j.aej.2020.10.044

[17]Fu D, Liu J, Zhong H, et al. A novel self-supervised representation learning framework based on time-frequency alignment and interaction for mechanical fault diagnosis. Knowledge-Based Systems. 2024; 295: 111846. doi: 10.1016/j.knosys.2024.111846

[18]Wang H, Sun W, Sun W, et al. A novel tool condition monitoring based on Gramian angular field and comparative learning. International Journal of Hydromechatronics. 2023; 6(2): 93. doi: 10.1504/ijhm.2023.130510

[19]Davies A. Handbook of condition monitoring: techniques and methodology. Springer Science & Business Media; 2012.

[20]Xu Y, Wang H, Liu Z, et al. Self-Supervised Defect Representation Learning for Label-Limited Rail Surface Defect Detection. IEEE Sensors Journal. 2023; 23(23): 29235–29246. doi: 10.1109/jsen.2023.3324668

[21]Zhuang L, Qi H, Wang T, et al. A Deep-Learning-Powered Near-Real-Time Detection of Railway Track Major Components: A Two-Stage Computer-Vision-Based Method. IEEE Internet of Things Journal. 2022; 9(19): 18806–18816. doi: 10.1109/jiot.2022.3162295

[22]Logan D, Mathew J. Using The Correlation Dimension for Vibration Fault Diagnosis of Rolling Element Bearings—I. Basic Concepts. Mechanical Systems and Signal Processing. 1996; 10(3): 241–250. doi: 10.1006/mssp.1996.0018

[23]Wang WJ, McFadden PD. Early detection of gear failure by vibration analysis i. calculation of the time-frequency distribution. Mechanical Systems and Signal Processing. 1993; 7(3): 193–203. doi: 10.1006/mssp.1993.1008

[24]Li W, Zhu Z, Jiang F, et al. Fault diagnosis of rotating machinery with a novel statistical feature extraction and evaluation method. Mechanical Systems and Signal Processing. 2015; 50-51: 414–426. doi: 10.1016/j.ymssp.2014.05.034

[25]Wang C, Dou M, Li Z, et al. Data-driven prognostics based on time-frequency analysis and symbolic recurrent neural network for fuel cells under dynamic load. Reliability Engineering & System Safety. 2023; 233: 109123. doi: 10.1016/j.ress.2023.109123

[26]Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need. Advances in Neural Information Processing Systems; 2017.

[27]Zhang X, Wang H, Wang C, et al. Time-segment-wise feature fusion transformer for multi-modal fault diagnosis. Engineering Applications of Artificial Intelligence. 2024; 138: 109358. doi: 10.1016/j.engappai.2024.109358

[28]Ma Y, Wang L, Chen F, et al. DSAN: An Integrated Bearing Diagnosis Strategy with Dual-Spectral Feature Transform and Adaptive Position Correction Algorithm. IEEE Transactions on Instrumentation and Measurement. 2025; 74: 1–11. doi: 10.1109/tim.2024.3502804

[29]Wang H, Liu Z, Ge Y, et al. Self-supervised signal representation learning for machinery fault diagnosis under limited annotation data. Knowledge-Based Systems. 2022; 239: 107978. doi: 10.1016/j.knosys.2021.107978

[30]Ding Y, Jia M, Miao Q, et al. A novel time—frequency Transformer based on self-attention mechanism and its application in fault diagnosis of rolling bearings. Mechanical Systems and Signal Processing. 2022; 168: 108616. doi: 10.1016/j.ymssp.2021.108616

[31]Yasuda M, Ohishi Y, Saito S, et al. Multi-View and Multi-Modal Event Detection Utilizing Transformer-Based Multi-Sensor Fusion. In: Proceedings of the ICASSP 2022–2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP); 23–27 May 2022; Singapore. doi: 10.1109/icassp43922.2022.9746006

[32]Ahmed HOA, Nandi AK. Convolutional-Transformer Model with Long-Range Temporal Dependencies for Bearing Fault Diagnosis Using Vibration Signals. Machines. 2023; 11(7): 746. doi: 10.3390/machines11070746

[33]Dou B, Zhu Z, Merkurjev E, et al. Machine Learning Methods for Small Data Challenges in Molecular Science. Chemical Reviews. 2023; 123(13): 8736–8780. doi: 10.1021/acs.chemrev.3c00189

[34]Li C, Li S, Feng Y, et al. Small data challenges for intelligent prognostics and health management: a review. Artificial Intelligence Review. 2024; 57(8). doi: 10.1007/s10462-024-10820-4

[35]Zhu Q, Sun B, Zhou Y, et al. Sample Augmentation for Intelligent Milling Tool Wear Condition Monitoring Using Numerical Simulation and Generative Adversarial Network. IEEE Transactions on Instrumentation and Measurement. 2021; 70: 1–10. doi: 10.1109/tim.2021.3077995

[36]Ding A, Qin Y, Wang B, et al. Brownian Distance Covariance-Based Few-Shot Learning Framework Considering Noisy Labels for Fault Diagnosis of Train Transmission Systems. IEEE Transactions on Industrial Informatics. 2025; 21(1): 136–145. doi: 10.1109/tii.2024.3441645

[37]Tian X, Jin Y, Tang X. Local-Global Transformer Neural Network for temporal action segmentation. Multimedia Systems. 2022; 29(2): 615–626. doi: 10.1007/s00530-022-00998-4

[38]Hei Z, Sun W, Yang H, et al. Novel domain-adaptive Wasserstein generative adversarial networks for early bearing fault diagnosis under various conditions. Reliability Engineering & System Safety. 2025; 257: 110847. doi: 10.1016/j.ress.2025.110847

[39]Zhou AY, Barati Farimani A. FaultFormer: Pretraining Transformers for Adaptable Bearing Fault Classification. IEEE Access. 2024; 12: 70719–70728. doi: 10.1109/access.2024.3399670

[40]Tanha J, Abdi Y, Samadi N, et al. Boosting methods for multi-class imbalanced data classification: an experimental review. Journal of Big Data. 2020; 7(1). doi: 10.1186/s40537-020-00349-y

[41]Peng L, Jian S, Li M, et al. A unified multimodal classification framework based on deep metric learning. Neural Networks. 2025; 181: 106747. doi: 10.1016/j.neunet.2024.106747

[42]Wang D, Guo X, Tian Y, et al. TETFN: A text enhanced transformer fusion network for multimodal sentiment analysis. Pattern Recognition. 2023; 136: 109259. doi: 10.1016/j.patcog.2022.109259

[43]Qi GJ, Luo J. Small Data Challenges in Big Data Era: A Survey of Recent Progress on Unsupervised and Semi-Supervised Methods. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2022; 44(4): 2168–2187. doi: 10.1109/tpami.2020.3031898

[44]He Z, Wang S, Shi J, et al. Physics-informed neural network supported wiener process for degradation modeling and reliability prediction. Reliability Engineering & System Safety. 2025; 258: 110906. doi: 10.1016/j.ress.2025.110906

[45]Li X, Wan S, Liu S, et al. Bearing fault diagnosis method based on attention mechanism and multilayer fusion network. ISA Transactions. 2022; 128: 550–564. doi: 10.1016/j.isatra.2021.11.020

[46]Xu Z, Li C, Yang Y. Fault diagnosis of rolling bearings using an Improved Multi-Scale Convolutional Neural Network with Feature Attention mechanism. ISA Transactions. 2021; 110: 379–393. doi: 10.1016/j.isatra.2020.10.054

[47]Ding A, Qin Y, Wang B, et al. Evolvable graph neural network for system-level incremental fault diagnosis of train transmission systems. Mechanical Systems and Signal Processing. 2024; 210: 111175. doi: 10.1016/j.ymssp.2024.111175

[48]Fernandez-Bobadilla HA, Martin U. Modern Tendencies in Vehicle-Based Condition Monitoring of the Railway Track. IEEE Transactions on Instrumentation and Measurement. 2023; 72: 1–44. doi: 10.1109/tim.2023.3243673

This paper introduces a universal framework for understanding the vibration responses of systems subjected to harmonic excitation. By examining a simplified cylinder-spring-damper model, the study refurbishes traditional scaling methods for the excitation frequency ratio and critical damping ratio. The findings indicate that in damped systems, the maximum amplitude of vibration does not align with the natural frequency. This observation leads to the introduction of a new scaling method for reduced frequency. This new approach aligns resonance peaks at the new reduced velocity of 1.0 across different damping ratios, providing a consistent characterization of vibration behavior. A new critical damping ratio of 0.707 is identified for an excited system as opposed to the traditional damping ratio of 1.0 for an unexcited system. Key properties such as maximum amplitude, phase lag, bandwidth, and quality factor are analyzed, demonstrating that the proposed reduced frequency and critical damping ratio effectively capture the dynamics of both damped and undamped excited systems. The findings offer significant insights for practical applications in engineering and various scientific fields.

This study makes significant contributions to the field of ultrasonic testing (UT) by offering a novel approach to the identification of artificially introduced defects within Japanese cedar wood (Cryptomeria japonica). The findings are of particular relevance for the heritage conservation and construction sectors, where non-invasive defect detection is paramount. The study establishes a robust framework for assessing the structural integrity of timber by correlating ultrasonic wave velocity reductions with defect size and distribution. Big-sized defects led to more substantial decreases in wave velocity. The study establishes a robust framework for assessing the structural integrity of historical timber by correlating ultrasonic wave velocity reductions with defect size and distribution. This framework has the potential to be applicable to diverse wood species and defect types.

The control of vehicle interior noise has become a critical metric for assessing noise, vibration, and harshness (NVH) in vehicles. During the initial phases of vehicle development, accurately predicting the impact of road noise on interior noise is essential for reducing noise levels and expediting the product development cycle. In recent years, data-driven methods based on machine learning have gained significant attention due to their robust capability in navigating complex data mapping relationships. Notably, surrogate models have demonstrated exceptional performance in this domain. Numerous researchers have integrated diverse intelligent algorithms into the study of vehicle noise, leveraging advantages such as the elimination of precise modeling requirements, extensive solution space exploration, continuous learning from data, and robust algorithmic versatility. However, in NVH engineering applications, data-driven models face inherent limitations, particularly in interpretability and stability. To address these issues, this paper introduces an improved Long Short-Term Memory (LSTM) network that combines knowledge and data. Inspired by the physical information neural network concept, this approach incorporates values calculated through empirical formulas into the neural network as constraints. Comparative assessments with traditional LSTM networks highlight the advantages of this deep learning model. By integrating empirical formulas constraints, the model not only enhances interpretability but also achieves robust generalization with fewer data samples. The proposed method is validated on a specific vehicle model, showing significant improvements in prediction accuracy and efficiency.