Visual Monitoring of Railway Obstacles using Deep Learning-Based Methods

Document Type : Scientific - Research

Authors
Simin Koulaeian 1
Mohammad Sandidzadeh 2
1 M.Sc., Department of Control and Signaling engineering, school of Railway Engineering, Iran University of Science and Technology, Tehran, Iran
2 Associate Professor, school of Railway Engineering, Iran University of Science and Technology, Tehran, Iran
10.22119/jte.2023.406622.2673
Abstract
One significant aspect contributing to the overall improvement of railway operations and safety is the detection of obstacles along the tracks. This advancement not only enhances the efficiency of rail transport but also ensures greater accuracy and reliability. Thanks to recent advancements in artificial intelligence technologies, obstacle detection, and self-driving methods have witnessed remarkable progress. In this project, a structure has been designed that enables the real-time detection of obstacles on the train track with a balance between accuracy and volume of calculations. The proposed method involves identifying the train path and checking for the presence of obstacles in this area. Three segmentation models, PAN+SE-ResNet, PAN+EfficientNet, and PAN+NF-Net, were designed, trained, and validated by a set of color images. Results show that the PAN+NF-Net rail segmentation model is more accurate than the other two models with a few tenths of a percent difference. Meanwhile, the computing load of the PAN+EfficientNet model is about half and one-fifth of the computational load of the PAN+NF-Net and PAN+SE-ResNet models, and it offers faster processing speed. To design the object recognition model, the fifth and seventh versions of the YOLO algorithm were trained by a dataset. The results show that the accuracy of the YOLOV7 model in detecting obstacles is 17.6 percent higher than the accuracy of the YOLOV5 model. Finally, obstacle detection Operation was accomplished by checking for overlaps between the mask of the train's path and the bounding boxes predicted by the object detection model.

Keywords

Subjects

  • Brock, A., et al. (2021). High performance large-scale image recognition without normalization. International Conference on Machine Learning. PMLR.

 

  • Belyaev, S., Popov, I., Shubnikov, V., Popov, P., Boltenkova, E., & Savchuk, D. (2020, September). Railroad semantic segmentation on high-resolution images. In 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC) (pp. 1-6). IEEE.

 

 

  • Gebauer, W. Pree, and B. Stadlmann. (2018). Autonomously Driving Trains on Open Tracks—Concepts, System Architecture and Implementation Aspects. IEEE Intelligent Transportation Systems Magazine, 10(1), 58-70.

 

  • Gleichauf, J., Vollet, J., Pfitzner, C., Koch, P., & May, S. (2020). Sensor Fusion Approach for an Autonomous Shunting Locomotive. In O. Gusikhin & K. Madani (Eds.), Informatics in Control, Automation and Robotics (pp. 603-624). Cham, Switzerland: Publishing House.

 

  • Haseeb, M.A.; Guan, J.; Risti´c-Durrant, D.; Gräser, A. A Novel Method for Distance Estimation from Monocular Camera. In Proceedings of the 10th Planning, Perception and Navigation for Intelligent Vehicles (PPNIV18), IROS, Madrid, Spain, 1 October 2018; Volume 10.

 

  • Hu, J., Shen, L., & Sun, G. (2018). Squeeze-and-Excitation Networks. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA (pp. 7132-7141). doi: 10.1109/CVPR.2018.00745.

 

  • Li, J.; Zhou, F.; Ye, T. Real-world railway traffic detection based on faster better network. IEEE Access 2018, 6, 68730–68739.

 

  • Lin, T.-Y.; Maire, M.; Belongie, S.; Hays, J.; Perona, P.; Ramanan, D.; Dollár, P.; Zitnick, C.L. Microsoft COCO: Common Objects
  • in ContextN. In Computer Vision—ECCV 2014, Part V; Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T., Eds.; Springer: Cham,Switzerland, 2020; pp. 740–755. [CrossRef]

 

  • Liu, Y. H. (2018). Feature extraction and image recognition with convolutional neural networks. In Journal of Physics: Conference Series (Vol. 1087, p. 062032). IOP Publishing.

 

  • E. Aidouni, October 2019.[Online]. Available:https://manalelaidouni.github.io/manalelaidouni.github.io/Evaluating-Object-Detection-Models-Guide-to-Performance-Metrics.html.

 

  • Nanni, L., Ghidoni, S., & Brahnam, S. (2017). Handcrafted vs. non-handcrafted features for computer vision classification. Pattern recognition, 71, 158-172.

 

  • Pavel Iakubovskii 2023. [Online]. Available:https://smp.readthedocs.io/en/latest/index.html

 

  • Ristić-Durrant, D., Haseeb, M.A., Franke, M., Banić, M., Simonović, M., & Stamenković, D. (2020). Artificial Intelligence for Obstacle Detection in Railways: Project SMART and Beyond. In S. Bernardi, F. Flammini, S. Marrone, D. Schneider, N. Nostro, A. Di Salle, V. Vittorini, R. Nardone, R. Adler, P. Schleiß, et al. (Eds.), Dependable Computing – EDCC 2020 (pp. 44–55). Springer.

 

  • Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016). You only look once: Unified, real-time object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 779-788).

 

  • Redmon, J., & Farhadi, A. (2019). YOLOv3: An Incremental Improvement. arXiv. Retrieved from https://arxiv.org/pdf/1804.02767.pdf

 

  • Ristić-Durrant, D., Franke, M., & Michels, K. (2021). A review of vision-based on-board obstacle detection and distance estimation in railways. Sensors, 21(10), 3452.

 

  • Tan, M., & Le, Q. (2019). EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. In International Conference on Machine Learning. PMLR.

 

  • Wang, H., Zhang, X., Damiani, L., Giribone, P., Revetria, R., & Ronchetti, G. (2018). Transportation Safety Improvements Through Video Analysis: An Application of Obstacles and Collision Detection Applied to Railways and Roads. In S.I. Ao, H. Kim, O. Castillo, A.S. Chan, & H. Katagiri (Eds.), Transactions on Engineering Technologies (pp. 1–15). Springer.

 

  • Wang, Y., Wang, L., Hu, Y.H., & Qiu, J. (2019). RailNet: A Segmentation Network for Railroad Detection. IEEE Access, 7, 143772–143779.

 

  • Wang, Z., Wu, X., Yu, G., & Li, M. (2018). Efficient Rail Area Detection Using Convolutional Neural Network. IEEE Access, 6, 77656–77664.

 

  • Xu, Y., Gao, C., Yuan, L., Tang, S., & Wei, G. (2019). Real-time Obstacle Detection Over Rails Using Deep Convolutional Neural Network. In Proceedings of the IEEE Intelligent Transportation Systems Conference (ITSC), Auckland, New Zealand, 27–30 October 2019 (pp. 1007–1012).

 

  • Ye, T., Wang, B., Song, P., & Li, J. (2018). Automatic Railway Traffic Object Detection System Using Feature Fusion Refine Neural Network under Shunting Mode. Sensors, 18, 1916.

 

  • Ye, T., Wang, B., Song, P., & Li, J. (2020). Railway Traffic Object Detection Using Differential Feature Fusion Convolution Neural Network. IEEE Transactions on Intelligent Transportation Systems.

 

  • Ye, T., Zhang, Z., Zhang, X., & Zhou, F. (2020). Autonomous railway traffic object detection using feature-enhanced single-shot detector. IEEE Access, 8, 145182-145193.
Quarterly Journal of Transportation Engineering
Volume 16, Issue 2 - Serial Number 63
Winter 2025
Pages 4443-4460

  • Receive Date 12 July 2023
  • Revise Date 12 October 2023
  • Accept Date 16 November 2023