Longitudinal and Lateral Acceleration Control of Unmanned Vehicle Using Deep Reinforcement Learning

Document Type : Scientific - Research

Authors
Mohsen Ebrahimi 1
FIROUZ ALLAHVERDIZADEH 2
Abdolreza Kashani Nia 2
1 Ph.D., Faculty of Electrical and Computer Engineering, Malek Ashtar University of Technology, Tehran, Iran
2 Assistant Professor, Faculty of Electrical and Computer Engineering, Malek Ashtar University of Technology, Tehran, Iran
10.22119/jte.2025.505444.2730
Abstract
Accurate control of longitudinal and lateral acceleration in unmanned vehicles is one of the key challenges in the development of safe and efficient autonomous systems. Given the rapid growth of autonomous technologies and the need to improve the performance of these systems in various road conditions, research in the field of intelligent control of these vehicles has gained special importance. In this paper, an innovative method for controlling the longitudinal and lateral acceleration of unmanned vehicles based on reinforcement learning is presented. The proposed method, by utilizing a disaggregated architecture, reduces the computational complexity and enables the use of more advanced reinforcement learning algorithms. In the initial stage, a single reinforcement learning agent is designed and trained to simultaneously control longitudinal and lateral acceleration. Then, in order to improve efficiency, the reinforcement learning agents are separated into two independent parts for longitudinal and lateral control, and trained separately. Simulation results show that this separation not only increases the convergence speed of the training process, but also significantly improves the accuracy of the control system's performance. Separating the agents results in a reduction of about 2 hours of deep network training time, a 38.2% reduction in average direction error and a 10.1% reduction in average distance error. These findings could help develop more advanced and safer control systems for autonomous vehicles and play an important role in improving the safety and efficiency of these technologies.
Keywords
Unmanned vehicle
reinforcement learning
vehicle lateral acceleration
vehicle longitudinal acceleration
Subjects
- Artuñedo, A., Moreno-Gonzalez, M., & Villagra, J. (2024). “Lateral control for autonomous vehicles: A comparative evaluation”. Annual reviews in control, 57, 100910.
 
- Baheri, A., Kolmanovsky, I., Girard, A., Tseng, H. E., & Filev, D. (2020, July) “Vision-based autonomous driving: A model learning approach”. In 2020 American Control Conference (ACC) (pp. 2520-2525). IEEE.
 
- Bay, O. (2021) “Abi research forecasts 8 million vehicles to ship with sae level 3, 4 and 5 autonomous technology in 2025”.
 
- Bellotti, F., Lazzaroni, L., Capello, A., Cossu, M., De Gloria, A., & Berta, R. (2023) “Explaining a deep reinforcement learning (DRL)-based automated driving agent in highway simulations”. IEEE Access, 11, 28522-28550.
 
- Cao, Y., Ni, K., Jiang, X., Kuroiwa, T., Zhang, H., Kawaguchi, T., & Jiang, W. (2023). Path following for autonomous ground vehicle using DDPG algorithm: a reinforcement learning approach. Applied Sciences, 13(11), 6847.
 
- Chen, J., Zhang, C., Luo, J., Xie, J., & Wan, Y. (2020). “Driving maneuvers prediction based autonomous driving control by deep Monte Carlo tree search”. IEEE transactions on vehicular technology, 69(7), 7146-7158.
 
- Claussmann, L., Revilloud, M., Gruyer, D., & Glaser, S. (2019) “A review of motion planning for highway autonomous driving”. IEEE Transactions on Intelligent Transportation Systems, 21(5), 1826-1848.
 
- Du, Y., Chen, J., Zhao, C., Liao, F., & Zhu, M. (2023). “A hierarchical framework for improving ride comfort of autonomous vehicles via deep reinforcement learning with external knowledge”. Computer‐Aided Civil and Infrastructure Engineering, 38(8), 1059-1078.
 
- Ebrahimi, M. (2023). “Broken rail detection with texture image processing using two-dimensional gray level co-occurrence matrix”. arXiv preprint arXiv:2304.11592.
 
- Ebrahimi, M., & Asgari, M. (2021). “Robust fractional-order fixed-structure controller design for uncertain non-commensurate fractional plants using fractional Kharitonov theorem”. Proceedings of the Institution of Mechanical Engineers, Part I: Journal of Systems and Control Engineering, 235(8), 1375-1387.
- Ebrahimi, M., & Nasrollahi, S. (2025). “Fractional Guidance Law with Impact Angle Constraint and Seeker’s Look Angle Limits”. Unmanned Systems, 13(01), 261-277.
- El Hamdani, S., Benamar, N., & Younis, M. (2020) “Pedestrian support in intelligent transportation systems: challenges, solutions and open issues”. Transportation research part C: emerging technologies, 121, 102856.
 
- Feng, D., Haase-Schütz, C., Rosenbaum, L., Hertlein, H., Glaeser, C., Timm, F., ... & Dietmayer, K. (2020) “Deep multi-modal object detection and semantic segmentation for autonomous driving: Datasets, methods, and challenges”. IEEE Transactions on Intelligent Transportation Systems, 22(3), 1341-1360.
 
- Grigorescu, S., Trasnea, B., Cocias, T., & Macesanu, G. (2020) “A survey of deep learning techniques for autonomous driving”. Journal of field robotics, 37(3), 362-386.
 
- Guo, J., Cheng, S., & Liu, Y. (2020) “Merging and diverging impact on mixed traffic of regular and autonomous vehicles”. IEEE Transactions on Intelligent Transportation Systems, 22(3), 1639-1649.
 
- Hoveidafard, A., Fardmoradinia, S., Golchin, B. and Ghaffari, A. (2023) "Simulation of autonomous vehicles using machine learning methods". International Journal of Transportation Engineering, 15(2), 3483-3507 (in  persian).
 
- Huang, Z., Zhang, J., Tian, R., & Zhang, Y. (2019) “End-to-end autonomous driving decision based on deep reinforcement learning”. In 2019 5th International Conference on Control, Automation and Robotics (ICCAR) (pp. 658-662). IEEE.
 
  -Kuutti, S., Bowden, R., Jin, Y., Barber, P., & Fallah, S. (2020) “A survey of deep learning applications to autonomous vehicle control”. IEEE Transactions on Intelligent Transportation Systems, 22(2), 712-733.
 
- Lamssaggad, A., Benamar, N., Hafid, A. S., & Msahli, M. (2021) “A survey on the current security landscape of intelligent transportation systems”. IEEE Access, 9, 9180-9208.
 
-  Lee, K., Isele, D., Theodorou, E. A., & Bae, S. (2022) “Spatiotemporal costmap inference for MPC via deep inverse reinforcement learning”. IEEE Robotics and Automation Letters, 7(2), 3194-3201.
 
- Li, Z., Yuan, S., Yin, X., Li, X., & Tang, S. (2023) “Research into autonomous vehicles following and obstacle avoidance based on deep reinforcement learning method under map constraints”. Sensors, 23(2), 844.
 
-Liu, Y., & Diao, S. (2024) “An automatic driving trajectory planning approach in complex traffic scenarios based on integrated driver style inference and deep reinforcement learning”. PLoS one, 19(1), e0297192.
 
 - Ma, Y., Wang, Z., Yang, H., & Yang, L. (2020) “Artificial intelligence applications in the development of autonomous vehicles: A survey”. IEEE/CAA Journal of Automatica Sinica, 7(2), 315-329.
 
- Mahaadevan, V. C., Narayanamoorthi, R., Panda, S., Dutta, S., & Dooly, G. (2024) “AViTRoN: Advanced vision track routing and navigation for autonomous charging of electric vehicles”. IEEE Access.
 
- Mao, Z., Liu, Y., & Qu, X. (2024) “Integrating big data analytics in autonomous driving: An unsupervised hierarchical reinforcement learning approach”. Transportation Research Part C: Emerging Technologies, 162, 104606.
 
- Ning, H., Yin, R., Ullah, A., & Shi, F. (2021) “A survey on hybrid human-artificial intelligence for autonomous driving”. IEEE Transactions on Intelligent Transportation Systems, 23(7), 6011-6026.
 
- Peličić, D., Ristić, B., & Radević, S. (2024) “Factors for the occurrence of road traffic injuries and better care of injured persons”. Zdravstvena zaštita, 53, 59.
 
- Pérez-Gil, Ó., Barea, R., López-Guillén, E., Bergasa, L. M., Gómez-Huélamo, C., Gutiérrez, R., & Díaz-Díaz, A. (2022) “Deep reinforcement learning based control for Autonomous Vehicles in CARLA”. Multimedia Tools and Applications, 81(3), 3553-3576.
 
- Rajamani, R. (2006). Vehicle dynamics and control. Boston, MA: Springer US.
 
- Singh, S. (2015) “Critical reasons for crashes investigated in the national motor vehicle crash causation survey” (No. DOT HS 812 115).
 
- Toromanoff, M., Wirbel, E., & Moutarde, F. (2020) “End-to-end model-free reinforcement learning for urban driving using implicit affordances”. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 7153-7162).
 
 
- Van Hasselt, H., Guez, A., & Silver, D. (2016) “Deep reinforcement learning with double q-learning”. In Proceedings of the AAAI conference on artificial intelligence (Vol. 30, No. 1).
 
 - Wu, Y., Tan, H., Qin, L., & Ran, B. (2020) “Differential variable speed limits control for freeway recurrent bottlenecks via deep actor-critic algorithm”. Transportation research part C: emerging technologies, 117, 102649.
 
- Wang, C., Cui, X., Zhao, S., Zhou, X., Song, Y., Wang, Y., & Guo, K. (2024) “A deep reinforcement learning-based active suspension control algorithm considering deterministic experience tracing for autonomous vehicle”. Applied Soft Computing, 153, 111259.
 
- Ye, F., Cheng, X., Wang, P., Chan, C. Y., & Zhang, J. (2020) “Automated lane change strategy using proximal policy optimization-based deep reinforcement learning”. In 2020 IEEE Intelligent Vehicles Symposium (IV) (pp. 1746-1752). IEEE.
 
- Yuan, M., Shan, J., & Mi, K. (2023) “From Naturalistic Traffic Data to Learning-Based Driving Policy: A Sim-to-Real Study”. IEEE Transactions on Vehicular Technology.
 
- Yusuf, S. A., Khan, A., & Souissi, R. (2024) “Vehicle-to-everything (V2X) in the autonomous vehicles domain–A technical review of communication, sensor, and AI technologies for road user safety”. Transportation Research Interdisciplinary Perspectives, 23, 100980.
 
- Zhang, Y., Sun, P., Yin, Y., Lin, L., & Wang, X. (2018) “Human-like autonomous vehicle speed control by deep reinforcement learning with double Q-learning”. In 2018 IEEE intelligent vehicles symposium (IV) (pp. 1251-1256). IEEE.
 
- Zhu, M., Wang, Y., Pu, Z., Hu, J., Wang, X., & Ke, R. (2020) “Safe, efficient, and comfortable velocity control based on reinforcement learning for autonomous driving”. Transportation Research Part C: Emerging Technologies, 117, 102662.
Quarterly Journal of Transportation Engineering
Volume 17, Issue 2 - Serial Number 67
Winter 2026
Pages 5333-5358

  • Receive Date 09 February 2025
  • Revise Date 02 August 2025
  • Accept Date 07 September 2025