ارائه‌ی الگوریتم هشدار نسبت به موتورسیکلت های پشت سر با استفاده از تک دوربین و روش‌های یادگیری عمیق

نوع مقاله : علمی - پژوهشی

نویسندگان

1 کارشناس ارشد سنجش از دور، دانشکده مهندسی نقشه‌برداری، دانشگاه صنعتی خواجه نصیرالدین طوسی، تهران، ایران

2 استادیار دانشکده مهندسی نقشه‌برداری، دانشگاه صنعتی خواجه نصیرالدین طوسی، تهران، ایران

چکیده

در ایران موتورسیکلت ­ها یکی از آسیب‌پذیرترین کاربران جاده­ ها هستند که حجم زیادی از آمار تصادفات را شامل می­ شوند. در این مقاله راهکار مناسبی برای کمک به کاهش تصادفات اتومبیل ­ها با موتورسیکلت ­ها و به ویژه موتورسیکلت­ های مجهز به بادگیرهای مشکی رنگ ارائه شده است که با استفاده از تنها یک دوربین مستقر بر روی آینه بغل سمت کمک راننده، آگاهی راننده نسبت به نقاط کور محدوده­ ی بغل و پشت سر افزایش می­ یابد، تا در صورت نزدیک شدن بیش از حد موتورسوار، با اعلام هشدار به راننده از بروز تصادف جلوگیری شود. این عملیات هشدار با توجه به تلفیق اطلاعات بدست آمده از دو مرحله تشخیص موتورسیکلت و سپس برآورد فاصله با استفاده از روش­ های یادگیری عمیق صورت گرفته است. در مرحله تشخیص، مدل­ های مختلفی از الگوریتم­ های YOLO با یکدیگر مقایسه شده ­اند که در میان آن ها، مدل بهبود یافته ­ی YOLOV4 با میانگین دقت 80 درصد و سرعت 35 فریم بر ثانیه بهترین عملکرد در شناسایی موتورسیکلت­ های مورد نظر را داشته است. این مدل بر روی پایگاه داده شامل 2000 تصویر اخذ شده از موتورسیکلت ­های شهر تهران آموزش داده شده است. در مرحله ­ی دوم برای برآورد نقشه عمق تک تصویر از آموزش مدل Monodepth2 بر روی 6000 جفت تصویر اخذ شده از خیابان ­های شهر تهران با استفاده از دوربین MYNT-EYE استفاده شده است. با ادغام نتایج بدست آمده از مختصات موتورسیکلت در تصویر و نقشه عمق بدست آمده، الگوریتم پیشنهادی به بهترین عملکرد در تشخیص و براورد فاصله ­ی دوربین تا موتورسیکلت­ موردنظر با میانگین خطای 36 سانتی متر و سرعت 20 فریم بر ثانیه دست یافته است.

کلیدواژه‌ها

عنوان مقاله [English]

A monocular rear-view motorcycle warning algorithm based on deep learning

نویسندگان [English]

  • zahra Badamchi Shabestari 1
  • Ali Hosseini naveh 2
1 M.Sc. Remote sensing engineering, Geodesy and Geomatics Engineering Faculty, K.N. Toosi University of Technology, Tehran, Iran
2 Assistant Professor, Department of Photogrammetry, Geodesy and Geomatics Engineering Faculty, K.N. Toosi University of Technology, Tehran, Iran
چکیده [English]

In Iran, motorcycles are one of the most vulnerable types of vehicles among other road users, with a significant volume of accident statistics in the country. Therefore, in this article, a suitable solution is proposed to reduce car accidents with these motorcycles equipped with black windshields. In this method, a single camera mounted on the side mirror of the driver's assistance is used. This intelligent automatic monitoring can help drivers to pay more attention to the surrounding. This warning algorithm was formed on a combination of motorcycle detection and depth estimation tasks based on deep learning methods. For detecting this category of motorcycles, different models of YOLO algorithms have been compared. According to the speed and accuracy, the fine-tuned YOLOV4 model with an average accuracy of 80% and at a real-time speed of 35 frames per second is used for the detection stage, which has the best performance in comparison to others. This model was fine-tuned based on 2000 images taken from the mentioned motorcycles in Tehran. In the second step, the Monodepth2 model is used to estimate the depth map of a single image. This model was fine-tuned on 6000 stereo images taken from the streets of Tehran by using the MYNT-EYE camera. The contribution of these two algorithms produces a state-of-the-art result for monocular depth estimation, which can estimate the distance of the detected motorcycle with the average error of 36 centimeters at a real-time speed of 20 frames per second.

کلیدواژه‌ها [English]

  • Deep Learning
  • motorcycle detection
  • Depth Image
  • YOLOV4
  • Monodepth2

مراجع

- برومندنیا, ع., و غلامی, م. (2019). "تشخیص کاراکتر فارسی پلاک خودرو، مستقل از چرخش و اندازه با استفاده از ممان‌های متعامد". فصلنامه مهندسی حمل و  نقل، شماره. 4، ص. 961-985.
 
- عصرخودرو. (2019). "آمارتصادفات موتورسیکلتها در سال97" http://www.asrekhodro.com/News/163208/
 
- عصرخودرو. (2020). "استفاده از بادگیر برای موتورسیکلت سوارن".http://www.asrekhodro.com/News/192517
 
- کربلایی, م., آیتی, ا., و صادقی, ع. (2019). "بررسی و مدل سازی تصادفات جرحی و حرکت های نامناسب موتورسیکلت سواران در میدان های شهری". فصلنامه مهندسی حمل و نقل.
 
- Anaya, J. J., Ponz, A., García, F., & Talavera, E. (2017). "Motorcycle detection for ADAS through camera and V2V Communication, a comparative analysis of two modern technologies". Expert Systems with Applications,Vol. 77, pp. 148-159.
- Bhujbal, A., & Mane, D.(2019). "Vehicle Type Classification Using Deep Learning". Paper presented at the International Conference on Soft Computing and Signal Processing, pp. 279-290.
- Bochkovskiy, A., & Liao, H.-Y. M. (2020). "YOLOv4: Optimal Speed and Accuracy of Object Detection". arXiv preprint arXiv:2004.10934.
- Borges, P. V. K., & Vidas, S. (2016). "Practical infrared visual odometry". IEEE Transactions on Intelligent Transportation Systems,Vol. 17, No. 8, pp. 2205-2213.
- Cao, Z., Yang, D., Jiang, K., Xu, S., Wang, S., Zhu, M., & Xiao, Z. (2019). "A geometry-driven car-following distance estimation algorithm robust to road slopes". Transportation research part C: emerging technologies,Vol. 102, pp. 274-288.
- Chabot, F., &  Chaouch, M. (2017). "Deep manta: A coarse-to-fine many-task network for joint 2d and 3d vehicle analysis from monocular image". Paper presented at the Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2040-2049.
- Chen, Y., & Liu, H.-L. (2016). "Overview of landmarks for autonomous, vision-based landing of unmanned helicopters". IEEE Aerospace and Electronic Systems Magazine,Vol. 31, No. 5, pp. 14-27.
- Chen, Z., & Khemmar, R.(2019). "Real Time Object Detection, Tracking, and Distance and Motion Estimation based on Deep Learning: Application to Smart Mobility". Paper presented at the 2019 Eighth International Conference on Emerging Security Technologies (EST), pp. 1-6.
- Chiverton, J. (2012). "Helmet presence classification with motorcycle detection and tracking". IET Intelligent Transport Systems,Vol. 6, No. 3, pp. 259-269.
- Clément Godard, O. M. A., Michael Firman and Gabriel J. Brostow. (2019). "monodepth2".   www.github.com/nianticlabs/monodepth2
- Cordts, M., & Omran, M. (2016). "The cityscapes dataset for semantic urban scene understanding". Paper presented at the Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3213-3223.
- Dabbour, E., & Easa, S. (2014). "Proposed collision warning system for right-turning vehicles at two-way stop-controlled rural intersections". Transportation research part C: emerging technologies,Vol. 42, pp. 121-131.
- Damian, C., Grigorescu, I., & Robu, M.(2019). "Using mono and stereo camera system for static and moving objects detection". Paper presented at the 2019 International Conference on Electromechanical and Energy Systems (SIELMEN), pp. 1-5.
- Deigmoeller, J., Einecke, N., Fuchs, O., & Janssen, H.(2018). "Road Surface Scanning using Stereo Cameras for Motorcycles". Paper presented at the VISIGRAPP (5: VISAPP), pp. 549-554.
- Deng, J., Dong, W., Socher, R.(2009). "Imagenet: A large-scale hierarchical image database". Paper presented at the 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255.
- Dijk, T. v., & Croon, G. d.(2019). "How do neural networks see depth in single images?". Paper presented at the Proceedings of the IEEE International Conference on Computer Vision, pp. 2183-2191.
- Eigen, D., & Puhrsch, C.(2014). "Depth map prediction from a single image using a multi-scale deep network".Paper presented at the Advances in neural information processing systems, pp. 2366-2374.
- Espinosa, J. E., & Velastin, S. A. (2017a). "Motorcycle classification in urban scenarios using convolutional neural networks for feature extraction".
- Espinosa, J. E., Velastin, S. A., & Branch, J. W. (2017b). "Motorcycle classification in urban scenarios using convolutional neural networks for feature extraction". pp. 26-26.
- Espinosa, J. E., Velastin, S. A., & Branch, J. W. (2018a). "Motorcycle detection and classification in urban Scenarios using a model based on Faster R-CNN". pp. 16-16.
- Espinosa, J. E., Velastin, S. A., & Branch, J. W. (2018b). "Motorcycle detection and classification in urban Scenarios using a model based on Faster R-CNN". arXiv preprint arXiv:1808.02299.
- Everingham, M., & Van Gool, L. (2010). "The pascal visual object classes (voc) challenge". International journal of computer vision,Vol. 88, No. 2, pp. 303-338.
- Fernández, C., & Llorca.(2013). "Real-time vision-based blind spot warning system: Experiments with motorcycles in daytime/nighttime conditions". International Journal of Automotive Technology,Vol. 14, No. 1, pp. 113-122.
- Garg, R., Bg, & V. K.(2016). "Unsupervised cnn for single view depth estimation: Geometry to the rescue". Paper presented at the European conference on computer vision, pp. 740-756.
- Geiger, A., Lenz, P., Stiller, & C. (2013). "Vision meets robotics: The kitti dataset". The International Journal of Robotics Research,Vol. 32, No. 11, pp. 1231-1237.
- Girshick, R.(2015). "Fast r-cnn". Paper presented at the Proceedings of the IEEE international conference on computer vision, pp. 1440-1448.
- Girshick, R., Donahue, J., Darrell, T., & Malik, J.(2014). "Rich feature hierarchies for accurate object detection and semantic segmentation". Paper presented at the Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 580-587.
- Glenn Jocher, L. C., & Adam Hogan. (2020). ultralytics/yolov5: Initial Release(Version v1.0): Zenodo. https://doi.org/10.5281/zenodo.3908560
- Godard, C., Mac Aodha, O., & Brostow, G. J.(2017). "Unsupervised monocular depth estimation with left-right consistency". Paper presented at the Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 270-279.
- Godard, C., Mac Aodha, O., Firman, M., & Brostow, G. J.(2019). "Digging into self-supervised monocular depth estimation". Paper presented at the Proceedings of the IEEE international conference on computer vision, pp. 3828-3838.
- GONG, D.-W., Dai, X., Chen, Y., & WANG, S.-F. (2019). "Single-layer Laser Scanner-based Approach for a Transportation Participants Recognition Task". Lasers in Engineering (Old City Publishing),Vol. 43, pp. 10-12.
- Gruyer, D., & Rahal, M.-C.(2019). "Multi-Layer Laser Scanner Strategy for Obstacle Detection and Tracking". Paper presented at the 2019 IEEE/ACS 16th International Conference on Computer Systems and Applications (AICCSA), pp. 1-8.
- He, K., Zhang, X., Ren, S., & Sun, J.(2016). "Deep residual learning for image recognition". Paper presented at the Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778.
- Hirschmuller, H.(2005). "Accurate and efficient stereo processing by semi-global matching and mutual information". Paper presented at the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05),Vol. 2, pp. 807-814.
- Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q.(2017). "Densely connected convolutional networks". Paper presented at the Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4700-4708.
- Jamaluddin, N. L., & Areni, I. S.(2019). "Detection and Distance Estimation against Motorcycles as Navigation Aids for Visually-impaired People". Paper presented at the 2019 12th International Conference on Information & Communication Technology and System (ICTS), pp. 224-228.
- Jiang, H., & Larsson, G.(2018). "Self-supervised relative depth learning for urban scene understanding". Paper presented at the Proceedings of the European Conference on Computer Vision (ECCV), pp. 19-35.
- Kim, M., Liu, Z., & Kang, D. (2016). "On road vehicle detection by learning hard samples and filtering false alarms from shadow features". Journal of Mechanical Science and Technology,Vol. 30, No. 6, pp. 2783-2791.
- Kulkarni, Y., Bodkhe, S., Kamthe, A., & Patil, A.(2018). "Automatic number plate recognition for motorcyclists riding without helmet". Paper presented at the 2018 International Conference on Current Trends towards Converging Technologies (ICCTCT), pp. 1-6.
- Laroca, R., & Severo, E..(2018). "A robust real-time automatic license plate recognition based on the YOLO detector". Paper presented at the 2018 international joint conference on neural networks (ijcnn), pp. 1-10.
- Lenz, I., Lee, H., & Saxena, A. (2015). "Deep learning for detecting robotic grasps". The International Journal of Robotics Research,Vol. 34, No. 4-5, pp. 705-724.
- Li, J., Klein, R., & Yao, A.(2017). "A two-streamed network for estimating fine-scaled depth maps from single rgb images". Paper presented at the Proceedings of the IEEE International Conference on Computer Vision, pp. 3372-3380.
- Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., . . . Zitnick, C. L.(2014). "Microsoft coco: Common objects in context". Paper presented at the European conference on computer vision, pp. 740-755.
- Liu, F., Shen, C., Lin, G., & Reid, I. (2015). "Learning depth from single monocular images using deep convolutional neural fields". IEEE transactions on pattern analysis and machine intelligence,Vol. 38, No. 10, pp. 2024-2039.
- Liu, J., Sun, Q., Fan, Z., & Jia, Y.(2018). "TOF lidar development in autonomous vehicle". Paper presented at the 2018 IEEE 3rd Optoelectronics Global Conference (OGC), pp. 185-190.
- Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., & Berg, A. C.(2016). "Ssd: Single shot multibox detector". Paper presented at the European conference on computer vision, pp. 21-37.
- Luvizon, D. C., Nassu, B. T., & Minetto, R. (2016). "A video-based system for vehicle speed measurement in urban roadways". IEEE Transactions on Intelligent Transportation Systems,Vol. 18, No. 6, pp. 1393-1404.
- Mahto, P., Garg, P., Seth, P., & Panda, J. (2020). "Refining Yolov4 for Vehicle Detection". International Journal of Advanced Research in Engineering and Technology (IJARET),Vol. 11, No. 5, pp. 409-419.
- Markiewicz, P., Długosz, M., & Skruch, P.(2017). "Review of tracking and object detection systems for advanced driver assistance and autonomous driving applications with focus on vulnerable road users sensing". Paper presented at the Polish Control Conference, pp. 224-237.
- Misra, D. (2019). "Mish: A self regularized non-monotonic neural activation function". arXiv preprint arXiv:1908.08681.
- Mistry, J., Misraa, A. K., & Agarwal.(2017a). "An automatic detection of helmeted and non-helmeted motorcyclist with license plate extraction using convolutional neural network". Paper presented at the Image Processing Theory, Tools and Applications (IPTA), 2017 Seventh International Conference on, pp. 1-6.
- Mistry, J., Misraa, & A. K.(2017b). "An automatic detection of helmeted and non-helmeted motorcyclist with license plate extraction using convolutional neural network". Paper presented at the 2017 Seventh International Conference on Image Processing Theory, Tools and Applications (IPTA), pp. 1-6.
- Mun, S., Nguyen, M. D., Kweon, S., & Bae, Y. H. (2019). "Deep Learning Object Detection to Clearly Differentiate Between Pedestrians and Motorcycles in Tunnel Environment Using YOLOv3 and Kernelized Correlation Filters",Vol. 24, No. 7, pp. 1266-1275.
- Nair, V., & Hinton, G. E.(2010). "Rectified linear units improve restricted boltzmann machines". Paper presented at the ICML.
- Ouyang, L., & Wang, H.(2019). "Vehicle target detection in complex scenes based on YOLOv3 algorithm". Paper presented at the IOP Conference Series: Materials Science and Engineering,Vol. 569, No. 5, pp. 052018.
- Pineda-Deom, D. (2019). "Motorcycle blind spot detection system and rear collision alert using mechanically aligned radar." Google Patents.
- Ramachandran, P., Zoph, B., & Le, Q. V. (2017). "Searching for activation functions". arXiv preprint arXiv:1710.05941.
- Redmon, J., Divvala, S., Girshick, R., & Farhadi, A.(2016). "You only look once: Unified, real-time object detection". Paper presented at the Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 779-788.
- Redmon, J., & Farhadi, A.(2017). "YOLO9000: better, faster, stronger". Paper presented at the Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7263-7271.
- Redmon, J., & Farhadi, A. (2018). "Yolov3: An incremental improvement". arXiv preprint arXiv:1804.02767.
- Redmon, J. a. F., Ali. (2018). "yolo". https://pjreddie.com/darknet/yolo/
- Ren, S., He, K., Girshick, R., & Sun, J.(2015). "Faster r-cnn: Towards real-time object detection with region proposal networks". Paper presented at the Advances in neural information processing systems, pp. 91-99.
- Rogers, S., & Papanikolopoulos, N. P.(2000). "Counting bicycles using computer vision". Paper presented at the Intelligent Transportation Systems, 2000. Proceedings. 2000 IEEE, pp. 33-38.
- Salahat, S., Al-Janahi, & A..(2017). "Speed estimation from smart phone in-motion camera for the next generation of self-driven intelligent vehicles". Paper presented at the 2017 IEEE 85th Vehicular Technology Conference (VTC Spring), pp. 1-5.
- Shine, L., & Jiji, C. V. (2020). "Automated detection of helmet on motorcyclists from traffic surveillance videos: a comparative analysis using hand-crafted features and CNN". Multimedia Tools and Applications, pp. 1-21.
- Shotton, J., Fitzgibbon, A., Cook, M., & Sharp, T.(2011). "Real-time human pose recognition in parts from single depth images". Paper presented at the CVPR 2011, pp. 1297-1304.
- Sivaraman, S., & Trivedi, M. M. (2013). "Looking at vehicles on the road: A survey of vision-based vehicle detection, tracking, and behavior analysis". IEEE Transactions on Intelligent Transportation Systems,Vol. 14, No. 4, pp. 1773-1795.
- Sochor, J., Juránek, R., & Špaňhel, J. (2018). "Comprehensive data set for automatic single camera visual speed measurement". IEEE Transactions on Intelligent Transportation Systems,Vol. 20, No. 5, pp. 1633-1643.
- Tang, Z., Wang, G., Xiao, H., Zheng, A., & Hwang, J.-N.(2018). "Single-camera and inter-camera vehicle tracking and 3D speed estimation based on fusion of visual and semantic features". Paper presented at the CVPR Workshop (CVPRW) on the AI City Challenge,
- Thakur, R. (2018). "Infrared Sensors for Autonomous Vehicles". Recent Development in Optoelectronic Devices, pp. 81.
- Van Ratingen, M., & Williams, A. (2016). "The European new car assessment programme: a historical review". Chinese journal of traumatology,Vol. 19, No. 2, pp. 63-69.
- Vishnu, C., Singh, D., & Mohan, C. (2017). "Detection of motorcyclists without helmet in videos using convolutional neural network". Paper presented at the 2017 International Joint Conference on Neural Networks (IJCNN), pp. 3036-3041.
- Wang, S., Liu, F., Gan, Z., & Cui, Z.(2016). "Vehicle type classification via adaptive feature clustering for traffic surveillance video". Paper presented at the 2016 8th International Conference on Wireless Communications & Signal Processing (WCSP), pp. 1-5.
- Wang, W., Yang, S., Li, Y., & Ding, W.(2015). "A rough vehicle distance measurement method using monocular vision and license plate". Paper presented at the 2015 IEEE International Conference on Cyber Technology in Automation, Control, and Intelligent Systems (CYBER), pp. 426-430.
- Wu, W., Kozitsky, V., Hoover, M. E., Loce, R., & Jackson, D. T.(2015). "Vehicle speed estimation using a monocular camera". Paper presented at the Video Surveillance and Transportation Imaging Applications 2015,Vol. 9407, pp. 940704.
- Xie, J., Girshick, R., & Farhadi, A.(2016). "Deep3d: Fully automatic 2d-to-3d video conversion with deep convolutional neural networks". Paper presented at the European Conference on Computer Vision, pp. 842-857.
- Yang, D., Jiang, K., Zhao, D., Yu, C., Cao, Z., Xie, S., Zhang, K. (2018). "Intelligent and connected vehicles: Current status and future perspectives". Science China Technological Sciences,Vol. 61, No. 10, pp. 1446-1471.
- YG, A. R., Kumar, S., Amaresh, H., & Chirag, H.(2015). "Real-time speed estimation of vehicles from uncalibrated view-independent traffic cameras". Paper presented at the TENCON 2015-2015 IEEE Region 10 Conference, pp. 1-6.
- Zhou, Y., & Tuzel, O.(2018). "Voxelnet: End-to-end learning for point cloud based 3d object detection". Paper presented at the Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4490-4499.
فصلنامه مهندسی حمل و نقل
دوره 13، شماره 3 - شماره پیاپی 52
فروردین 1401
صفحه 1710-1683

فایل ها

سابقه مقاله

  • تاریخ دریافت: 30 مهر 1399
  • تاریخ بازنگری: 23 آذر 1399
  • تاریخ پذیرش: 23 آذر 1399
  • تاریخ اولین انتشار: 23 آذر 1399

اشتراک گذاری

ارجاع به این مقاله

آمار