中国机械工程 ›› 2021, Vol. 32 ›› Issue (09): 1114-1125.DOI: 10.3969/j.issn.1004-132X.2021.09.013

• 工程前沿 • 上一篇    下一篇

基于多模态特征融合的自主驾驶车辆低辨识目标检测方法#br#

邹伟;殷国栋;刘昊吉;耿可可;黄文涵;吴愿;薛宏伟   

  1. 东南大学机械工程学院,南京,211189
  • 出版日期:2021-05-10 发布日期:2021-05-28
  • 通讯作者: 殷国栋(通信作者),男,1976年生,教授、博士研究生导师。研究方向为智能网联汽车、无人驾驶与智能辅助驾驶系统、车路协同、新能源汽车控制系统、车辆动力学及其控制等。E-mail:ygd@seu.edu.cn。
  • 作者简介:邹伟,男,1994年生,硕士研究生。研究方向为计算机视觉、深度学习、视觉感知等。
  • 基金资助:
    国家自然科学基金(51975118);
    江苏省重点研发计划(BE2019004-2);
    江苏省成果转化项目(BA2018023)

Low-observable Target Detection Method for Autonomous Vehicles Based on Multi-modal Feature Fusion#br#

ZOU Wei;YIN Guodong;LIU Haoji;GENG Keke;HUANG Wenhan;WU Yuan;XUE Hongwei#br#   

  1. School of Mechanical Engineering,Southeast University,Nanjing,211189
  • Online:2021-05-10 Published:2021-05-28

摘要: 针对自主驾驶车辆在真实驾驶环境下对低辨识目标的识别问题,提出了基于多模态特征融合的目标检测方法。基于Faster R-CNN算法设计多模态深度卷积神经网络,融合彩色图像、偏振图像、红外图像特征,提高对低辨识目标的检测性能;开发多模态(3种)图像低辨识度目标实时检测系统,探索多模态图像特征融合在自动驾驶智能感知系统中的应用。建立了人工标注过的多模态(3种)图像低辨识目标数据集,对深度学习神经网络进行训练,优化内部参数,使得该系统适用于复杂环境下对行人、车辆目标的检测和识别。实验结果表明,相对于传统的单模态目标检测算法,基于多模态特征融合的深度卷积神经网络对复杂环境下的低辨识目标具有更好的检测和识别性能。

关键词: 自主驾驶, 多模态特征融合, 深度卷积神经网络, 低辨识目标, 智能感知

Abstract: Aiming at the problems of low-observable target detection in autonomous vehicles under real driving conditions, a target detection method was proposed based on multi-modal feature fusion. In order to improve the detection on the low-observable targets, the multi-modal deep convolutional neural network was designed based on Faster R-CNN to fuse the features of RGB images, polarized images and infrared images, the development of the multi-modal (three) image low-observable target real-time detection system was studied and the applications of multi-modal image feature fusion in the intelligent perception system of autonomous vehicles were explored herein. A manually labeled multi-modal image dataset of low-observable targets was built. The deep learning neural network was trained to optimize internal parameters to make the system capable for both pedestrians and vehicle recognition in the complex environments. The experimental results indicate that the deep convolutional neural network based on the multi-modal fusion has a better performance on the low-observable target detection and recognition in complex environments than that of traditional single-modal methods.

Key words: autonomous driving, multi-modal feature fusion, deep convolutional neural network, low-observable target, intelligent perception

中图分类号: