A review and adaptability study of deep learning models for vehicle detection based on high-resolution remote sensing images

LYU Yanan; ZHU Hong; MENG Jian; CUI Chengling; SONG Qiqi

doi:10.6046/zrzyyg.2022010

2022 Vol. 34, No. 4

Article Contents

Article navigation > Remote Sensing for Natural Resources > 2022 > 34(4): 22-32

Next Article Previous Article

LYU Yanan, ZHU Hong, MENG Jian, CUI Chengling, SONG Qiqi. 2022. A review and adaptability study of deep learning models for vehicle detection based on high-resolution remote sensing images. Remote Sensing for Natural Resources, 34(4): 22-32. doi: 10.6046/zrzyyg.2022010

Citation:

LYU Yanan, ZHU Hong, MENG Jian, CUI Chengling, SONG Qiqi. 2022. A review and adaptability study of deep learning models for vehicle detection based on high-resolution remote sensing images. Remote Sensing for Natural Resources, 34(4): 22-32. doi: 10.6046/zrzyyg.2022010

A review and adaptability study of deep learning models for vehicle detection based on high-resolution remote sensing images

1. College of Ecology Environment, Institute of Disaster Prevention, Langfang 065201, China；
2. College of Earth Sciences, Institute of Disaster Prevention, Langfang 065201, China；
3. Beijing Geoway Information-Technology Co., Ltd., Beijing 100043, China

Received Date: 12 January 2022

Revised Date: 15 December 2022

Publish Date: 27 December 2022

Abstract

Abstract

Vehicle detection is a hot research topic in the fields of computer vision, photogrammetry, and remote sensing. With the continuous development of deep learning technology, vehicle detection based on remote sensing images has been applied in fields such as smart city construction and intelligent transportation. This study systematically summarized existent vehicle detection algorithms based on remote sensing images and deep learning models and highlighted the classification, analysis, and comparison of one-stage and two-stage vehicle detection algorithms. Moreover, this study summarized the key technologies of vehicle detection in large-scale and complex backgrounds and analyzed the advantages and disadvantages of mainstream deep learning models of vehicle detection based on remote sensing images. Experiments were conducted to evaluate the YOLOv5, Faster-RCNN, FCOS, and SSD algorithms using DOTA and DIOR datasets. The vehicle detection precision based on the DOTA dataset was 0.695, 0.410, 0.370, and 0.251, respectively and that based on the DIOR dataset was 0.566, 0.243, 0.231, and 0.154, respectively. The experimental results show that the small target scale is still the main factor restricting the vehicle detection performance based on remote sensing images and that the application of deep learning models to the detection of small targets is to be further improved. Finally, based on public datasets and the analysis of existing algorithms, this study proposed the solution and development trend of vehicle detection based on remote sensing images in large-scale and complex backgrounds.
- remote sensing image /
- vehicle detection /
- deep learning /
- analysis method

FullText(HTML)

References

[1]	刘金明. 基于深度卷积神经网络的遥感图像中车辆检测方法研究[D]. 开封: 河南大学, 2020. Google Scholar
[2]	Liu J M. Research on vehicle detection method in remote sensing images based on deep convolution neural network[D]. Kaifeng: Henan University, 2020. Google Scholar
[3]	刘天颖, 李文根, 关佶红. 基于深度学习的光学遥感图像目标检测方法综述[J]. 无线电通信技术, 2020, 46(6):624-634. Google Scholar
[4]	Liu T Y, Li W G, Guan J H. Deep learning based object detection in optical remote sensing images:A survey[J]. Radio Communications Technology, 2020, 46(6):624-634. Google Scholar
[5]	Cheng G, Han J. A survey on object detection in optical remote sensing images[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2016, 117:11-28. Google Scholar
[6]	成喆, 吕京国, 白颖奇, 等. 结合RPN网络与SSD算法的遥感影像目标检测算法[J]. 测绘科学, 2021, 46(4):75-82,99. Google Scholar
[7]	Cheng Z, Lyu J G, Bai Y Q, et al. High-resolution remote sensing image object detection algorithm combining RPN network and SSD algorithm[J]. Science of Surveying and Mapping, 2021, 46(4):75-82,99. Google Scholar
[8]	Alam M, Wang J F, Cong G, et al. Convolutional neural network for the semantic segmentation of remote sensing images[J]. Mobile Networks and Applications, 2021, 26:200-215. Google Scholar
[9]	Ji H, Gao Z, Mei T, et al. Vehicle detection in remote sensing images leveraging on simultaneous super-resolution[J]. IEEE Geoscience and Remote Sensing Letters, 2020, 17(4):676-680. Google Scholar
[10]	张昭, 姚国愉, 李雪纯, 等. 基于改进Faster R-CNN算法的小目标车辆检测[J]. 科技创新与应用, 2021(4):28-32. Google Scholar
[11]	Zhang Z, Yao G Y, Li X C, et al. Small target vehicle detection based on improved Faster-RCNN algorithm[J]. Science and Technology Innovation and Application, 2021(4):28-32. Google Scholar
[12]	Girshick R, Donahue J, Darrell T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE, 2014:580-587. Google Scholar
[13]	南晓虎, 丁雷. 深度学习的典型目标检测算法综述[J]. 计算机应用研究, 2020(s2):15-21. Google Scholar
[14]	Nan X H, Ding L. Overview of typical object detection algorithms based on deep learning[J]. Application Research of Computers, 2020(s2):15-21. Google Scholar
[15]	Girshick R. Fast R-CNN[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE, 2015:1440-1448. Google Scholar
[16]	Ren S, He K, Girshick R, et al. Faster R-CNN:Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6):1137-1149. Google Scholar
[17]	罗峰. 基于超分辨率迁移学习的遥感图像车辆检测[D]. 厦门: 厦门大学, 2017. Google Scholar
[18]	Luo F. Vehicle detection in remote sensing images based on super-resolution transfer learning[D]. Xiamen: Xiamen University, 2017. Google Scholar
[19]	Deng Z P, Hao S, Zhou S L, et al. Toward fast and accurate vehicle detection in aerial images using coupled region-based convolutional neural networks[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2017, 10(8):3652-3664. Google Scholar
[20]	Liu K, Mattyus G. Fast multiclass vehicle detection on aerial images[J]. IEEE Geoscience and Remote Sensing Letters, 2015, 12(9):1938-1942. Google Scholar
[21]	王雪, 隋立春, 李顶萌, 等. 区域卷积神经网络用于遥感影像车辆检测[J]. 公路交通科技, 2018, 35(3):103-108. Google Scholar
[22]	Wang X, Sui L C, Li D M, et al. Regional convolutional neural network for vehicle detection in remote sensing images[J]. Journal of Electronics and Information Technology, 2018, 35(3):103-108. Google Scholar
[23]	高鑫, 李慧, 张义, 等. 基于可变形卷积神经网络的遥感影像密集区域车辆检测方法[J]. 电子与信息学报, 2018, 40(12):2812-2819. Google Scholar
[24]	Gao X, Li H, Zhang Y, et al. Vehicle detection in remote sensing images of dense areas based on deformable convolution neural network[J]. Journal of Electronics and Information Technology, 2018, 40(12):2812-2819. Google Scholar
[25]	阳理理. 基于人工神经网络的遥感图像车辆检测[D]. 南宁: 广西大学, 2018. Google Scholar
[26]	Yang L L. Vehicle detection based on artificial neural network in remote sensing images[D]. Nanning: Guangxi University, 2018. Google Scholar
[27]	孙秉义. 基于遥感图像处理的交通量检测与分析[D]. 上海: 上海交通大学, 2019. Google Scholar
[28]	Sun B Y. Traffic volume detection and analysis based on remote sensing images processing[D]. Shanghai: Shanghai Jiaotong University, 2019. Google Scholar
[29]	Xia G S, Bai X, Ding J, et al. DOTA:A large-scale dataset for object detection in aerial images[C]// 2018 IEEE Conference on Computer Vision and Pattern Recognition.IEEE, 2018:3974-3983. Google Scholar
[30]	黄国捷. 基于深度学习的遥感图像车辆目标检测[D]. 苏州: 苏州大学, 2019. Google Scholar
[31]	Huang G J. Vehicle target detection from remote sensing images based on deep learning[D]. Suzhou: Soochow University, 2019. Google Scholar
[32]	Ji H, Gao Z, Mei T, et al. Improved Faster R-CNN with multiscale feature fusion and homography augmentation for vehicle detection in remote sensing images[J]. IEEE Geoscience and Remote Sensing Letters, 2019, 16(11):1761-1765. Google Scholar
[33]	Rottensteiner F, Sohn G, Jung J, et al. The ISPRS benchmark on urban object classification and 3D building reconstruction[J]. ISPRS Annals of Photogrammetry,Remote Sensing and Spatial Information Sciences, 2012,1-3:293-298. Google Scholar
[34]	Razakarivony S, Jurie F. Vehicle detection in aerial imagery:A small target detection benchmark[J]. Journal of Visual Communication and Image Representation, 2016, 34:187-203. Google Scholar
[35]	Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 39(4):640-651. Google Scholar
[36]	梁哲恒, 黎宵, 邓鹏, 等. 融和多尺度特征注意力的融合遥感影像变化检测方法[J]. 测绘学报, 2022, 51(5):668-676. Google Scholar
[37]	Liang Z H, Li X, Deng P, et al. Remote sensing images change detection fusion method integrating multi-scale feature attention[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(5):668-676. Google Scholar
[38]	Yang X, Sun H, Sun X, et al. Position detection and direction prediction for arbitrary-oriented ships via multitask rotation region convolutional neural network[J]. IEEE Access, 2018, 6:50839-50849. Google Scholar
[39]	Fu Y, Wu F, Zhao J. Context-Aware and depth wise-based detection on orbit for remote sensing image[C]// 2018 24th International Conference on Pattern Recognition(ICPR).IEEE, 2018:1725-1730. Google Scholar
[40]	Li Q, Mou L, Xu Q, et al. R³-Net:A deep network for multioriented vehicle detection in aerial images and videos[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(7):5028-5042. Google Scholar
[41]	Zhang Z H, Guo W W, Zhu S N, et al. Toward arbitrary-oriented ship detection with rotated region proposal and discrimination networks[J]. IEEE Geoscience and Remote Sensing Letters, 2018, 15(11):1745-1749. Google Scholar
[42]	林钊. 基于深度学习的遥感图像舰船目标检测与识别[D]. 长沙: 国防科技大学, 2018. Google Scholar
[43]	Lin Z. Ship detection and recognition in remote sensing images based on deep learning[D]. Changsha: National University of Defense Technology, 2018. Google Scholar
[44]	刘万军, 高健康, 曲海成, 等. 多尺度特征增强的遥感图像舰船目标检测[J]. 自然资源遥感, 2021, 33(3):97-106.doi:10.6046/zrzyyg.2020372. Google Scholar
[45]	Liu W J, Gao J K, Qu H C, et al. Ship detection based on multi-scale feature enhancement of remote sensing images[J]. Remote Sensing for Natural Resources, 2021, 33(3):97-106.doi:10.6046/zrzyyg.2020372. Google Scholar
[46]	许德刚, 王露, 李凡. 深度学习的典型目标检测算法研究综述[J]. 计算机工程与应用, 2021, 57(8):10-25. Google Scholar
[47]	Xu D G, Wang L, Li F. A review of typical object detection algorithms based on deep learning[J]. Computer Engineering and Applications, 2021, 57(8):10-25. Google Scholar
[48]	Redmon J, Divvala S, Girshick R, et al. You only look once:Unified,real-time object detection[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE, 2016:779-788. Google Scholar
[49]	Redmon J, Farhadi A. YOLO9000:Better,faster,stronger[C]// IEEE Conference on Computer Vision and Pattern Recognition.IEEE, 2017:6517-6525. Google Scholar
[50]	Redmon J, Farhadi A. YOLOv3:An incremental improvement[EB/OL].(2019-12-25)[2021-12-29]. https://arxiv.org/abs/1804.02767 . Google Scholar
[51]	Bochkovskiy A, Wang C Y, Liao H. YOLOv4:Optimal speed and accuracy of object detection[EB/OL].(2020-04-23)[2021-12/29]. https://arxiv.org/abs/2004.10934 . Google Scholar
[52]	Liu W, Anguelov D, Erhan D, et al. SSD:Single shot multibox detector[J]. Springer,Cham, 2016, 9905:21-37. Google Scholar
[53]	Fu C, Liu W, Ranga A, et al. DSSD:Deconvolutional single shot detector[EB/OL].(2017-01-23)[2021-12/29]. https://arxiv.org/abs/1701.06659v1 . Google Scholar
[54]	Tian Z, Shen C H, Chen H, et al. FCOS:Fully convolutional one-stage object detection[C]// 2019 IEEE/CVF International Conference on Computer Vision(ICCV).IEEE, 2020:9626-9635. Google Scholar
[55]	李圣琀, 邵峰晶. 基于深度学习的轻量遥感图像车辆检测模型[J]. 工业控制计算机, 2020, 33(6):66-69. Google Scholar
[56]	Li S H, Shao F J. Vehicle detection model of light weight remote sensing image based on deep learning[J]. Industrial Control Computer, 2020, 33(6):66-69. Google Scholar
[57]	Etten A V. You only look twice:Rapid multi-scale object detection in satellite imagery[EB/OL].(2018-05-24)[2021-12-29]. https://arxiv.org/abs/1805.09512 . Google Scholar
[58]	彭新月, 张吴明, 钟若飞. 改进YOLOv3模型的GF-2卫星影像车辆检测[J]. 测绘科学, 2021, 46(12):147-154. Google Scholar
[59]	Peng X Y, Zhang W M, Zhong R F. GF-2 satellite image vehicle detection based on improved YOLOv3 model[J]. Science of Surveying and Mapping, 2021, 46(12):147-154. Google Scholar
[60]	汤田玉. 基于深度学习的高分辨率光学遥感影像车辆目标检测方法研究[D]. 长沙: 国防科技大学, 2017. Google Scholar
[61]	Tang T Y. Deep convolutional neural network based vehicle detection methods on high resolution optical remote sensing images[D]. Changsha: National University of Defense Technology, 2017. Google Scholar
[62]	侯涛, 蒋瑜. 改进YOLOv4在遥感飞机目标检测中的应用研究[J]. 计算机工程与应用, 2021, 57(12):224-230. Google Scholar
[63]	Hou T, Jiang Y. Resrarch of improved YOLOv4 in remote sensing aircraft target detection[J]. Computer Engineering and Applications, 2021, 57(12):224-230. Google Scholar
[64]	赵鹏飞, 谢林柏, 彭力. 融合注意力机制的深层次小目标检测算法[J]. 计算机科学与探索, 2022, 16(4):927-937. Google Scholar
[65]	Zhao P F, Xie L B, Peng L. A deep small target detection algorithm based on attention mechanism[J]. Computer Science and Technolo-gy, 2022, 16(4):927-937. Google Scholar
[66]	王明阳, 王江涛, 刘琛. 基于关键点的遥感图像旋转目标检测[J]. 电子测量与仪器学报, 2021, 35(6):102-108. Google Scholar
[67]	Wang M Y, Wang J T, Liu C. Detection of rotating targets in remote sensing images based on key points[J]. Journal of Electronic Measurement and Instrument, 2021, 35(6):102-108. Google Scholar
[68]	唐建宇, 唐春晖. 基于旋转框和注意力机制的遥感图像目标检测算法[J]. 电子测量技术, 2021, 44(13):114-120. Google Scholar
[69]	Tang J Y, Tang C H. Remote sensing image target detection algorithm based on rotating frame and attention mechanism[J]. Electronic Measurement Technique, 2021, 44(13):114-120. Google Scholar
[70]	陈俊. 基于R-YOLO的多源遥感图像海面目标融合检测算法研究[D]. 武汉: 华中科技大学, 2019. Google Scholar
[71]	Chen J. Research on maritime target fusion detection in multi-source remote sensing images based on R-YOLO[D]. Wuhan: Huazhong University of Science and Technology, 2019. Google Scholar
[72]	谢俊章, 彭辉, 唐健峰, 等. 改进YOLOv4的密集遥感目标检测[J]. 计算机工程与应用, 2021, 57(22):247-256. Google Scholar
[73]	Xie J Z, Peng H, Tang J F, et al. Improved YOLOv4 for dense remote sensing target detection[J]. Computer Engineering and Applications, 2021, 57(22):247-256. Google Scholar
[74]	杨治佩, 丁胜, 张莉, 等. 无锚点的遥感图像任意角度密集目标检测方法[J]. 计算机应用, 2022, 42(6):1965-1971. Google Scholar
[75]	Yang Z P, Ding S, Zhang L, et al. An arbitrary angle dense target detection method for remote sensing images without anchor points[J]. Computer Application, 2022, 42(6):1965-1971. Google Scholar
[76]	张宏群, 班勇苗, 郭玲玲, 等. 基于YOLOv5的遥感图像舰船的检测方法[J]. 电子测量技术, 2021, 44(8):87-92. Google Scholar
[77]	Zhang H Q, Ban Y M, Guo L L, et al. Remote sensing images ship detection method based on YOLOv5[J]. Electronic Measurement Technology, 2021, 44(8):87-92. Google Scholar
[78]	张玉莲. 光学图像海面舰船目标智能检测与识别方法研究[D]. 长春: 中国科学院大学(中国科学院长春光学精密机械与物理研究所), 2021. Google Scholar
[79]	Zhang Y L. Research on intelligent detection and recognition metho-ds of ship targets on the sea surface in optical images[D]. Changchun: University of Chinese Academy of Sciences (Changchun Institute of Optics,Fine Mechanics and Physics,CAS), 2021. Google Scholar
[80]	Li K, Wan G, Cheng G, et al. Object detection in optical remote sensing images:A survey and a new benchmark[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 159:296-307. Google Scholar