A two-stage remote sensing image inpainting network combined with spatial semantic attention

LIU Yujia; XIE Shizhe; DU Yang; YAN Jin; NAN Yanyun; WEN Zhongkai

doi:10.6046/gtzyyg.2022362

2024 Vol. 36, No. 1

Article Contents

Article navigation > Remote Sensing for Natural Resources > 2024 > 36(1): 58-66

Next Article Previous Article

LIU Yujia, XIE Shizhe, DU Yang, YAN Jin, NAN Yanyun, WEN Zhongkai. 2024. A two-stage remote sensing image inpainting network combined with spatial semantic attention. Remote Sensing for Natural Resources, 36(1): 58-66. doi: 10.6046/gtzyyg.2022362

Citation:

LIU Yujia, XIE Shizhe, DU Yang, YAN Jin, NAN Yanyun, WEN Zhongkai. 2024. A two-stage remote sensing image inpainting network combined with spatial semantic attention. Remote Sensing for Natural Resources, 36(1): 58-66. doi: 10.6046/gtzyyg.2022362

A two-stage remote sensing image inpainting network combined with spatial semantic attention

1. School of Geomatics, Liaoning Technical University, Fuxin 123000, China
;2. School of Information Engineering, China University of Geosciemces(Beijing), Beijing 100083, China
;3. College of Astronautics, Nanjing University of Aeronautics and Astronautics, Nanjing 210016, China
;4. National Earthquake Response Support Service, Beijing 100049, China
;5. College of Resource Environment and Toursim, Capital Normal University, Beijing 100048, China
;6. Institute of Remote Sensing Satellite, CAST,Beijing 100094, China

Received Date: 13 September 2022

Revised Date: 16 November 2022

Publish Date: 13 March 2024

Abstract

Abstract

In high-resolution remote sensing images, missing areas feature intricate surface features and pronounced spatial heterogeneity, causing the image inpainting results to suffer texture blurring and structural distortion, particularly for boundaries and areas with complex textures. This study proposed a two-stage remote sensing image inpainting network combined with spatial semantic attention (SSA). The network comprised two networks in series: one for coarse image inpainting and one for fine-scale image inpainting (also referred to as the coarse and fine-scale networks, respectively). This network was designed to guide the fine-scale network to restore the missing areas using the priori information provided by the coarse network. In the coarse network, a multi-level loss structure was constructed to enhance the stability of network training. In the fine-scale network, a novel SSA mechanism was proposed, with SSA being embedded differentially in the encoder and decoder based on the distribution of network features. This ensured the continuity of local features and the correlation of global semantic information. The experimental results show that the network proposed in this study can further improve the image inpainting effects compared to other existing algorithms.
- two-stage network /
- remote sensing image inpainting /
- spatial semantic attention /
- continuity of local features /
- correlation of global semantic information