Classification of hyperspectral images based on deep Transformer network combined with spatial-spectral information

ZHANG Pengqiang; GAO Kuiliang; LIU Bing; TAN Xiong

doi:10.6046/zrzyyg.2021271

2022 Vol. 34, No. 3

Article Contents

Article navigation > Remote Sensing for Natural Resources > 2022 > 34(3): 27-32

Next Article Previous Article

ZHANG Pengqiang, GAO Kuiliang, LIU Bing, TAN Xiong. 2022. Classification of hyperspectral images based on deep Transformer network combined with spatial-spectral information. Remote Sensing for Natural Resources, 34(3): 27-32. doi: 10.6046/zrzyyg.2021271

Citation:

ZHANG Pengqiang, GAO Kuiliang, LIU Bing, TAN Xiong. 2022. Classification of hyperspectral images based on deep Transformer network combined with spatial-spectral information. Remote Sensing for Natural Resources, 34(3): 27-32. doi: 10.6046/zrzyyg.2021271

Classification of hyperspectral images based on deep Transformer network combined with spatial-spectral information

Information Engineering University, Zhengzhou 450001, China

Received Date: 30 August 2021

Revised Date: 15 September 2022

Publish Date: 21 September 2022

Abstract

Abstract

The local convolution operation in convolutional neural networks cannot fully learn the global semantic information in hyperspectral images. Given this, this study designed a novel deep network model based on Transformer in order to further improve the classification precision of hyperspectral images. Firstly, this study reduced the dimensionality of hyperspectral images using the principal component analysis method and selected the neighborhood data around pixels as input samples to fully utilize the spatial-spectral information in the images. Secondly, the convolutional layer was used to transform the input samples into sequential characteristic vectors. Finally, image classification was conducted using the designed deep Transformer network. The multi-head attention mechanism in the Transformer model can make full use of the rich discriminative information. Experimental results show that the method proposed in this study can achieve better classification performance than the existing convolutional neural network model.
- hyperspectral image classification /
- Transformer /
- deep learning /
- self-attention mechanism

FullText(HTML)

References

[1]	He L, Li J, Liu C, et al. Recent advances on spectral-spatial hyperspectral image classification:An overview and new guidelines[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(3):1579-1597. Google Scholar
[2]	Ghamisi P, Plaza J, Chen Y, et al. Advanced spectral classifiers for hyperspectral images:A review[J]. IEEE Geoscience and Remote Sensing Magazine, 2017, 5(1):8-32. Google Scholar
[3]	Tao C, Pan H, Li Y, et al. Unsupervised spectral-spatial feature learning with stacked sparse autoencoder for hyperspectral imagery classification[J]. IEEE Geoscience and Remote Sensing Letters, 2015, 12(12):2438-2442. Google Scholar
[4]	Li T, Zhang J, Zhang Y. Classification of hyperspectral image based on deep belief networks[C]// Proceedings of the 2014 IEEE International Conference on Image Processing (ICIP), 2014. Google Scholar
[5]	Zhang X R, Sun Y J, Jiang K, et al. Spatial sequential recurrent neural network for hyperspectral image classification[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2018, 11(11):4141-4155. Google Scholar
[6]	Xu Q, Xiao Y, Wang D, et al. CSA-MSO3DCNN:Multiscale octave 3D CNN with channel and spatial attention for hyperspectral image classification[J]. Remote Sensing, 2020, 12(1):188. Google Scholar
[7]	Gao K, Guo W, Yu X, et al. Deep induction network for small samples classification of hyperspectral images[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2020, 13: 3462-3477. Google Scholar
[8]	高奎亮, 张鹏强, 余旭初, 等. 基于Network In Network网络结构的高光谱影像分类方法[J]. 测绘科学技术学报, 2019, 36(5):500-504,510. Google Scholar
[9]	Gao K L, Zhang P Q, Yu X C, et al. Classification method of hyperspectral image based on Network In Network structure[J]. Journal of Geomatics Science and Technology, 2019, 36(5):500-504,510. Google Scholar
[10]	Li Y, Zhang H, Shen Q. Spectral-spatial classification of hyperspectral imagery with 3D convolutional neural network[J]. Remote Sensing, 2017, 9(1):67. Google Scholar
[11]	Xu X, Li J, Li S. Multiview intensity-based active learning for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(2):669-680. Google Scholar
[12]	He X, Chen Y. Transferring CNN ensemble for hyperspectral image classification[J]. IEEE Geoscience and Remote Sensing Letters, 2021, 18(5):876-880. Google Scholar
[13]	Mou L, Ghamisi P, Zhu X X. Unsupervised spectral-spatial feature learning via deep residual Conv-Deconv network for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(1):391-406. Google Scholar
[14]	Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[C]// Thirty-first Conference on Neural Information Processing Systems, 2017. Google Scholar
[15]	Dosovitskiy A, Beyer L, Kolesnikov A, et al. An image is worth 16×16[Z]. Transformers for Image Recognition at Scale, 2020. Google Scholar
[16]	Yue J, Zhao W, Mao S, et al. Spectral-spatial classification of hyperspectral images using deep convolutional neural networks[J]. Remote Sensing Letters, 2015, 6(4-6):468-477. Google Scholar
[17]	刘冰, 余旭初, 张鹏强, 等. 联合空-谱信息的高光谱影像深度三维卷积网络分类[J]. 测绘学报, 2019, 48(1):53-63. Google Scholar
[18]	Liu B, Yu X C, Zhang P Q, et al. Deep 3D convolutional network combined with spatial-spectral features for hyperspectral image classification[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(1):53-63. Google Scholar