Swin Transformer based Siamese Network for Thermal and Optical Image Registration Termal ve Optik Görüntü Çakiştirmasi için Swin Dönüştürücü tabanli Siyam Aǧi

Elsaeidy M., Yagmur I. C., Ateş H. F., GÜNTÜRK B. K.

31st IEEE Conference on Signal Processing and Communications Applications, SIU 2023, İstanbul, Turkey, 5 - 08 July 2023, (Full Text) identifier

  • Publication Type: Conference Paper / Full Text
  • Doi Number: 10.1109/siu59756.2023.10224035
  • City: İstanbul
  • Country: Turkey
  • Keywords: Keypoint, Multi-modal image registration, Transformer network
  • Istanbul Medipol University Affiliated: Yes

Abstract

The process of multi-modal image registration is fundamental in remote sensing and visual navigation applications. However, existing image registration methods that are designed for single modality images do not provide satisfactory results when applied to multi-modal image registration. In this research, our objective is to achieve highly accurate alignment of both infrared and optical (visible range) images. To accomplish this goal, we explore the effectiveness of the Swin Transformer encoder and cosine loss in enhancing the keypoint-based image registration process. Simulation results show the improvement achieved in multi-modal registration by using a transformer based Siamese network.