December 6, 2021 Figure Open Access

Unsupervised Data Fusion with Deeper Perspective: A Novel Multi-Sensor Deep Clustering Algorithm

Rafiezadeh Shahi, Kasra; Ghamisi, Pedram; Rasti, Behnood; Scheunders, Paul; Gloaguen, Richard

The ever-growing developments in technology to capture different types of image data (e.g., hyperspectral imaging and Light Detection and Ranging (LiDAR)-derived digital surface model (DSM)), along with new processing techniques, have led to a rising interest in imaging applications for Earth observation. However, analyzing such datasets in parallel, remains a challenging task. In this paper, we propose a multi-sensor deep clustering (MDC) algorithm for the joint processing of multi-source imaging data. The architecture of MDC is inspired by autoencoder (AE)-based networks. The MDC paradigm includes three parallel networks, a spectral network using an autoencoder structure, a spatial network using a convolutional autoencoder structure, and lastly, a fusion network that reconstructs the concatenated image information from the concatenated latent features from the spatial and spectral network. The proposed algorithm combines the reconstruction losses obtained by the aforementioned networks to optimize the parameters (i.e., weights and bias) of all three networks simultaneously. To validate the performance of the proposed algorithm, we use two multi-sensor datasets from different applications (i.e., geological and rural sites) as benchmarks. The experimental results confirm the superiority of our proposed deep clustering algorithm compared to a number of state-of-the-art clustering algorithms. The code will be available at: https://github.com/Kasra2020/MDC.