Enriching visual information in images through fusion with thermal imaging in low-light surveillance scenes

Multimodal image fusion regards the combination of information from a pair of sensors operating in different segments of the electromagnetic spectrum. In the surveillance of low-light scenes, the visible spectrum alone does not capture a significant amount of information. Thus, fusing the information available from a far infrared (FIR) sensor can boost the final image to create a more complete scene. Existing thermal and visible image fusion methods are poorly optimized for use in surveillance of dark scenes, where contrast and textures are mostly absent, thus producing an image with low perceptual quality. With the introduction of a novel method, we improved the state-of-the-art (SOTA) multimodal image fusion algorithm. Our proposed method is capable of successfully recovering most textures and produces an image with relatively high contrast, sharpness, and less noise, thus improving the perceptual quality of the final image. This is particularly important for the surveillance of dark scenes in premises that require high security. We experimentally validate our method (qualitatively and quantitatively) to show the increased performance of the proposed method. The results of the mean opinion scoring (MOS) in our method show an improvement of 5.42% compared to the next-best method.