AAAI Publications, Thirty-Second AAAI Conference on Artificial Intelligence

Font Size: 
Multispectral Transfer Network: Unsupervised Depth Estimation for All-Day Vision
Namil Kim, Yukyung Choi, Soonmin Hwang, In So Kweon

Last modified: 2018-04-27


To understand the real-world, it is essential to perceive in all-day conditions including cases which are not suitable for RGB sensors, especially at night. Beyond these limitations, the innovation introduced here is a multispectral solution in the form of depth estimation from a thermal sensor without an additional depth sensor.Based on an analysis of multispectral properties and the relevance to depth predictions, we propose an efficient and novel multi-task framework called the Multispectral Transfer Network (MTN) to estimate a depth image from a single thermal image. By exploiting geometric priors and chromaticity clues, our model can generate a pixel-wise depth image in an unsupervised manner. Moreover, we propose a new type of multitask module called Interleaver as a means of incorporating the chromaticity and fine details of skip-connections into the depth estimation framework without sharing feature layers. Lastly, we explain a novel technical means of stably training and covering large disparities and extending thermal images to data-driven methods for all-day conditions. In experiments, we demonstrate the better performance and generalization of depth estimation through the proposed multispectral stereo dataset, including various driving conditions.


multispectral learning; depth estimation; all-day vision; deep neural network; transfer learning; multi-task learning

Full Text: PDF