Color-based free-space segmentation using online disparity-supervised learning

This work contributes to vision processing for Advanced Driver Assist Systems (ADAS) and intelligent vehicle applications. We propose a color-only stixel segmentation framework to segment traffic scenes into free, drivable space and obstacles, which has a reduced latency to improve the real-time processing capabilities. Our system learns color appearance models for free-space and obstacle classes in an online and self-supervised fashion. To this end, it applies a disparity-based segmentation, which can run in the background of the critical system path, either with a time delay of several frames or at a frame rate that is only a third of that of the color-based algorithm. In parallel, the most recent video frame is analyzed solely with these learned color appearance models, without an actual disparity estimate and the corresponding latency. This translates into a reduced response time from data acquisition to data analysis, which is a critical property for high-speed ADAS. Our evaluation on two publicly available datasets, one of which we introduce as part of this work, shows that the color-only analysis can achieve similar or even better results in difficult imaging conditions, compared to the disparity-only method. Our system improves the quality of the free-space analysis, while simultaneously lowering the latency and the computational load.