Exploiting street-level panoramic images for large-scale automated surveying of traffic sign

Accurate and up-to-date inventories of traffic signs contribute to efficient road maintenance and a high road safety. This paper describes a system for the automated surveying of road signs from street-level images. This is an extremely challenging task, as the involved capturings are non-densely sampled, captured under a wide range of weather conditions and signs may be distorted. The described system is designed in a generic and learning-based fashion, which enables the recognition of different sign appearance classes with the same algorithms, based on class-specific training data. The system starts with detection of the signs visible within each image, using a detection cascade. Next, the 3D position of the signs that are detected consequently within consecutive capturings is calculated. Afterwards, each positioned road sign is classified to retrieve its sign type, thereby exploiting all detections used during positioning of the respective sign. The presented system is intended for large-scale application and currently supports 11 sign appearance classes, containing 176 different sign types. Performance evaluations conducted on a large, real-world dataset (68,010 images) show that our approach accurately positions 95.5 % of the 3,385 present signs, where 96.3 % of them are also correctly classified. Furthermore, our system localized 98.5 % of the signs in at least a single image. Our system design allows for appending a limited manual correction stage to attain a very high performance, so that sign inventories can be created cost effectively.