Automated Instance Label Generation for Traffic Domain to Enhance YOLACT-YOLO

This paper presents a novel approach to generate accurate instance masks for challenging traffic surveillance scenes to allow re-training YOLACT-YOLOv9 model for surveillance domain. We propose a two-stage method as follows. First, we automate the generation of instance segmentation labels by applying the Segment Anything Model using 2D bounding boxes as prompts to generate instance masks. Three specific adaptations are proposed to improve the segmentation quality: adopting multiple masks per object, incorporating foreground/background knowledge for each object, and filtering of challenging instance masks. Second, we introduce a novel multi-scale soft loss function for training YOLACT-YOLOv9 to handle missing instance masks based on consistency regularization. Experimental results show that detection and segmentation accuracy is significantly improved, particularly in challenging traffic scenes. The final model obtains the detection and instance segmentation performance of 92% mAP and 84% mAP, respectively.