My research focuses on advancing real-time traffic surveillance through cutting-edge object detection and instance segmentation techniques. One of my key contributions is addressing the challenge of instance segmentation in traffic surveillance, where suitable public datasets are lacking. I have explored automated methods for generating instance segmentation labels for custom datasets, enabling the fine-tuning of state-of-the-art segmentation models. In particular, I present a novel two-stage approach for generating accurate instance masks to retrain the YOLACT-YOLOv9 model for traffic surveillance. This method leverages the Segment Anything Model (SAM) with 2D boxes as prompts, and incorporates three key adaptations—multi-mask per object generation, object-level foreground/background priors, and filtering of low-quality masks. Additionally, I introduce a multi-scale soft loss function based on consistency regularization to handle incomplete labels. As a result, this pipeline achieves significant improvements in detection and segmentation accuracy in complex traffic scenes.

Beyond 2D detection, my work extends to 3D object detection using monocular cameras. Employing the KM3D CNN-based 3D detection model, I have adapted it for traffic surveillance applications, which traditionally lack 3D annotation datasets. To overcome this limitation, I developed four annotation configurations that leverage camera calibration and scene information. Notably, my novel Simple Box method provides an efficient 3D box construction approach and precise 3D box estimation up to 125 meters.

Additionally, I have contributed to optimizing computational efficiency in edge-based surveillance by integrating an early-out branch (EOBranch) into YOLO architectures. This technique reduces processing time and energy consumption without compromising detection accuracy. The EOBranch allows early exit for background frames, significantly lowering computational demands. We evaluated the approach in YOLOv6 and YOLOv9 under various training strategies, branch placements, and architectural extensions. Reducing processing time by up to 46% over 24 hours of traffic video. These findings highlight the potential for substantial energy savings and improved throughput in real-time edge-based surveillance systems.

My latest research explores multi-modal object detection from a drone perspective, expanding surveillance capabilities beyond fixed-camera setups to enhance situational awareness and detection accuracy in dynamic environments.

Dick Scholte received his MSc degree in signal processing within electrical engineering deparment at the Eindhoven University of Technology in 2021. He started working as a research and development engineer at ViNotion B.V. in Eindhoven. From June 2024, Scholte started his PhD career at the AIMS research lab at Eindhoven University of Technology.