Geo-spatial traffic behaviour analysis and anomaly detection for ITS applications

Understanding the behaviour of traffic participants within the geo-spatial context of road/intersection topology is a vital prerequisite for any smart ITS application. This article presents a video-based traffic analysis and anomaly detection system covering the complete data processing pipeline, including sensor data acquisition, analysis, and digital twin reconstruction. The system solves the challenge of geo-spatial mapping of captured visual data onto the road/intersection topology by semantic analysis of aerial data. Additionally, the automated camera calibration component enables instant camera pose estimation to map traffic agents onto the road/intersection surface accurately. A novel aspect is approaching the anomaly detection problem by AI analysis of both the spatio-temporal visual clues and the geo-spatial trajectories for all type of traffic participants, such as pedestrians, bicyclists, and vehicles. This enables recognition of anomalies related to either traffic-rule violations, for example, jaywalking, improper turns, zig-zag driving, unlawful stops, or behavioural anomalies: littering, accidents, falling, vandalism, violence, infrastructure collapse etc. The method achieves leading anomaly detection results on benchmark datasets World Cup 2014, UCF-Crime, XD-Violence, and ShanghaiTech. All the obtained results are streamed and rendered in real-time by the developed TGX digital twin visualizer. The complete system has been deployed and validated on the roads of Helmond town in The Netherlands.