1374341 - 3D Structural Information Sensing System (3D-SISS) based on Single Monocular Surveillance Camera for Fully-automatic Traffic Scene Perception

With the development of intelligent transportation system (ITS), surveillance cameras are becoming one of the most widely used traffic sensors worldwide for traffic data collection, monitoring and management. However, a notable amount of them is only used for data transmission from the on-site to human operators directly. The heavily manual process is always necessary to obtain the 3D-related traffic information, sometimes with limited accuracy and efficiency. As a result, the paper proposes a 3D structural information sensing system (3D-SISS), supported by a customized fully-automatic camera calibration, for real-time vehicle localization and scene reconstruction based on a single monocular surveillance camera. Three steps are integrated into the proposed 3D-SISS: 1) Pre-processing module for target vehicle detection, classification, and segmentation; 2) 2D car keypoints estimation; 3) Fully-automatic camera calibration for 2D keypoints projection to 3D coordinate. The traffic engineers and research communities can benefit from the proposed 3D-SISS in the following three perspectives with the novel framework. Firstly, in theory, an innovative fully-automatic 3D-SISS for traffic scene perception is customized and thoroughly tested. The system can provide a detailed traffic status analysis without any human input or initialization. Secondly, from the methodology perspective, the paper proposes an innovative crucial car keypoints detection (CKPD) model for accurate (overall 95.1%) car keypoints detection with single image input. Finally, in practice, 3D-SISS can be applied to any camera at any angle for real-time comprehensive traffic sensing. By taking advantage of the most widely deployed surveillance camera without installing new sensors, 3D-SISS can expand the widely-deployed cameras from the 2D to 3D view at a negligible cost.