Identification of ship trajectory using deep learning-based segmentation and stereovision

River transportation is a significant component of the overall transportation system. Typically, there are surveillance cameras implemented on river bank to avoid collisions between ships and bridges across rivers. However, some of the routes may only contain limited or malfunctioned cameras, making the monitoring of ships occluded. In this study, we propose a deep learning-based framework that identifies the trajectory of a ship in the real world by using the surveillance videos. The proposed framework consists of three modules: object detection, object tracking, and coordinate projection. We implement the Mask RCNN model for object detection to determine the ship position in each video frame and compute the ship centroid as the image coordinates of the ship. We then employ DeepSort as the object tracker, which matches and tracks the detected object in each frame and combines all instances of object detection in the video to output the ship trajectory. For coordinate projection, we incorporate the P3P method and Zhang's algorithm to determine the intrinsic matrix and extrinsic matrix, respectively. The image coordinates of the ships are therefore converted into world coordinates. In addition, we develop an approach to calibrate the ship trajectory out of the coverage using the results from multi-camera triangulation. Meanwhile, the continuity in ship trajectory is enhanced as well. Results demonstrate that the ship trajectory becomes smoother in the evaluation using acceleration variability and directional change. The proposed approach reduces the acceleration variability score from 2.75 to 1.54 and improves the directional hange score from 0.85 to 0.09.