Smartphone assisted 3D structural reconstruction and real-time scene segmentation modeling research

Since it is difficult to extract and understand the multi-dimensional spatial feature information from twodimensional images, three-dimensional modeling and scene segmentation of structures are necessary. At present, most three-dimensional reconstruction technologies have the issues of expensive data acquisition equipment, long modelling time, and poor robustness under low texture. Considering the above concerns, this study proposed an integrated laser and visible light method based on the built-in lidar sensor of a smartphone to reduce the cost and achieve fast reconstruction of structural 3D point cloud models. In addition, a lightweight segmentation backbone network based on spatio-temporal redundancy downsampling mechanism was developed and deployed, which can realize real-time segmentation and information understanding of the environment. An unfurnished apartment was selected to verify the performance of the proposed method, and a floor plan was finally generated for comparison with the real values. Results show that the robustness of point cloud reconstruction was enhanced and the mIoU of scene segmentation on the test set reached 65.2% at 200FPS. The dimension of the generated floor plan has also obtained high accuracy (over 93% and mostly up to 97%), which is efficient for engineers to quickly and accurately understand on-site spatial structure information.