Detection and quantification of bolt loosening using RGB-D camera and Mask R-CNN

Bolt loosening is one of the most common types of damage for bolt-connected plates. Existing vision techniques detect bolt loosening based on the measurement of bolt rotation or the exposure of bolt threads. However, these techniques examine bolt tightness only in a qualitative manner, or require a reference measurement at the initially tightened state of the bolt for quantitative estimation. In this study, the exposed shank length of a bolt is quantitatively measured using an RGB-depth camera and a mask-region-based convolutional neural network but without requiring any measurement from the initial state of the bolt. The performance of the proposed technique is validated by conducting lab-scale experiments, in which the angle and distance of the camera are varied with respect to a target inspection area. The proposed technique successfully detects bolt loosening at exposed shank length over 3 mm with a resolution of 1 mm and 97% accuracy at different camera angles (40°–90°) and distances (up to 65 cm).