Conference proceeding
Robotic identification and localization of visual defects in concrete structures using a visual-language processing artificial intelligence model with prompt optimization
Proceedings of SPIE, the international society for optical engineering, 1343613
12 May 2025
Abstract
This research presents a novel artificial intelligence (AI)-based system to robotic visual inspection of defects in concrete structures. This system enables interactions with the surrounding environment of concrete structures. By utilizing zero-shot learning textual prompt optimization, the proposed method eliminates the need for labor-intensive data labeling and training sessions to identify and localize visual concrete defects in the video feed collected by the robotic system. The video feed is captured by a depth camera, which provides information about each pixel's distance from the camera. Subsequently, the system uses this depth data and the robot’s location to identify and locate visual concrete defects on the environmental map, followed by refining the location tags through post-processing. The effectiveness of the system is validated using a comprehensive real-world dataset collected by the authors. It is further evaluated and compared in a simulation environment against traditional deep-learning modeling techniques. In this context, the authors fine-tuned the You Only Look Once (YOLO)v8 model as the benchmark for deep learning models. The optimized prompt in zero-shot learning shows the localization error of Open-World Localization Vision Transformer (OWL-ViT) outperforms the results obtained from YOLOv8.
Metrics
2 Record Views
Details
- Title
- Robotic identification and localization of visual defects in concrete structures using a visual-language processing artificial intelligence model with prompt optimization
- Creators
- Farzad Azizi Zade - Ferdowsi University of MashhadArvin Ebrahimkhanlou - Drexel University
- Contributors
- Tzuyang Yu (Editor) - University of Massachusetts LowellAndrew L. Gyekenyesi (Editor) - Ohio Aerospace InstitutePeter J. Shull (Editor) - Pennsylvania State UniversityH. Felix Wu (Editor) - U.S. Dept. of Energy (United States)
- Publication Details
- Proceedings of SPIE, the international society for optical engineering, 1343613
- Publisher
- SPIE
- Resource Type
- Conference proceeding
- Language
- English
- Academic Unit
- Civil, Architectural, and Environmental Engineering; Mechanical Engineering and Mechanics
- Scopus ID
- 2-s2.0-105014738266
- Other Identifier
- 991022078781104721