Journal article
Vision-Language Artificial Intelligence for Robotic-Based Monitoring: Concrete Defect Detection, Classification, and Localization in Two-Dimensional Maps
Journal of computing in civil engineering, v 40(2), 04025157
01 Mar 2026
Featured in Collection : Drexel's Newest Publications
Abstract
AbstractThis paper introduces a novel framework that combines vision-language models (VLMs) and localization techniques to detect, classify, and localize visual structural defects using moving platforms such as robots and handheld devices, with an emphasis on concrete defects. The framework interactively searches for defects by analyzing images captured from various locations and perspectives, employing, but not limited to, the vision transformer for open-world localization (OWL-ViT). Upon detection, defect localization is estimated using the moving platform’s position, orientation, view angles, and depth measurements, with a postprocessing module further enhancing detection relevancy via mixing estimations from distinct views. Evaluations in the real world, in simulation, and on a custom dataset include prompt engineering and a comparison with the classic models (e.g., YOLO). The framework achieves an average Euclidean error of 0.56 m with OWL-ViT’s optimal prompt, compared to 0.75 m with YOLO and 0.97 with DETR, demonstrating its potential for robotic inspection of concrete structures.
Metrics
21 Record Views
Details
- Title
- Vision-Language Artificial Intelligence for Robotic-Based Monitoring: Concrete Defect Detection, Classification, and Localization in Two-Dimensional Maps
- Creators
- Farzad Azizi Zade - Ferdowsi University of MashhadArvin Ebrahimkhanlou (Corresponding Author) - Drexel University
- Publication Details
- Journal of computing in civil engineering, v 40(2), 04025157
- Publisher
- American Society of Civil Engineers
- Number of pages
- 20
- Resource Type
- Journal article
- Language
- English
- Academic Unit
- Civil, Architectural, and Environmental Engineering; Mechanical Engineering and Mechanics
- Web of Science ID
- WOS:001663011200026
- Scopus ID
- 2-s2.0-105023275190
- Other Identifier
- 991022133488504721
UN Sustainable Development Goals (SDGs)
This publication has contributed to the advancement of the following goals:
InCites Highlights
Data related to this publication, from InCites Benchmarking & Analytics tool:
- Collaboration types
- Domestic collaboration
- International collaboration
- Web of Science research areas
- Computer Science, Interdisciplinary Applications
- Engineering, Civil