Conference proceeding
Assessing semantic similarity in automated civil engineering image descriptions: evaluation of pretrained vision language models
Proceedings of SPIE, the international society for optical engineering, v 13438, 1343807
13 May 2025
Abstract
This study explores the use of pre-trained vision language models (VLM) to generate descriptive interpretations of images related to civil engineering materials and construction components. The research focuses on semantic analysis through the comparison of image descriptions derived from a publicly available online dataset. The online dataset includes 70 images from 10 different materials/objects. Categories include brick, concrete, electric, rebar, scaffolding, site work, stairs, stone, structural, and wood. The VLM's descriptive capabilities are compared with human-provided descriptions from a civil engineering graduate and two engineering interns. To assess the semantic similarity between the VLM-generated and human-generated descriptions, SentenceTransformers are employed to capture the overarching semantic meaning of the textual descriptions. The findings reveal that the best-performing model achieved an average similarity score of 81%, with a standard deviation of 5%. These outcomes indicate the potential of VLMs to support automated image description tasks in civil engineering, offering a promising direction for future applications in areas such as digital twinning and construction site monitoring.
Metrics
10 Record Views
Details
- Title
- Assessing semantic similarity in automated civil engineering image descriptions: evaluation of pretrained vision language models
- Creators
- Pedram Bazrafshan - Drexel UniversityKris Melag - Drexel UniversityArvin Ebrahimkhanlou - Drexel University
- Contributors
- Christopher Niezrecki (Editor) - University of Massachusetts LowellSaman Farhangdoust (Editor) - Embry-Riddle Aeronautical Univ. (United States)
- Publication Details
- Proceedings of SPIE, the international society for optical engineering, v 13438, 1343807
- Conference
- SPIE Smart Structures + Nondestructive Evaluation (Vancouver, British Columbia, Canada, 17 Mar 2025–20 Mar 2025)
- Publisher
- SPIE
- Number of pages
- 8
- Resource Type
- Conference proceeding
- Language
- English
- Academic Unit
- Civil, Architectural, and Environmental Engineering; Mechanical Engineering and Mechanics
- Scopus ID
- 2-s2.0-105014757184
- Other Identifier
- 991022078900504721