Conference proceeding
Fusing visual and sensor data with multimodal transformers for indoor environmental quality assessment in educational settings
E3S web of conferences, v 716, 06005
01 Jan 2026
Abstract
Indoor environmental quality (IEQ) plays a central role in the health, comfort, and cognitive performance of building occupants, particularly in educational environments where students spend extended periods of time. Prior work has shown that elevated carbon dioxide (CO2) concentrations, increased total volatile organic compounds (TVOCs), particulate matter (PM), and suboptimal hygrothermal conditions negatively affect attention, memory, and decision-making among students. As educational buildings move toward more occupant-centric and energy-efficient operation, artificial intelligence (AI) tools offer new opportunities for real-time IEQ assessment. However, existing approaches often rely solely on sensor data and lack the contextual awareness needed to interpret environmental shifts that are influenced by occupancy, activity patterns, or space configuration. This study presents a multimodal transformer (MulT) model that fuses synchronized environmental sensor data with time-lapse RGB images to assess seven IEQ indicators: dry-bulb temperature (T), relative humidity (RH), CO2 concentrations, TVOCs, and PM (PM1, PM2.5, PM10). Data were collected at 5-minute intervals from four representative educational spaces, namely, a conference room, a hybrid laboratory, and two classrooms, over 7 days each from March to May 2025. The model combines a convolutional neural network (CNN) for local visual features, a Vision Transformer (ViT) for global spatial reasoning, and a cross-modal fusion transformer to jointly estimate all IEQ variables. Results show significantly strong convergence and robust generalization. On a dedicated test set containing unseen paired samples from all environments, the model achieved a test mean squared error (MSE) of 85.05 and a mean absolute error (MAE) of 3.64, which demonstrates that the learned multimodal embeddings transfer well to new images and sensor combinations, preserving accuracy outside the training distribution. The MAE of 3.6 across seven heterogeneous IEQ variables further highlights the model's strength, given that the targets span different physical scales. The approach effectively captures both environmental dynamics and visual cues such as occupancy level, equipment use, and lighting conditions, improving assessment stability in diverse indoor scenarios. The findings highlight the potential of MulT-based architectures for real-time IEQ monitoring in educational settings and point toward broader applications in automated building diagnostics, early-warning systems, and adaptive HVAC control strategies.
Metrics
1 Record Views
Details
- Title
- Fusing visual and sensor data with multimodal transformers for indoor environmental quality assessment in educational settings
- Creators
- Carlos Faubel Alama (Corresponding Author) - Drexel UniversityMowen Derian - The University of Texas at San AntonioMiri Marjan - Drexel University, Architecture, Design, and UrbanismAlban Ursula Demarquet - Drexel University, Architecture, Design, and UrbanismMartinez-Molina Antonio - Drexel University, Architecture, Design, and UrbanismAlamaniotis Miltiadis - The University of Texas at San Antonio
- Publication Details
- E3S web of conferences, v 716, 06005
- Publisher
- EDP Sciences
- Resource Type
- Conference proceeding
- Language
- English
- Academic Unit
- Civil, Architectural, and Environmental Engineering; Architecture, Design, and Urbanism
- Other Identifier
- 991022199650104721