Preprint
New Spiking Architecture for Multi-Modal Decision-Making in Autonomous Vehicles
01 Dec 2025
Abstract
This work proposes an end-to-end multi-modal reinforcement learning framework for high-level decision-making in autonomous vehicles. The framework integrates heterogeneous sensory input, including camera images, LiDAR point clouds, and vehicle heading information, through a cross-attention transformer-based perception module. Although transformers have become the backbone of modern multi-modal architectures, their high computational cost limits their deployment in resource-constrained edge environments. To overcome this challenge, we propose a spiking temporal-aware transformer-like architecture that uses ternary spiking neurons for computationally efficient multi-modal fusion. Comprehensive evaluations across multiple tasks in the Highway Environment demonstrate the effectiveness and efficiency of the proposed approach for real-time autonomous decision-making.
Metrics
5 Record Views
Details
- Title
- New Spiking Architecture for Multi-Modal Decision-Making in Autonomous Vehicles
- Creators
- Aref GhoreisheeAbhishek MishraLifeng ZhouJohn WalshNagarajan Kandasamy
- Resource Type
- Preprint
- Language
- English
- Academic Unit
- Electrical and Computer Engineering
- Other Identifier
- 991022136465604721