Conference proceeding
Make a Feint to the East While Attacking in the West: Blinding LLM-Based Code Auditors with Flashboom Attacks
Proceedings - IEEE Symposium on Security and Privacy, pp 576-594
12 May 2025
Abstract
LLM-based vulnerability auditors (e.g., GitHub Copilot) represent a significant advancement in automated code analysis, offering precise detection of security vulnerabilities. This paper explores the potential to circumvent LLM-based vulnerability auditors by diverting their focus, decided by the LLM attention mechanism, away from real vulnerable code segments. In these LLM-based vulnerability auditors, the attention mechanism is supposed to focus on potentially vulnerable code sections to identify security issues. Our approach introduces high-attention code snippets (code fragments designed to draw focus) into the codebase under review. By strategically diverting the model's focus away from actual vulnerabilities, this technique effectively "blinds" the LLM, resulting in missed detections. To scale this approach, we present Crazy-Ivan 1 1 Source code, dataset and attack results are available at https://github.com/oxygen-hunter/Flashboom., an automated system that identifies and seamlessly integrates high-attention code snippets, shifting focus away from genuine vulnerabilities to decoy functions. Through systematic function-level prioritization and refinement, Crazy-Ivan optimizes the blinding effect, producing the Flashboom that can reduce the model's capacity to detect true security risks. Our evaluation underscores the effectiveness of Flashboom, achieving blinding success rates of up to 96.3% on CodeLlama and 83.05% on Gemma, with notable cross-model transferability and applicability across multiple programming languages. In a case study with GitHub Copilot, Flashboom led the tool to overlook a critical blockchain vulnerability, underscoring the security implications of such attention-diverting attacks and the risks inherent in relying solely on LLM-based automated auditing systems. We have reported our findings to the respective LLM-based code auditor vendors, who have acknowledged the issues and are currently working on fixes.
Metrics
11 Record Views
Details
- Title
- Make a Feint to the East While Attacking in the West: Blinding LLM-Based Code Auditors with Flashboom Attacks
- Creators
- Xiao Li - Nanjing UniversityYue Li - Nanjing UniversityHao Wu - Nanjing UniversityYue Zhang - Drexel UniversityKaidi Xu - Drexel UniversityXiuzhen Cheng - Shandong UniversitySheng Zhong - Nanjing UniversityFengyuan Xu - Nanjing University
- Publication Details
- Proceedings - IEEE Symposium on Security and Privacy, pp 576-594
- Series
- IEEE Symposium on Security and Privacy
- Publisher
- IEEE
- Number of pages
- 19
- Grant note
- National Key R&D Program of China: 2022YFF0604503 NSFC: 62272224, 62432004, 62302207, 62272215 Leading Edge Technology Program of Jiangsu Natural Science Foundation: BK20202001 Science Foundation for Youths of Jiangsu Province: BK20220772
We sincerely thank the anonymous shepherd and reviewers for their constructive feedback and insightful suggestions. This work was supported by the National Key R&D Program of China under Grant 2022YFF0604503; in part by NSFC under Grant 62272224, Grant 62432004, Grant 62302207, and Grant 62272215; in part by the Leading Edge Technology Program of Jiangsu Natural Science Foundation under Grant BK20202001; and in part by the Science Foundation for Youths of Jiangsu Province under Grant BK20220772.
- Resource Type
- Conference proceeding
- Language
- English
- Academic Unit
- Computer Science (Computing)
- Web of Science ID
- WOS:001540716400032
- Scopus ID
- 2-s2.0-105009323114
- Other Identifier
- 991022060031904721