Files and links (1)
Preprint (Author's original)arXiv.org - Non-exclusive license to distribute, Open
Preprint
Do LLMs Consider Security? An Empirical Study on Responses to Programming Questions
19 Feb 2025
Abstract
The widespread adoption of conversational LLMs for software development has
raised new security concerns regarding the safety of LLM-generated content. Our
motivational study outlines ChatGPT's potential in volunteering
context-specific information to the developers, promoting safe coding
practices. Motivated by this finding, we conduct a study to evaluate the degree
of security awareness exhibited by three prominent LLMs: Claude 3, GPT-4, and
Llama 3. We prompt these LLMs with Stack Overflow questions that contain
vulnerable code to evaluate whether they merely provide answers to the
questions or if they also warn users about the insecure code, thereby
demonstrating a degree of security awareness. Further, we assess whether LLM
responses provide information about the causes, exploits, and the potential
fixes of the vulnerability, to help raise users' awareness. Our findings show
that all three models struggle to accurately detect and warn users about
vulnerabilities, achieving a detection rate of only 12.6% to 40% across our
datasets. We also observe that the LLMs tend to identify certain types of
vulnerabilities related to sensitive information exposure and improper input
neutralization much more frequently than other types, such as those involving
external control of file names or paths. Furthermore, when LLMs do issue
security warnings, they often provide more information on the causes, exploits,
and fixes of vulnerabilities compared to Stack Overflow responses. Finally, we
provide an in-depth discussion on the implications of our findings and present
a CLI-based prompting tool that can be used to generate significantly more
secure LLM responses.
Metrics
10 Record Views
Details
- Title
- Do LLMs Consider Security? An Empirical Study on Responses to Programming Questions
- Creators
- Amirali SajadiBinh LeAnh NguyenKostadin DamevskiPreetha Chatterjee
- Resource Type
- Preprint
- Language
- English
- Academic Unit
- Computer Science (Computing)
- Other Identifier
- 991022030047404721