How Secure is Code Generated by ChatGPT?

2023/04/19 by Raphaël Khoury, Anderson R. Avila, Khoury, Raphaël +5 · 1 voice · 29 citations
Computer Science · Medicine · #Advanced Malware Detection Techniques #Adversarial Robustness in Machine Learning #Artificial Intelligence in Healthcare and Education #cs.CR

paper · pdf · doi:10.48550/arxiv.2304.09655

arxiv published 2023/04/19 · arxiv updated 2026/04/07

Abstract

In recent years, large language models have been responsible for great advances in the field of artificial intelligence (AI). ChatGPT in particular, an AI chatbot developed and recently released by OpenAI, has taken the field to the next level. The conversational model is able not only to process human-like text, but also to translate natural language into code. However, the safety of programs generated by ChatGPT should not be overlooked. In this paper, we perform an experiment to address this issue. Specifically, we ask ChatGPT to generate a number of program and evaluate the security of the resulting source code. We further investigate whether ChatGPT can be prodded to improve the security by appropriate prompts, and discuss the ethical aspects of using AI to generate code. Results suggest that ChatGPT is aware of potential vulnerabilities, but nonetheless often generates source code that are not robust to certain attacks.

Cited by

• WildCode: An Empirical Analysis of Code Generated by ChatGPT
• LLM-CSEC: Empirical Evaluation of Security in C/C++ Code Generated by Large Language Models
• Security Vulnerabilities in AI-Generated Code: A Large-Scale Analysis of Public GitHub Repositories
• A Large-Scale Comprehensive Measurement of AI-Generated Code in Real-World Repositories
• Is Your Prompt Poisoning Code? Defect Induction Rates and Security Mitigation Strategies
• TypePilot: Leveraging the Scala Type System for Secure LLM-generated Code
• CLASP: Training-Free LLM-Assisted Source Code Watermarking via Semantic-Preserving Transformations
• Fortifying LLM-Based Code Generation with Graph-Based Reasoning on Secure Coding Practices
• Secure-Instruct: An Automated Pipeline for Synthesizing Instruction-Tuning Datasets Using LLMs for Secure Code Generation
• Ethical AI prompt recommendations in large language models using collaborative filtering
• Paladin: Defending LLM-enabled Phishing Emails with a New Trigger-Tag Paradigm
• Amazon Nova AI Challenge -- Trusted AI: Advancing secure, AI-assisted software development
• Detecting LLM-generated Code with Subtle Modification by Adversarial Training
• Do Generative AI Tools Ensure Green Code? An Investigative Study
• Small Data Explainer -- The impact of small data methods in everyday life
• Is Quantization a Deal-breaker? Empirical Insights from Large Code Models
• How Safe Are AI-Generated Patches? A Large-scale Study on Security Risks in LLM and Agentic Automated Program Repair on SWE-bench
• Guiding AI to Fix Its Own Flaws: An Empirical Study on LLM-Driven Secure Code Generation
• I Know Which LLM Wrote Your Code Last Summer: LLM generated Code Stylometry for Authorship Attribution
• The Scales of Justitia: A Comprehensive Survey on Safety Evaluation of LLMs
• From Bias To Improved Prompts: A Case Study of Bias Mitigation of Clone Detection Models
• Teaching an Old LLM Secure Coding: Localized Preference Optimization on Distilled Preferences
• CodeMirage: A Multi-Lingual Benchmark for Detecting AI-Generated and Paraphrased Source Code from Production-Level LLMs
• Eradicating the Unseen: Detecting, Exploiting, and Remediating a Path Traversal Vulnerability across GitHub
• The Illusion of Safety: Multi-Tier Verification of AI vs. Human C++ Code
• SoK: AI Secure Code Generation: Progress, Pitfalls, and Paths Forward
• The Hidden Risks of LLM-Generated Web Application Code: A Security-Centric Evaluation of Code Generation Capabilities in Large Language Models
• Hallucination by Code Generation LLMs: Taxonomy, Benchmarks, Mitigation, and Challenges
• Surgical Repair of Insecure Code Generation in LLMs
• Achilles Heel of Distributed Multi-Agent Systems

Discussions

• Code, der nicht robustg gegen Manipulationen ist, also leicht gehackt werden kann. Hier sind einige Beispiele. "How Secure is Code Generated by ChatGPT?" arxiv.org/pdf/2304.09655 [bsky, 1 points, 1 comments]

Related

• Secure Coding with AI -- From Detection to Repair
• A Qualitative Study on Using ChatGPT for Software Security: Perception vs. Practicality
• Security Analysis of ChatGPT: Threats and Privacy Risks
• WildCode: An Empirical Analysis of Code Generated by ChatGPT
• Just another copy and paste? Comparing the security vulnerabilities of ChatGPT generated code and StackOverflow answers