2025/01/30 by Md Motaleb Hossen Manik, Manik, Md Motaleb Hossen · 2 voices · 2 citations
Computer Science · Medicine · Social Sciences · #Artificial Intelligence (cs.AI) #Artificial Intelligence in Healthcare and Education #Code generation #Coding (social sciences) #Correctness #Ethics and Social Impacts of AI #FOS: Computer and information sciences #Leverage (statistics) #Python (programming language) #Software Engineering (cs.SE) #Software Engineering Research #Software quality #Source code #Source lines of code #cs.AI #cs.SE
paper · pdf · doi:10.48550/arxiv.2502.18467
openalex publication_date 2025/01/30 · arxiv published 2025/01/30 · arxiv updated 2025/01/30 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Background: AI-powered code generation, fueled by Large Language Models (LLMs), is revolutionizing software development. Models like OpenAI's Codex and GPT-4, alongside DeepSeek, leverage vast code and natural language datasets. However, ensuring code quality, correctness, and managing complex tasks remains challenging, necessitating thorough evaluation. Methodology: This research compares ChatGPT (version o1) and DeepSeek (version R1) for Python code generation using online judge coding challenges. It evaluates correctness (online judge verdicts, up to three attempts), code quality (Pylint/Flake8), and efficiency (execution time/memory usage). Results: DeepSeek demonstrated higher correctness, particularly on algorithmic tasks, often achieving 'Accepted' on the first attempt. ChatGPT sometimes requires multiple attempts or failures. ChatGPT encountered fewer issues, used comparable or slightly less memory, consumed less execution times and wrote fewer lines of code. Conclusion: DeepSeek exhibited superior correctness in Python code generation, often requiring fewer attempts, suggesting an advantage in algorithmic problem-solving. Both models showed almost similar efficiency in execution time and memory use. Finally, this research provides insights for developers choosing AI coding assistants and informs future AI-driven software development research.