LLM4TDD: Best Practices for Test Driven Development Using Large Language Models

2023/12/07 by Sanyogita Piya, Piya, Sanyogita, Allison Sullivan +1 · 1 citation
Computer Science · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Software Engineering (cs.SE) #Software Engineering Research #Software Reliability and Analysis Research #Software Testing and Debugging Techniques

paper · pdf · doi:10.48550/arxiv.2312.04687

openalex publication_date 2023/12/07 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In today's society, we are becoming increasingly dependent on software systems. However, we also constantly witness the negative impacts of buggy software. Program synthesis aims to improve software correctness by automatically generating the program given an outline of the expected behavior. For decades, program synthesis has been an active research field, with recent approaches looking to incorporate Large Language Models to help generate code. This paper explores the concept of LLM4TDD, where we guide Large Language Models to generate code iteratively using a test-driven development methodology. We conduct an empirical evaluation using ChatGPT and coding problems from LeetCode to investigate the impact of different test, prompt and problem attributes on the efficacy of LLM4TDD.

Cited by

• TENET: Leveraging Tests Beyond Validation for Code Generation

Related

• Examination of Code generated by Large Language Models
• Tests as Prompt: A Test-Driven-Development Benchmark for LLM Code Generation
• Holistic Evaluation of State-of-the-Art LLMs for Code Generation
• Test-Driven Development and LLM-based Code Generation
• The Program Testing Ability of Large Language Models for Code