2024/02/06 by Nadia Alshahwan, Alshahwan, Nadia, Mark Harman +10 · 3 voices · 1 citation
Business, Management and Accounting · Computer Science · #Business Process Modeling and Analysis #FOS: Computer and information sciences #Software Engineering (cs.SE) #cs.SE
paper · pdf · doi:10.48550/arxiv.2402.04380
openalex publication_date 2024/02/06 · arxiv published 2024/02/06 · arxiv updated 2024/02/06 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
In this paper we address the following question: How can we use Large Language Models (LLMs) to improve code independently of a human, while ensuring that the improved code - does not regress the properties of the original code? - improves the original in a verifiable and measurable way? To address this question, we advocate Assured LLM-Based Software Engineering; a generate-and-test approach, inspired by Genetic Improvement. Assured LLMSE applies a series of semantic filters that discard code that fails to meet these twin guarantees. This overcomes the potential problem of LLM's propensity to hallucinate. It allows us to generate code using LLMs, independently of any human. The human plays the role only of final code reviewer, as they would do with code generated by other human engineers. This paper is an outline of the content of the keynote by Mark Harman at the International Workshop on Interpretability, Robustness, and Benchmarking in Neural Software Engineering, Monday 15th April 2024, Lisbon, Portugal.