2023/05/16 by Giuseppe Destefanis, Destefanis, Giuseppe, Silvia Bartolucci +3 · 1 citation
Computer Science · Decision Sciences · #Advanced Data Storage Technologies #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning and Data Classification #Scientific Computing and Data Management #Software Engineering (cs.SE)
paper · pdf · doi:10.48550/arxiv.2305.09402
openalex publication_date 2023/05/16 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
This paper evaluates the capability of two state-of-the-art artificial intelligence (AI) models, GPT-3.5 and Bard, in generating Java code given a function description. We sourced the descriptions from CodingBat.com, a popular online platform that provides practice problems to learn programming. We compared the Java code generated by both models based on correctness, verified through the platform's own test cases. The results indicate clear differences in the capabilities of the two models. GPT-3.5 demonstrated superior performance, generating correct code for approximately 90.6% of the function descriptions, whereas Bard produced correct code for 53.1% of the functions. While both models exhibited strengths and weaknesses, these findings suggest potential avenues for the development and refinement of more advanced AI-assisted code generation tools. The study underlines the potential of AI in automating and supporting aspects of software development, although further research is required to fully realize this potential.