2022/02/15 by Ranci Ren, John W. Castro, Adrian Santos +5 · 13 citations
Computer Science · #AI in Service Interactions #Applications of UML #Chatbot #Computer science #Data science #Programming language #Software #Software engineering #Spreadsheets and End-User Computing #UML tool #Unified Modeling Language #Usability and User Interface Design #World Wide Web
paper · pdf · doi:10.1109/tse.2022.3150720
published in IEEE Transactions on Software Engineering 49(1), 364-383 (IEEE Computer Society)
openalex publication_date 2022/02/15 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/01
Context:Recent developments in natural language processing have facilitated the adoption of chatbots in typically collaborative software engineering tasks (such as diagram modelling). Families of experiments can assess the performance of tools and processes and, at the same time, alleviate some of the typical shortcomings of individual experiments (e.g., inaccurate and potentially biased results due to a small number of participants).Objective:Compare the usability of a chatbot for collaborative modelling (i.e., SOCIO) and an online web tool (i.e., Creately).Method:We conducted a family of three experiments to evaluate the usability of SOCIO against the Creately online collaborative tool in academic settings.Results:The student participants were faster at building class diagrams using the chatbot than with the online collaborative tool and more satisfied with SOCIO. Besides, the class diagrams built using the chatbot tended to be more concise —albeit slightly less complete.Conclusion:Chatbots appear to be helpful for building class diagrams. In fact, our study has helped us to shed light on the future direction for experimentation in this field and lays the groundwork for researching the applicability of chatbots in diagramming.