2021/10/14 by Forrest Huang, Gang Li, Huang, Forrest +9 · 5 citations
Computer Science · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Human-Computer Interaction (cs.HC) #Innovative Human-Technology Interaction #Machine Learning (cs.LG) #Persona Design and Applications #Software Engineering Research #cs.AI #cs.HC #cs.LG
paper · pdf · doi:10.48550/arxiv.2110.07775
arxiv created 2021/10/14 · openalex publication_date 2021/10/14 · arxiv updated 2021/10/18 · openalex created_date 2021/10/25 · openalex updated_date 2026/07/28
The design process of user interfaces (UIs) often begins with articulating high-level design goals. Translating these high-level design goals into concrete design mock-ups, however, requires extensive effort and UI design expertise. To facilitate this process for app designers and developers, we introduce three deep-learning techniques to create low-fidelity UI mock-ups from a natural language phrase that describes the high-level design goal (e.g. "pop up displaying an image and other options"). In particular, we contribute two retrieval-based methods and one generative method, as well as pre-processing and post-processing techniques to ensure the quality of the created UI mock-ups. We quantitatively and qualitatively compare and contrast each method's ability in suggesting coherent, diverse and relevant UI design mock-ups. We further evaluate these methods with 15 professional UI designers and practitioners to understand each method's advantages and disadvantages. The designers responded positively to the potential of these methods for assisting the design process.