Write a Blog >>
ICSE 2021
Mon 17 May - Sat 5 June 2021
Wed 26 May 2021 12:55 - 13:15 at Blended Sessions Room 3 - 2.2.3. GUI Design Chair(s): Ignacio Panach
Thu 27 May 2021 00:55 - 01:15 at Blended Sessions Room 3 - 2.2.3. GUI Design

UI design is an integral part of software development. For many developers who do not have much UI design experience, exposing them to a large database of real-application UI designs can help them quickly build up a realistic understanding of the design space for a software feature and get design inspirations from existing applications. However, existing keyword-based, image-similarity-based, and component-matching-based methods cannot reliably find relevant high-fidelity UI designs in a large database alike to the UI wireframe that the developers sketch, in face of the great variations in UI designs. In this article, we propose a deep-learning-based UI design search engine to fill in the gap. The key innovation of our search engine is to train a wireframe image autoencoder using a large database of real-application UI designs, without the need for labeling relevant UI designs. We implement our approach for Android UI design search, and conduct extensive experiments with artificially created relevant UI designs and human evaluation of UI design search results. Our experiments confirm the superior performance of our search engine over existing image-similarity or component-matching-based methods and demonstrate the usefulness of our search engine in real-world UI design tasks.

Conference Day
Wed 26 May

Displayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change

12:55 - 13:55
2.2.3. GUI DesignTechnical Track / Journal-First Papers at Blended Sessions Room 3 +12h
Chair(s): Ignacio PanachUniversidad de Valencia
12:55
20m
Paper
Wireframe-based UI Design Search through Image AutoencoderJournal-First
Journal-First Papers
Jieshan ChenAustralian National University, Australia, Chunyang ChenMonash University, Zhenchang XingAustralian National University, Xin XiaHuawei Software Engineering Application Technology Lab, Liming ZhuData61 at CSIRO, Australia / UNSW, Australia, John GrundyMonash University, Jinshui WangFujian University of Technology
Pre-print Media Attached
13:15
20m
Paper
GUIGAN: Learning to Generate GUI Designs Using Generative Adversarial NetworksTechnical Track
Technical Track
Tianming ZhaoJilin University, Chunyang ChenMonash University, Yuanning LiuJilin University, Xiaodong ZhuJilin University
Pre-print Media Attached
13:35
20m
Paper
Don't Do That! Hunting Down Visual Design Smells in Complex UIs against Design GuidelinesTechnical Track
Technical Track
Bo YangZhejiang University, Zhenchang XingAustralian National University, Xin XiaHuawei Software Engineering Application Technology Lab, Chunyang ChenMonash University, Deheng YeTencent AI Lab, Shanping LiZhejiang University
Pre-print Media Attached

Conference Day
Thu 27 May

Displayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change

00:55 - 01:55
00:55
20m
Paper
Wireframe-based UI Design Search through Image AutoencoderJournal-First
Journal-First Papers
Jieshan ChenAustralian National University, Australia, Chunyang ChenMonash University, Zhenchang XingAustralian National University, Xin XiaHuawei Software Engineering Application Technology Lab, Liming ZhuData61 at CSIRO, Australia / UNSW, Australia, John GrundyMonash University, Jinshui WangFujian University of Technology
Pre-print Media Attached
01:15
20m
Paper
GUIGAN: Learning to Generate GUI Designs Using Generative Adversarial NetworksTechnical Track
Technical Track
Tianming ZhaoJilin University, Chunyang ChenMonash University, Yuanning LiuJilin University, Xiaodong ZhuJilin University
Pre-print Media Attached
01:35
20m
Paper
Don't Do That! Hunting Down Visual Design Smells in Complex UIs against Design GuidelinesTechnical Track
Technical Track
Bo YangZhejiang University, Zhenchang XingAustralian National University, Xin XiaHuawei Software Engineering Application Technology Lab, Chunyang ChenMonash University, Deheng YeTencent AI Lab, Shanping LiZhejiang University
Pre-print Media Attached