Generative Adversarial Networks for Image-Based UI Code Generation

Theme

Image and Video

Topic

UI codes generation based on user intentions

 

Background

To increase the efficiency of development in front-end industry, it is expected to generate UI codes automatically from design document. With this automatic process, the development cycle will significantly be shortened. The time spent on one module will be approximately reduced from 3 weeks to 1 week. Specifically, it will only take 5 minutes to write GUI codes with this technique. 

 

In the meantime, it has been proved that if we can provide pages with personalized design with different user interfaces, layouts and contents according to user’s intentions, the user experience will be improved a lot, which results in longer stay time, page visits, GMV etc. If we can use models to automatically generated this kind of raw page information and use technique above to give out codes, the goal can be achieved.

 

 

 

 

We can divide the user intentions to several sub-modules, including spending power, gender, stay time, scroll speed, gesture and etc. From these intentions, we will use models such as GAN or GNN to generate pages, including different combinations of UI components, contents, layouts. 

 

 

 

Target

  • Develop a neural network models to generate UI automatically

  • Develop a generative network to generate UI components, layouts and

    contents

  • The model can perform from specific user intentions

  • The model could achieve high accuracy and good research results (paper,

    patent)

  • The model could help with increment of GMV, DAU, etc.

  • Increase influence of Alibaba front-end team in the industry. 

Related Research Topics

 

  • Generative adversarial network

  • Automatic UI generation

  • Generating code from a graphical user interface screenshot

  • Program codes generation on deep learning 

 

Suggested Collaboration Method

AIR (Alibaba Innovative Research), one-year collaboration project. 

 

Scan QR code
关注Ali TechnologyWechat Account