Research Themes
Other
Background
With the diversified growth of e-commerce, live broadcasting, social networking, interactive entertainment and other scenes, the development of a new generation of interactive methods such as VR / AR has attracted much attention. In recent years, the integration of computer vision based on deep learning and traditional computer graphics has greatly promoted the development of 3D reconstruction and rendering technology. The increasing maturity of related technologies allows us to explore the next generation of e-commerce beyond traditional e-commerce.
The three elements of e-commerce scene are commodity (object), scene and human. Photorealistic reconstruction and rendering of commodity can show consumer more details in all directions and angles than traditional multimedia. Arbitrary scene reconstruction can create an experience and atmosphere more in line with the shopping scene. High quality human reconstruction, which can be driven to interact with commodity and scene, greatly improves consumers' shopping interest. The above three are combined to jointly build the integration and linkage between the virtual world and the real physical world, serve the vast e-commerce industry and create the next generation of immersive shopping and sales experience for consumers and merchants.
Target
- An algorithm framework for high-quality reconstruction of human, commodity (object) and scene.
- A rendering engine supporting mobile real-time neural rendering.
- A set of algorithms for editing and generation, which can edit the shape and texture of objects, transform the style of scenes and enable multimodal human animation.
Related Research Topics
- Fast and photorealistic head / half body / full body reconstruction from RGB video in unrestricted scene.
- High quality outdoor large-scale scene reconstruction, mass production of scene digital assets suitable for mainstream rendering engines.
- Fast and high-quality reconstruction of objects in unrestricted scene, and accurately decouple the attributes of geometry, texture, material and lighting.
- High-quality new view synthesis of object, human and scene.
- Robust object editing, face/body attribute editing, scene editing and style transfer.
- Multimodal high fidelity face reenactment, talk head generation, and human body animation.
- Explore neural rendering technology to improve the quality of rendering and realize real-time rendering.