Image and Video
Efficient facial and body landmark detection and segmentation with prior co-location knowledge and its application in avatar action generation and augmented reality
3D avatar system becomes increasingly important for live video streaming, augmented reality, virtual lecturer and customer service. Its success depends on a variety of computer vision and computer graphic techniques, some of which are unready for industrial applications or not even being considered in research communities. Hence, we propose this university collaboration for boosting the development of the key techniques that can help realize a successful industry-level 3D avatar system.
Develop techniques that are crucial for 3D avatar model generation, and its lip, facial and limb animation generation. There are three key components in a successful 3D avatar system: 3D avatar model generation, video to animation and speech to animation. The first component helps create 3D avatar mode automatically given an input photo. The second component allow a person to control avatar’s action via his or her own action. The third component helps generate avatar’s animation based on speeches.
We are looking for university collaborators who could contribute to any part of the aforementioned comments including but not limited to facial or body landmark detection, segmentation, speech to lip/facial/limb animation generation, 3D face reconstruction, human action synthesis, motion retargeting.
- Develop the state-of-the-art techniques that can be integrated into real applications.
- Publish high-quality papers in top-notch internal conferences or journals.
- Participate international challenges for the related techniques, e.g., ECCV, ICCV or CVPR challenge workshops for 3D face reconstruction, landmark detection, image or video segmentation.
Related Research Topics
- Facial and body landmark detection and pose estimation
- Image and video segmentation
- Speech to action generation
- Speech to lip animation generation
- 3D face reconstruction and facial expression estimation
- Light-weight CNN model development and model compression
Suggested Collaboration Method
AIR (Alibaba Innovative Research), one-year collaboration project.