Theme
Chip Technology
Topic
Hardware-friendly AI-assisted ISP
Background
Image signal processors (ISP) transform camera sensor data into images, via a significant number of digital signal processing operations, e.g., demosaicing, noise reduction, auto exposure, autofocus, auto white balance, image sharpening, and so on. Unfortunately, the quality of the images heavily depends on the algorithm designs and parameter tuning. In recent years, we have witnessed an increased interest in using deep learning (DL) to help improve the ISP, via either (1) replacing single or multiple stages by DL, or (2) automatically tuning the parameters via DL. Nonetheless, DL requires much higher computation (often 2-3 magnitude) than the traditional ISP algorithms. Therefore, we must (1) understand the trade-off between image quality improvement and computational complexity, (2) develop efficient DL-based ISP algorithms, and/or (3) design an efficient hardware architecture to support the mixture of DL-based ISP and traditional ISP pipeline.
Target
- Computationally efficient DL-based ISP algorithms
- DL-based ISP tuning algorithms
- A hardware architecture and data flow to accelerate traditional and DL-based ISP algorithms
Related Research Topics
- Trade-off study between image quality improvement and computational complexity
- DL-based ISP algorithms
- Data set for AI-assisted ISP algorithm design or tuning
- Benchmark/metrics to estimate perceptual image quality
- Network optimization techniques for image enhancement DL networks
Suggested Collaboration Method
AIR (Alibaba Innovative Research), one-year collaboration project.
