Hardware-friendly AI-assisted ISP

Theme

Chip Technology

Topic

Hardware-friendly AI-assisted ISP 

Background

Image signal processors (ISP) transform camera sensor data into images, via a significant number of digital signal processing operations, e.g., demosaicing, noise reduction, auto exposure, autofocus, auto white balance, image sharpening, and so on. Unfortunately, the quality of the images heavily depends on the algorithm designs and parameter tuning. In recent years, we have witnessed an increased interest in using deep learning (DL) to help improve the ISP, via either (1) replacing single or multiple stages by DL, or (2) automatically tuning the parameters via DL. Nonetheless, DL requires much higher computation (often 2-3 magnitude) than the traditional ISP algorithms. Therefore, we must (1) understand the trade-off between image quality improvement and computational complexity, (2) develop efficient DL-based ISP algorithms, and/or (3) design an efficient hardware architecture to support the mixture of DL-based ISP and traditional ISP pipeline. 

Target

  • Computationally efficient DL-based ISP algorithms
  • DL-based ISP tuning algorithms
  • A hardware architecture and data flow to accelerate traditional and DL-based ISP algorithms

Related Research Topics

  • Trade-off study between image quality improvement and computational complexity
  • DL-based ISP algorithms
  • Data set for AI-assisted ISP algorithm design or tuning
  • Benchmark/metrics to estimate perceptual image quality
  • Network optimization techniques for image enhancement DL networks

 

Suggested Collaboration Method

AIR (Alibaba Innovative Research), one-year collaboration project. 

 

Scan QR code
关注Ali TechnologyWechat Account