- Multi-modal Big Data Analysis
Research in this area focuses on comprehensively analyzing pedestrians, vehicles, and incidents from various dimensions based on a fusion of multi-view learning with multi-source heterogeneous data.
- Traffic Forecasting and Intervention
Research in this area focuses on analyzing, forecasting, and intelligently intervening in urban traffic to mitigate congestion based on large-scale road networks.
- Large-scale Parallel Heterogeneous Computing
Research in this area focuses on leveraging parallel heterogeneous computing technologies to process large amounts of real-time data on heterogeneous networks at high speed.
- Environment Analysis and Understanding
Research in this area focuses on creating effective perception models and designing robust, adaptive computer vision algorithms to better analyze and understand complex urban environments.
- Visual Search Engine
Research in this area focuses on modeling the characteristics of pedestrians and their behavior based on dynamic video data to improve search and recognition accuracies.
- Urban Planning and Public Resource Analysis
Research in this area focuses on intelligently analyzing and making decisions on the layout of urban infrastructure and the allocation of public resources based on insights into big data and urban development patterns.
City Visual Intelligence EngineLearn moreCity Visual Intelligence Engine (CVIE) is powered by the distributed computing and storage platform of Alibaba Cloud. It leverages a wide range of advanced video image processing, computer graphics processing, and deep learning algorithms to create city-scale artificial intelligence models. These models help access, compute, analyze, index, and mine values from city-wide video data. Based on the computational and analysis results, CVIE provides insights into municipal affairs such as public security, transportation, urban governance, business development, law enforcement, office park, power and energy supply, medical care, and education.
City Brain "Tianqing"Tianqing is a large-scale security-oriented visual computing platform that consists of three components: a video access system, a stream/batch computing system, and a visual search system. Tianqing supports rapid, scalable deployment on the cloud to provide on-demand smart analysis capabilities. Tianqing accelerates video data analysis by up to 1,000 times and takes only 1 minute to process 16 hours of video data.
City Brain “Tianyao”Tianyao is a smart 24/7 global traffic patrol and alert system that covers every corner of the city where it is deployed. Tianyao monitors and analyzes urban traffic conditions in real time. Based on the monitoring and analysis results, Tianyao detects unusual pedestrians, vehicles, environments, and incidents to identify traffic accidents and violations with more than 95% accuracy. Tianyao then pushes alerts to the traffic command center within 20 seconds. Tianyao reduces the frequency that traffic police require to perform patrols and traffic analysis. This reduces the safety risks of traffic police and increases law enforcement efficiency.
City Brain “Tianji”Tianji is a vehicle and pedestrian traffic flow forecasting system. It accurately predicts future vehicle and pedestrian traffic flows by analyzing both historical and real-time video data. The prediction results serve as a reference for decision-making on traffic control to avoid incidents, such as congestion and pedestrian injuries. Tianji can predict next-hour vehicle and pedestrian traffic flows with more than 90% accuracy.
City Brain “Tianying”Tianying is a progressive video search engine that can quickly locate specific objects by searching city-wide videos in real time. For example, Tianying requires only 1 second to 2 seconds to track missing persons or hit-and-run suspects, with more than 96% accuracy.
- City Brain “Tianjing”
Tianjing is a smart municipal construction and management system. Tianjing monitors the municipal projects that are run by government agencies such as urban governance, security monitoring, fire prevention and control, emergency management, housing construction, law enforcement, and environment protection. Tianjing eradicates hidden risks in municipal construction together with patrols to make the city where it is deployed a safer place to live.
- City Brain Tianpu
Tianpu produces static three-dimensional images of the city where it is deployed and updates these images based on real-time data. This allows Tianpu to build a spatio-temporal digital world that is parallel to the city in the real world. The digital world provides all the spatio-temporal data that is needed for fine-grained municipal management, smart forecasting, and smart alerting.
City Solutions- Hangzhou:
City Brain systems achieve a recognition accuracy rate of more than 92% in video inspection. The systems are connected to a network of traffic lights to automate traffic control and management. With the help of City Brain systems, the transportation speed is increased by 15%, and route optimization for special vehicles, such as ambulances and fire trucks, is increased by 50%. Also, the average forecast deviation for passenger traffic at the entrances and exits of subway stations at a granularity of 10 minutes is lower than 15 persons.
- Beijing:
City Brain systems are deployed in Tongzhou District and Daxing District to implement 24/7 automatic identification of environmental issues, such as uncovered construction sites, uncovered muck trucks, and road debris. City Brain systems also implement real-time sensing of indicators related to urban management measures in Xicheng District, including ecological and environmental protection, safety assurance, fine-grained governance, and comprehensive law enforcement. All these measures ensure the safety of Xicheng District during festivals.
- Suzhou:
City Brain systems optimize public transportation based on comprehensive data analysis. The passenger traffic for two pilot bus lines is increased by 17% and 10%. This marks an improvement in offloading passengers from buses.
- Shanghai:
City Brain systems implement digital processing and analysis on urban infrastructure, such as transportation, energy supply, water supply, and construction facilities, to provide support for smart decision-making on public transportation and public services.
- Xiong'an New Area:
The City Brain Lab, in cooperation with the government of Xiong'an New Area, has built a smart city, which adopts a cloud computing-based infrastructure, an IoT-based urban neural network, and a City Brain-based artificial intelligence hub.
- Chengdu:
The City Brain Tianyao system is deployed to Sichuan expressways. Tianyao implements multi-modal fusion on the video data, the data on the big data platform of Sichuan Expressway Construction & Development Group Co Ltd., and the real-time data of AutoNavi's Internet business. With the help of Tianyao, traffic congestion on main roads is reduced by 30%, and accidents are reduced by 20%. Also, the incident handling speed is increased by 50%, the efficiency of combating toll evasion is increased by 10%, and the operational efficiency is increased by 20%.
- Zhengzhou:
City Brain systems adopt an industry-leading edge cloud-based visual AI solution to build a visually intelligent computing and scheduling platform. The systems implement cognition and understanding of videos and images, and use inductive inference algorithms to provide intelligent applications for various government departments, such as transportation, political and legal affairs, emergency management, urban administrative and law enforcement, environmental protection, and fire prevention and control departments.
- Haikou:
City Brain systems are deployed to island entrances and exits, such as ports, and analyze multi-modal data to provide early warning of problems. The systems allow government departments to perform multi-service data fusion and push alerts. The systems also assist executive bodies in dealing with illegal acts and implementing collaborative command and scheduling.
- Quzhou:
City Brain systems are the core of the Quzhou Xueliang project. During the first two-month period after City Brain systems were launched, close to 10,000 incidents of public wrongdoing were captured and exposed. These incidents include wrong-way driving of electric motorcycles, jaywalking, and trespassing. Public wrongdoing is reduced by 36% one month after such exposure. The implementation is extended to other cities, such as Chengdu, Jiujiang, and Shaoxing.
- China Southern Power Grid and State Grid
City Brain systems implement visual identification of faults, defects, and exceptions for UAV inspection on power transmission corridors and capital constructions. This increases the efficiency of grid inspection and decreases the time consumed and costs of labor.
- Towards Precise Intra-camera Supervised Person Re-identification. Menglin Wang, Baisheng Lai, Haokun Chen, Jianqiang Huang, Xiaojin Gong, Xian-Sheng Hua, WACV, 2021.
- MaCAR: Urban Traffic Light Control via Active Multi-agent Communication and Action Rectification. Zhengxu Yu, Shuxian Liang, Long Wei, Zhongming Jin, Jianqiang Huang, Deng Cai, Xiaofei He, Xian-Sheng Hua. IJCAI, 2020.
- Adversarial Mutual Information for Text Generation. Boyuan Pan, Yazheng Yang, Kaizhao Liang, Bhavya Kailkhura, Zhongming Jin, Xian-Sheng Hua, Deng Cai, Bo Li . ICML, 2020.
- PCPL: Predicate-Correlation Perception Learning for Unbiased Scene Graph Generation. Shaotian Yan, Chen Shen, Zhongming Jin, Jianqiang Huang, Rongxin Jiang, Yaowu Chen, Xian-Sheng Hua. ACM Multimedia, 2020.
- Spatial-Temporal Inception Graph Convolutional Neural Networks for Skeleton-based Action Recognition. Zhen Huang, Xu Shen, Xinmei Tian, Houqiang Li, Jianqiang Huang, Xiansheng Hua. ACM Multimedia, 2020.
- Self-Adaptive Neural Module Transformer for Visual Question Answering. Huasong Zhong, Jingyuan Chen, Chen Shen, Hanwang Zhang, Jianqiang Huang, Xian-Sheng Hua. IEEE Transactions on Multimedia, 2020.
- Hongwei Yong, Jianqiang Huang, Wangmeng Xiang, Xiansheng Hua, Lei Zhang. Panoramic Background Image Generation for PTZ Cameras. IEEE Transactions on Image Processing, 2019(99):1-1.
- Zhihang Fu, Yaowu Chen, Hongwei Yong, Rongxin Jiang, Lei Zhang, Xian-Sheng Hua. Foreground Gating and Background Refining Network for Surveillance Object Detection[J]. IEEE Transactions on Image Processing, 2019.
- Long Wei, Zhenyong Wei, Zhongming Jin, Zhengxu Yu, Jianqiang Huang, Deng Cai, Xiaofei He, Xian-Sheng Hua. SIF: Self-Inspirited Feature Learning for Person Re-identication. IEEE Transactions on Image Processing, ACCEPT.
- Long Wei,Zhenyong Wei,Zhongming Jin,Qianxiao Wei,Jianqiang Huang,Xian-Sheng Hua,Deng Cai,Xiaofei He. Decouple co-adaptation: Classifier randomization for person re-identification[J]. Neurocomputing,2020,383.
- Long Wei*, Zhengxu Yu*, Zhongming Jin, Liang Xie, Jianqiang Huang, Deng Cai, Xiaofei He, and Xian-Sheng Hua. "Dual Graph for Traffic Forecasting." IEEE Access (2019).
- Jiali Xi, Qin Zhou, Yiru Zhao, Shibao Zheng. Fine-Grained Fusion With Distractor Suppression for Video-Based Person Re-Identification. IEEE Access 2019.
- Chen Shen, Zhongming Jin, Wenqing Chu, Guojun Qi, Xian-Sheng Hua. Multi-level Similarity Perception Network for Person Re-identification. ACM TOMM 2019.
- Yiru Zhao, Xu Shen, Zhongming Jin, Hongtao Lu, Xiansheng Hua. Attribute-Driven Feature Disentangling and Temporal Aggregation for Video Person Re-Identification. CVPR 2019.
- Jiwei Yang, Xu Shen, Jun Xing, Xinmei Tian, Houqiang Li, Bing Deng, Jianqiang Huang, Xiansheng Hua. Quantization Networks. CVPR 2019.
- Jiaxin Shi, Hanwang Zhang, Juanzi Li. Explainable and Explicit Visual Reasoning over Scene Graphs. CVPR, 2019.
- Yulei Niu, Hanwang Zhang, et al. Recursive Visual Attention in Visual Dialog. CVPR, 2019.
- Xu Yang, Kaihua Tang, Hanwang Zhang, Jianfei Cai. Auto-Encoding Scene Graphs for Image Captioning. CVPR, 2019.
- Yunke Zhang, Lixue Gong, Weiwei Xu, Lubin Fan, Peiran Ren. A Late Fusion CNN for Digital Matting. CVPR, 2019.
- Yuan Yao, Jianqiang Ren, Xuansong Xie, Weidong Liu, Yong-Jin Liu锛Jun Wang. Attention-aware Multi-stroke Style Transfer. CVPR, 2019.
- Hui Zeng, Lida Li, Zisheng Cao, Lei Zhang. Reliable and Efficient Image Cropping: A Grid Anchor based Approachs. CVPR, 2019.
- Xixi Jia, Sanyang Liu, Xiagnchu Feng, Lei Zhang. FOCNet: A Fractional Optimal Control Network for Image Denoising. CVPR, 2019.
- Kai Zhang, Wangmeng Zuo, Lei Zhang. Deep Plug-and-Play Super-Resolution for Arbitrary Blur Kernels. CVPR, 2019.
- Shi Guo, Wangmeng Zuo, Zifei Yan, Kai Zhang, Lei Zhang. Toward Convolutional Blind Denoising of Real-world Noisy Photographs. CVPR, 2019.
- Tao Dai, Jianrui Cai, Yongbing Zhang, Shutao Xia, Lei Zhang. Second-order Attention Network for Single Image Super-resolution. CVPR, 2019.
- Jingyuan Chen, Lin Ma, Xinpeng Chen, Zequn Jie, Jiebo Luo. Localizing Natural Language in Videos via Boundary Pointer. AAAI, 2019.
- Zhengxu Yu, Zhongming Jin, Long Wei, Jishun Guo, Jianqiang Huang, Deng Cai, Xiaofei He, Xian-Sheng Hua. Progressive Transfer Learning for Person Re-identification. IJCAI 2019.
- Xinzhe Li, Qianru Sun, Yaoyao Liu, Shibao Zheng, Qin Zhou, Tat-Seng Chua, Bernt Schiele. Learning to Self-Train for Semi-Supervised Few-Shot Classification. NeurIPS 2019.
- Zhigang Chang, Qin Zhou, Mingyang Yu, Shibao Zheng, Hua Yang, Tai-Pang Wu. Distribution Context Aware Loss for Person Re-identification. VCIP 2019.
- Mingyang Yu, Zhigang Chang, Qin Zhou, Shibao Zheng, Tai-Pang Wu. "Reference-oriented Loss for Person Re-identification". IJCNN 2019.
- Yiru Zhao, Zhongming Jin, Guojun Qi, Hongtao Lu, Xiansheng Hua. An Adversarial Approach to Hard Triplet Generation. ECCV, 2018.
- Sijia Cai, Wangmeng Zuo, Larry Davis, Lei Zhang. Weakly-supervised Video Summarization using Variational Inference and Web Prior. ECCV, 2018.
- Zhihang Fu, Zhongming Jin, Guojun Qi, Chen Shen, Rongxin Jiang, Yaowu Chen, Xiansheng Hua. Previewer for Multiple-Scale Object Detector. ACM Multimedia, 2018.
- Jiwei Yang, Xu Shen, Xinmei Tian, Jianqiang Huang, Houqiang Li, Xiansheng Hua. Local Convolutional Neural Networks for Person Re-Identification. ACM Multimedia, 2018.
- Xie G T, Wang J D, Zhang Ting, et al. Interleaved Structured Sparse Convolutional Neural Networks [C]. CVPR, 2018.
- Qi G J, Zhang L, Hu H, et al. Global versus Localized Generative Adversarial Nets[C]. CVPR, 2018.
- Zhang K, Zuo W, Zhang L. Learning a Single Convolutional Super-Resolution Network for Multiple Degradations[C]. CVPR, 2018.
- Yang J W, Shen X, Tian X M, et al. Local Convolutional Neural Networks for Person Re-Identification[C]. ACM on Multimedia Conference, 2018.
- Fu Z H, Jin Z M, Qi G J, et al. Previewer for Multiple-Scale Object Detector[C]. ACM on Multimedia Conference, 2018.
- Cai S J, Zuo W M, Davis L, el al. Weakly-supervised Video Summarization using Variational Inference and Web Prior[C]. European Conference on Computer Vision, 2018.
- Zhao Y R, Jin Z M, Qi G J, et al.An Adversarial Approach to Hard Triplet Generation[C]. European Conference on Computer Vision, 2018.
- Liu Y F, Jaw D W, Huang S C, et al. DesnowNet: Context-Aware Deep Network for Snow Removal[J]. IEEE Transactions on Image Processing, 2018, 27(6): 3064-3073.
- Chu W, Liu Y, Shen C, et al. Multi-Task Vehicle Detection With Region-of-Interest Voting[J]. IEEE Transactions on Image Processing, 2018, 27(1): 432-441.
- Shen C, Jin Z, Zhao Y, et al. Deep Siamese Network with Multi-level Similarity Perception for Person Re-identification[C]. Proceedings of the 2017 ACM on Multimedia Conference. ACM, 2017: 1942-1950.
- Zhao Y, Deng B, Shen C, et al. Spatio-Temporal AutoEncoder for Video Anomaly Detection[C]. Proceedings of the 2017 ACM on Multimedia Conference. ACM, 2017: 1933-1941.
- Zhao Y, Deng B, Huang J, et al. Stylized Adversarial AutoEncoder for Image Generation[C]. Proceedings of the 2017 ACM on Multimedia Conference. ACM, 2017: 244-251.
- Champion of the second Visual Dialogue challenge at CVPR 2019
- Champion of the Weakly-supervised Semantic Segmentation challenge in the first Learning from Imperfect Data workshop at CVPR 2019
- Champion of the pedestrian detection at KITTI in January 2018
- Champion of the vehicle detection at KITTI in May 2017
