Title

Reconfigurable Processor for Deep Learning in Autonomous Vehicles

Abstract

The rapid growth of civilian vehicles has stimulated the development of advanced driver assistance systems (ADASs) to be equipped in-car. Real-time autonomous vision (RTAV) is an essential part of the overall system, and the emergence of deep learning methods has greatly improved the system quality, which also requires the processor to offer a computing speed of tera operations per second (TOPS) and a power consumption of no more than 30 W with programmability. This article gives an overview of the trends of RTAV algorithms and different hardware solutions, and proposes a development route for the reconfigurable RTAV accelerator. We propose our field programmable gate array (FPGA) based system Aristotle, together with an all-stack software-hardware co design workflow including compression, compilation, and customized hardware architecture. Evaluation shows that our FPGA system can realize real-time processing on modern RTAV algorithms with a higher efficiency than peer CPU and GPU platforms. Our outlook based on the ASIC-based system design and the ongoing implementation of next generation memory would target a 100 TOPS performance with around 20 W power.

Keywords

Advanced driver assistance system (ADAS), autonomous vehicles, computer vision, deep learning, reconfigurable processor

Authors

​Yu Wang (Department of Electronic Engineering, Tsinghua University, Beijing, China and Deephi Tech, Beijing, China) Shuang Liang (Institute of Microelectronics, Tsinghua University, Beijing, China) Song Yao (Deephi Tech, Beijing, China) Song Yao is the CEO and Co-Founder of DeePhi Tech, a startup focuses on providing more efficient deep learning platform. Before founding DeePhi Tech, he graduated from Department of Electronic Engineering, Tsinghua University. His research interest is energy efficient deep learning computing, ranges from model compression algorithm to computer architecture design.  He has received awards including FPGA 2017 Best Paper, Top 30 AI Entrepreneurs in China, and Forbes 30 Under 30 Asia.                        Yi Shan (Deephi Tech, Beijing, China) Song Han (Deephi Tech, Beijing, China and Department of Electrical Engineering, Stanford University, Stanford CA, USA) Jinzhang Peng (Deephi Tech, Beijing, China) Hong Luo (Deephi Tech, Beijing, China)