Shidiannao Shifting Vision Processing Closer To The Sensor

• doi: 10.1145/2749469.2750389
• title: ShiDianNao: Shifting vision processing closer to the sensor
• publisher: IEEE
• isbn: 978-1-5090-0255-9
• issn: 1063-6897
• rank: 3531
• access_type: LOCKED
• content_type: Conferences
• abstract: In recent years, neural network accelerators have been shown to achieve both high energy efficiency and high performance for a broad application scope within the important category of recognition and mining applications. Still, both the energy efficiency and peiformance of such accelerators remain limited by memory accesses. In this paper, we focus on image applications, arguably the most important category among recognition and mining applications. The neural networks which are state-of-the-art for these applications are Convolutional Neural Networks (CNN), and they have an important property: weights are shared among many neurons, considerably reducing the neural network memory footprint. This property allows to entirely map a CNN within an SRAM, eliminating all DRAM accesses for weights. By further hoisting this accelerator next to the image sensor, it is possible to eliminate all remaining DRAM accesses, i.e., for inputs and outputs. In this paper, we propose such a CNN accelerator, placed next to a CMOS or CCD sensor. The absence of DRAM accesses combined with a careful exploitation of the specific data access patterns within CNNs allows us to design an accelerator which is 60x more energy efficient than the previous state-of-the-art neural network accelerator. We present a fult design down to the layout at 65 nm, with a modest footprint of 4.86 mm2 and consuming only 320 mW, but still about 30x faster than high-end GPUs.
• article_number: 7284058
• pdf_url: https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=7284058
• html_url: https://ieeexplore.ieee.org/document/7284058/
• abstract_url: https://ieeexplore.ieee.org/document/7284058/
• publication_title: 2015 ACM/IEEE 42nd Annual International Symposium on Computer Architecture (ISCA)
• conference_location: Portland, OR, USA
• conference_dates: 13-17 June 2015
• publication_number: 7270878
• is_number: 7284049
• publication_year: 2015
• publication_date: 13-17 June 2015
• start_page: 92
• end_page: 104
• citing_paper_count: 396
• citing_patent_count: 26
• download_count: 8993
• insert_date: 20151001
• index_terms:
  • ieee_terms:
    • Kernel
    • Neural networks
    • Sensors
    • Energy efficiency
    • Smart phones
    • Filtering
    • Neurons
  • dynamic_index_terms:
    • Visual Processing
    • Neural Network
    • Convolutional Neural Network
    • CCD Camera
    • Charge-coupled Device
    • Energy Efficiency
    • Mining Applications
    • Category Recognition
    • Deep Neural Network
    • Convolutional Layers
    • Feature Maps
    • Input Output
    • Neurons In Layer
    • Pooling Layer
    • Normalization Layer
    • Average Layer
    • Single Neuron
    • Output Neurons
    • Classification Layer
    • Types Of Layers
    • Input Neurons
    • Output Feature Map
    • Synaptic Weights
    • Read Mode
    • Local Storage
    • Hardware Cost
    • Kernel Values
    • Maximum Operating
    • Adjacent Neurons
    • General Architecture
    • Vision Applications
    • Visual Recognition
• isbn_formats:
  • format: USB ISBN, value: 978-1-5090-0255-9, isbnType: New-2005
  • format: Electronic ISBN, value: 978-1-5090-0256-6, isbnType: New-2005
• authors:
  • Author Name: Zidong Du
    Affiliation: University of CAS, China
    Author URL: https://ieeexplore.ieee.org/author/37085367291
    ID: 37085367291
    Order: 1
    Author Affiliations:
    • University of CAS, China
    • State Key Laboratory of Computer Architecture, Institute of Computing Technology (ICT), CAS, China
  • Author Name: Robert Fasthuber
    Affiliation: EPFL, Switzerland
    Author URL: https://ieeexplore.ieee.org/author/37540520300
    ID: 37540520300
    Order: 2
    Author Affiliations:
    • EPFL, Switzerland
  • Author Name: Tianshi Chen
    Affiliation: State Key Laboratory of Computer Architecture, Institute of Computing Technology (ICT), CAS, China
    Author URL: https://ieeexplore.ieee.org/author/37402474000
    ID: 37402474000
    Order: 3
    Author Affiliations:
    • State Key Laboratory of Computer Architecture, Institute of Computing Technology (ICT), CAS, China
  • Author Name: Paolo Ienne
    Affiliation: University of CAS, China
    Author URL: https://ieeexplore.ieee.org/author/37271141000
    ID: 37271141000
    Order: 4
    Author Affiliations:
    • University of CAS, China
  • Author Name: Ling Li
    Affiliation: State Key Laboratory of Computer Architecture, Institute of Computing Technology (ICT), CAS, China
    Author URL: https://ieeexplore.ieee.org/author/37635092300
    ID: 37635092300
    Order: 5
    Author Affiliations:
    • State Key Laboratory of Computer Architecture, Institute of Computing Technology (ICT), CAS, China
  • Author Name: Tao Luo
    Affiliation: University of CAS, China
    Author URL: https://ieeexplore.ieee.org/author/38548624900
    ID: 38548624900
    Order: 6
    Author Affiliations:
    • University of CAS, China
    • State Key Laboratory of Computer Architecture, Institute of Computing Technology (ICT), CAS, China
  • Author Name: Xiaobing Feng
    Affiliation: State Key Laboratory of Computer Architecture, Institute of Computing Technology (ICT), CAS, China
    Author URL: https://ieeexplore.ieee.org/author/37632787100
    ID: 37632787100
    Order: 7
    Author Affiliations:
    • State Key Laboratory of Computer Architecture, Institute of Computing Technology (ICT), CAS, China
  • Author Name: Yunji Chen
    Affiliation: State Key Laboratory of Computer Architecture, Institute of Computing Technology (ICT), CAS, China
    Author URL: https://ieeexplore.ieee.org/author/37293328300
    ID: 37293328300
    Order: 8
    Author Affiliations:
    • State Key Laboratory of Computer Architecture, Institute of Computing Technology (ICT), CAS, China
  • Author Name: Olivier Temam
    Affiliation: Inria, France
    Author URL: https://ieeexplore.ieee.org/author/37282483400
    ID: 37282483400
    Order: 9
    Author Affiliations:
    • Inria, France

Image Sensor

• sensor_type: CMOS
• resolution: Not explicitly stated in the paper
• dynamic_range: Not explicitly stated in the paper
• pixel_size: Not explicitly stated in the paper
• dark_current: Not explicitly stated in the paper

Optical Data

• focal_length: Not explicitly stated in the paper
• aperture: Not explicitly stated in the paper
• field_of_view: Not explicitly stated in the paper
• distortion: Not explicitly stated in the paper

Performance Metrics

• frame_rate: Not explicitly stated in the paper
• signal_to_noise_ratio: Not explicitly stated in the paper
• sensitivity: Not explicitly stated in the paper
• shutter_speed: Not explicitly stated in the paper
• power_consumption: 320.10 mW
• noise: Not explicitly stated in the paper

Applications & Benefits

• cell_imaging: Not explicitly stated in the paper
• benefits: Not explicitly stated in the paper

Supporting Organizations

• supported_by: State Key Laboratory of Computer Architecture, Institute of Computing Technology (ICT), CAS, China; University of CAS, China; EPFL, Switzerland; Inria, France

Manuscript Details

• publication_date: 13-17 June 2015

Relevancy Score

• score: 7
• missing_fields:
  • fill factor
  • quantum efficiency
  • readout speed
  • noise sources
  • analog-to-digital conversion techniques
  • microlenses
  • on-chip colour filters
  • global or rolling shutters
  • backside illumination

Processed JSON Filename

request_4491ebb1-b9f5-4597-801d-dd1ad1e3aae9-shidiannao_shifting_vision_processing_closer_to_the_sensor.json