A Bioinspired Spiking Vision Chip Based On Spad Imaging And Direct Spike Computing For Versatile Edge Vision

• doi: 10.1109/JSSC.2023.3340018
• title: A Bio-Inspired Spiking Vision Chip Based on SPAD Imaging and Direct Spike Computing for Versatile Edge Vision
• publisher: IEEE
• isbn:
• issn: 1558-173X
• rank: 523
• access_type: LOCKED
• content_type: Journals
• abstract: Edge vision devices face real 3-D scenes with lighting fluctuations. They require a vision chip capable of acquiring 2-D/3-D vision with a high dynamic range and performing real-time intelligent in situ processing, while maintaining low end-to-end latency. Aimed at this versatile edge vision, this article presents a bio-inspired full-spiking vision chip that consists of a single-photon-avalanche-diode (SPAD) image sensor and a spiking vision processor monolithically. To decrease latency while realizing versatile functions, we build a bio-inspired full spiking vision on chip by directly coupling spike-like SPAD imaging with a neuroinspired spike-computing paradigm. By modeling this spiking visual flow, we theoretically demonstrate direct coupling and derive a spike preprocessing method for dim-vision enhancement. The SPAD image sensor equipped with a configurable exposure gating scheme can realize three modes: 2-D, 3-D depth, and dim-vision imaging. The spiking vision processor contains a parallel processing element (PE) array and can be reconfigured as a preprocessor or a spiking neural network (SNN) processor. A prototype chip integrated with a $128 \,\, \times \,\, 128$ SPAD image sensor with a reconfigurable spiking vision processor (256 PEs) was implemented in a 0.18- $\mu \text{m}$ CMOS technology. The chip experimentally demonstrates three imaging modes, 99.30% classification accuracy on the MNIST dataset with 2-D vision, only 3.9% accuracy loss under 0.02 lux with dim-vision, 1.68 cm obstacle detection error with 3-D depth vision, and a 3.58 $\mu \text{s}$ latency for light adaptation. The chip realizes a 100 dB dynamic range, 100 000 spiking maps per second (SMps), and 81.92 GSOPS at an 80-MHz clock.
• article_number: 10375547
• pdf_url: https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10375547
• html_url: https://ieeexplore.ieee.org/document/10375547/
• abstract_url: https://ieeexplore.ieee.org/document/10375547/
• publication_title: IEEE Journal of Solid-State Circuits
• conference_location:
• conference_dates:
• publication_number: 4
• is_number: 10540332
• publication_year: 2024
• publication_date: June 2024
• start_page: 1883
• end_page: 1898
• citing_paper_count: 3
• citing_patent_count: 0
• download_count: 1932
• insert_date: 20231228
• index_terms:
  • ieee_terms:
    • Imaging
    • Single-photon avalanche diodes
    • Image sensors
    • Visualization
    • Image edge detection
    • High dynamic range
    • Parallel processing
  • author_terms:
    • Adaptive edge applications
    • single-photon-avalanche-diode (SPAD) imaging
    • spike-computing
    • spiking neural networks (SNNs)
    • vision chip
  • dynamic_index_terms:
    • Vision Chips
    • Neural Network
    • Classification Accuracy
    • Dynamic Range
    • Imaging Modalities
    • Image Sensor
    • Camera Sensor
    • Direct Coupling
    • Spiking Neural Networks
    • Spiking Neurons
    • High Dynamic Range
    • Obstacle Avoidance
    • Obstacle Detection
    • MNIST Dataset
    • Light Adaptation
    • Full Spike
    • Light Conditions
    • 3D Images
    • Visual Information
    • Rich Information
    • Information-rich
    • Visual Task
    • Object Classification
    • Depth Images
    • High Dynamic Range Image
    • Synapse Weights
    • Spike Data
    • Pixel Array
    • Local Registration
    • Clock Frequency
    • Data Sharing
    • Neural Network Approximation
    • Neural Network Computations
    • Neural Network Estimation
    • Input Spike
• authors:
  • Author Name: Xu Yang
    Affiliation: State Key Laboratory of Superlattices and Microstructures, Institute of Semiconductors, Chinese Academy of Sciences, Beijing, China
    Author URL: https://ieeexplore.ieee.org/author/37089043514
    ID: 37089043514
    Order: 1
    Author Affiliations:
    • State Key Laboratory of Superlattices and Microstructures, Institute of Semiconductors, Chinese Academy of Sciences, Beijing, China
  • Author Name: Chunhe Yao
    Affiliation: State Key Laboratory of Superlattices and Microstructures, Institute of Semiconductors, Chinese Academy of Sciences, Beijing, China
    Author URL: https://ieeexplore.ieee.org/author/37088215779
    ID: 37088215779
    Order: 2
    Author Affiliations:
    • State Key Laboratory of Superlattices and Microstructures, Institute of Semiconductors, Chinese Academy of Sciences, Beijing, China
  • Author Name: Lei Kang
    Affiliation: State Key Laboratory of Superlattices and Microstructures, Institute of Semiconductors, Chinese Academy of Sciences, Beijing, China
    Author URL: https://ieeexplore.ieee.org/author/37089625144
    ID: 37089625144
    Order: 3
    Author Affiliations:
    • State Key Laboratory of Superlattices and Microstructures, Institute of Semiconductors, Chinese Academy of Sciences, Beijing, China
  • Author Name: Qian Luo
    Affiliation: State Key Laboratory of Superlattices and Microstructures, Institute of Semiconductors, Chinese Academy of Sciences, Beijing, China
    Author URL: https://ieeexplore.ieee.org/author/37088216568
    ID: 37088216568
    Order: 4
    Author Affiliations:
    • State Key Laboratory of Superlattices and Microstructures, Institute of Semiconductors, Chinese Academy of Sciences, Beijing, China
  • Author Name: Nan Qi
    Affiliation: State Key Laboratory of Superlattices and Microstructures, Institute of Semiconductors, Chinese Academy of Sciences, Beijing, China
    Author URL: https://ieeexplore.ieee.org/author/37085494119
    ID: 37085494119
    Order: 5
    Author Affiliations:
    • State Key Laboratory of Superlattices and Microstructures, Institute of Semiconductors, Chinese Academy of Sciences, Beijing, China
    • Center of Materials Science and Optoelectronics Engineering, University of Chinese Academy of Sciences, Beijing, China
  • Author Name: Runjiang Dou
    Affiliation: State Key Laboratory of Superlattices and Microstructures, Institute of Semiconductors, Chinese Academy of Sciences, Beijing, China
    Author URL: https://ieeexplore.ieee.org/author/37086070202
    ID: 37086070202
    Order: 6
    Author Affiliations:
    • State Key Laboratory of Superlattices and Microstructures, Institute of Semiconductors, Chinese Academy of Sciences, Beijing, China
  • Author Name: Shuangming Yu
    Affiliation: State Key Laboratory of Superlattices and Microstructures, Institute of Semiconductors, Chinese Academy of Sciences, Beijing, China
    Author URL: https://ieeexplore.ieee.org/author/37088212043
    ID: 37088212043
    Order: 7
    Author Affiliations:
    • State Key Laboratory of Superlattices and Microstructures, Institute of Semiconductors, Chinese Academy of Sciences, Beijing, China
  • Author Name: Peng Feng
    Affiliation: State Key Laboratory of Superlattices and Microstructures, Institute of Semiconductors, Chinese Academy of Sciences, Beijing, China
    Author URL: https://ieeexplore.ieee.org/author/267564703089543
    ID: 267564703089543
    Order: 8
    Author Affiliations:
    • State Key Laboratory of Superlattices and Microstructures, Institute of Semiconductors, Chinese Academy of Sciences, Beijing, China
    • Center of Materials Science and Optoelectronics Engineering, University of Chinese Academy of Sciences, Beijing, China
  • Author Name: Zhongming Wei
    Affiliation: State Key Laboratory of Superlattices and Microstructures, Institute of Semiconductors, Chinese Academy of Sciences, Beijing, China
    Author URL: https://ieeexplore.ieee.org/author/37086931656
    ID: 37086931656
    Order: 9
    Author Affiliations:
    • State Key Laboratory of Superlattices and Microstructures, Institute of Semiconductors, Chinese Academy of Sciences, Beijing, China
  • Author Name: Jian Liu
    Affiliation: State Key Laboratory of Superlattices and Microstructures, Institute of Semiconductors, Chinese Academy of Sciences, Beijing, China
    Author URL: https://ieeexplore.ieee.org/author/37085851169
    ID: 37085851169
    Order: 10
    Author Affiliations:
    • State Key Laboratory of Superlattices and Microstructures, Institute of Semiconductors, Chinese Academy of Sciences, Beijing, China
    • Center of Materials Science and Optoelectronics Engineering, University of Chinese Academy of Sciences, Beijing, China
  • Author Name: Kaiyou Wang
    Affiliation: State Key Laboratory of Superlattices and Microstructures, Institute of Semiconductors, Chinese Academy of Sciences, Beijing, China
    Author URL: https://ieeexplore.ieee.org/author/37089322939
    ID: 37089322939
    Order: 11
    Author Affiliations:
    • State Key Laboratory of Superlattices and Microstructures, Institute of Semiconductors, Chinese Academy of Sciences, Beijing, China
  • Author Name: Nanjian Wu
    Affiliation: Center of Materials Science and Optoelectronics Engineering, University of Chinese Academy of Sciences, Beijing, China
    Author URL: https://ieeexplore.ieee.org/author/37286754000
    ID: 37286754000
    Order: 12
    Author Affiliations:
    • Center of Materials Science and Optoelectronics Engineering, University of Chinese Academy of Sciences, Beijing, China
    • State Key Laboratory of Superlattices and Microstructures, Institute of Semiconductors, and the Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Beijing, China
  • Author Name: Liyuan Liu
    Affiliation: State Key Laboratory of Superlattices and Microstructures, Institute of Semiconductors, Chinese Academy of Sciences, Beijing, China
    Author URL: https://ieeexplore.ieee.org/author/37085745851
    ID: 37085745851
    Order: 13
    Author Affiliations:
    • State Key Laboratory of Superlattices and Microstructures, Institute of Semiconductors, Chinese Academy of Sciences, Beijing, China
    • Center of Materials Science and Optoelectronics Engineering, University of Chinese Academy of Sciences, Beijing, China

Image Sensor

• sensor_type: Single-Photon-Avalanche-Diode (SPAD)
• resolution: 128x128 pixels
• dynamic_range: 100 dB
• pixel_size: 10 µm
• dark_current: not specified in the text

Optical Data

• focal_length: not specified in the text
• aperture: not specified in the text
• field_of_view: not specified in the text
• distortion: not specified in the text

Performance Metrics

• frame_rate: 100,000 spiking maps per second
• signal_to_noise_ratio: not explicitly provided
• sensitivity: not specified in the text
• shutter_speed: not specified in the text
• power_consumption: 922 mW in 2-D imaging mode, 875 mW in dim-vision mode, and 913 mW in 3-D imaging mode
• noise: not specified in the text

Applications & Benefits

• cell_imaging: not specified in the text
• benefits: Versatile functionalities for 2-D, 3-D, and dim vision, instant light-adaptation

Supporting Organizations

• supported_by: National Key Research and Development Program of China, National Natural Science Foundation of China, Beijing Natural Science Foundation, Beijing Municipal Science and Technology, Strategic Priority Research Program of the Chinese Academy of Science, and Youth Innovation Promotion Association Program Chinese Academy of Sciences.

Manuscript Details

• publication_date: June 2024

Relevancy Score

• score: 10

Processed JSON Filename

request_592b7646-5e92-48cc-9085-44777db38684-a_bioinspired_spiking_vision_chip_based_on_spad_imaging_and_direct_spike_computing_for_versatile_edge_vision.json