A 05V Realtime Computational Cmos Image Sensor With Programmable Kernel For Feature Extraction

• doi: 10.1109/JSSC.2020.3034192
• title: A 0.5-V Real-Time Computational CMOS Image Sensor With Programmable Kernel for Feature Extraction
• publisher: IEEE
• isbn:
• issn: 1558-173X
• rank: 158
• access_type: LOCKED
• content_type: Journals
• abstract: As the growing demand on artificial intelligence (AI) Internet-of-Things (IoT) devices, smart vision sensors with energy-efficient computing capability are required. This article presents a low-power and low-voltage dual mode 0.5-V computational CMOS image sensor (C2IS) with array-parallel computing capability for feature extraction using convolution. In the feature extraction mode, by applying the pulsewidth modulation (PWM) pixel and switch-current integration (SCI) circuit, the in-sensor eight-directional matrix-parallel multiply-accumulate (MAC) operation is realized. Furthermore, the analog-domain convolution-on-readout (COR) operation, the programmable 3×3 kernel with ±3-bit weights, and the tunable-resolution column-parallel analog-to-digital converter (ADC) (1-8 bit) are implemented to achieve the real-time feature extraction without using additional memory and sacrificing frame rate. In the image capturing mode, the sensor provides the linear-response 8-bit raw image data. The C2IS prototype has been fabricated in the TSMC 0.18-μm standard process technology and verified to demonstrate the raw and feature images at 480 frames/s with a power consumption of 77/ 117 μW and the resultant FoM of 9.8/14.8 pJ/pixel/frame, respectively. The prototype sensor is used as a real-time edge feature detection frond-end camera and accompanied with a simplified convolutional neural network (CNN) architecture to demonstrate the hand gesture recognition. The prototype system achieves more than 95% validation accuracy.
• article_number: 9250500
• pdf_url: https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9250500
• html_url: https://ieeexplore.ieee.org/document/9250500/
• abstract_url: https://ieeexplore.ieee.org/document/9250500/
• publication_title: IEEE Journal of Solid-State Circuits
• conference_location:
• conference_dates:
• publication_number: 4
• is_number: 9411776
• publication_year: 2021
• publication_date: May 2021
• start_page: 1588
• end_page: 1596
• citing_paper_count: 52
• citing_patent_count: 0
• download_count: 4063
• insert_date: 20201106
• index_terms:
  • ieee_terms:
    • Convolution
    • Pulse width modulation
    • Feature extraction
    • Partial discharges
    • Kernel
    • Real-time systems
    • Prototypes
  • author_terms:
    • Artificial intelligence (AI)
    • computational CMOS image sensor (C²IS)
    • convolution
    • feature extraction
    • multiply–accumulate (MAC)
    • processing-in-sensor (PIS)
    • pulsewidth modulation (PWM)
    • switch-current integration (SCI)
  • dynamic_index_terms:
    • Image Sensor
    • Camera Sensor
    • Computer Image
    • Image Calculator
    • Real-time Estimation
    • Real-time Computation
    • Real-time Calculation
    • Raw Data
    • Convolutional Neural Network
    • Image Features
    • Imaging Characteristics
    • Pulse Width
    • Pulsewidth Modulation
    • Pulsewidth
    • Pulse Width Modulation
    • Frame Rate
    • Raw Images
    • Analog-to-digital Converter
    • Analog-to-digital
    • Digital-to-analog Converter
    • Validation Accuracy
    • Gesture Recognition
    • Raw Image Data
    • Additional Memory
    • Sensor Prototype
    • Hand Gesture Recognition
    • Deep Neural Network
    • Energy Efficiency
    • Polarizability
    • Convolution Operation
    • Convolution Results
    • Application Of Convolutional Neural Networks
    • Image Convolution
    • Mode Calculations
    • Computation Mode
    • Image Recognition
    • Power Constraint
    • Considerable Power
    • Convolutional Neural Network Model
    • Intermediate Storage
• authors:
  • Author Name: Tzu-Hsiang Hsu
    Affiliation: Department of Electrical Engineering, National Tsing Hua University, Hsinchu, Taiwan
    Author URL: https://ieeexplore.ieee.org/author/37086335074
    ID: 37086335074
    Order: 1
    Author Affiliations:
    • Department of Electrical Engineering, National Tsing Hua University, Hsinchu, Taiwan
  • Author Name: Yi-Ren Chen
    Affiliation: Department of Electrical Engineering, National Tsing Hua University, Hsinchu, Taiwan
    Author URL: https://ieeexplore.ieee.org/author/37088358977
    ID: 37088358977
    Order: 2
    Author Affiliations:
    • Department of Electrical Engineering, National Tsing Hua University, Hsinchu, Taiwan
  • Author Name: Ren-Shuo Liu
    Affiliation: Department of Electrical Engineering, National Tsing Hua University, Hsinchu, Taiwan
    Author URL: https://ieeexplore.ieee.org/author/37086262728
    ID: 37086262728
    Order: 3
    Author Affiliations:
    • Department of Electrical Engineering, National Tsing Hua University, Hsinchu, Taiwan
  • Author Name: Chung-Chuan Lo
    Affiliation: Institute of Systems Neuroscience, National Tsing Hua University, Hsinchu, Taiwan
    Author URL: https://ieeexplore.ieee.org/author/37728954000
    ID: 37728954000
    Order: 4
    Author Affiliations:
    • Institute of Systems Neuroscience, National Tsing Hua University, Hsinchu, Taiwan
  • Author Name: Kea-Tiong Tang
    Affiliation: Department of Electrical Engineering, National Tsing Hua University, Hsinchu, Taiwan
    Author URL: https://ieeexplore.ieee.org/author/37290163100
    ID: 37290163100
    Order: 5
    Author Affiliations:
    • Department of Electrical Engineering, National Tsing Hua University, Hsinchu, Taiwan
  • Author Name: Meng-Fan Chang
    Affiliation: Department of Electrical Engineering, National Tsing Hua University, Hsinchu, Taiwan
    Author URL: https://ieeexplore.ieee.org/author/37420875800
    ID: 37420875800
    Order: 6
    Author Affiliations:
    • Department of Electrical Engineering, National Tsing Hua University, Hsinchu, Taiwan
  • Author Name: Chih-Cheng Hsieh
    Affiliation: Department of Electrical Engineering, National Tsing Hua University, Hsinchu, Taiwan
    Author URL: https://ieeexplore.ieee.org/author/37365236900
    ID: 37365236900
    Order: 7
    Author Affiliations:
    • Department of Electrical Engineering, National Tsing Hua University, Hsinchu, Taiwan

Image Sensor

• sensor_type: CMOS
• resolution: 128x128
• dynamic_range: 52.3 dB
• pixel_size: Not specified
• dark_current: Not specified

Optical Data

• focal_length: Not specified
• aperture: Not specified
• field_of_view: Not specified
• distortion: Not specified

Performance Metrics

• frame_rate: 480 fps
• signal_to_noise_ratio: 42.7 dB
• power_consumption: 117 mW

Applications & Benefits

• cell_imaging: Used for feature extraction and gesture recognition in real-time applications.
• benefits: Low voltage operation, high-speed processing, and energy efficiency, enabling AI applications.

Supporting Organizations

• supported_by: Qualcomm, NOVATEK, Taiwan Semiconductor Research Institute, Ministry of Science and Technology (MOST)

Manuscript Details

• publication_date: May 2021

Relevancy Score

• score: 10
• missing_fields:
  • pixel_size
  • dark_current
  • focal_length
  • aperture
  • field_of_view
  • distortion
  • sensitivity
  • shutter_speed
  • noise

Processed JSON Filename

request_1b7f8243-b28b-4ce4-bc76-80e40cdef5ef-a_05v_realtime_computational_cmos_image_sensor_with_programmable_kernel_for_feature_extraction.json