Impact Of Inpixel Processing Circuit Nonidealities On Multiobject Tracking Accuracy For Autonomous Driving

• doi: 10.1109/MWSCAS60917.2024.10658869
• title: Impact of In-Pixel Processing Circuit Non-idealities on Multi-object Tracking Accuracy for Autonomous Driving
• publisher: IEEE
• isbn: 979-8-3503-8718-6
• issn: 1548-3746
• rank: 2722
• access_type: LOCKED
• content_type: Conferences
• abstract: Deep learning algorithms are robust to a small amount of noise in the input image. Traditionally, image signal processors (ISP) are used with the CMOS image sensor (CIS) to enhance image quality which consume additional energy and latency. Here, we evaluate an ISP-Iess CIS architecture in the presence of noise and other in-pixel circuit non-idealities for the autonomous driving application. By integrating the in-pixel processing circuits to CIS, we filter out the redundant frames and only pass the critical bit information downstream to the backend processor. Such in-pixel processing does not allow ISP operations to be applied to the captured raw image. To reflect these limitations, we model and apply circuit non-idealities to the regenerated artificial raw images as an input to the QDTrack network for multi-object tracking. We evaluate the accuracy loss on the BDD100K dataset and examine its sensitivity on each of the image processing steps. We observe an overall accuracy drop of less than 1.2% in Identification Fl-score (IDFl) and 2.1 % in Multi-Object Tracking Accuracy (MOTA), suggesting that an ISP-Iess in-pixel processing circuit is feasible to reject 40% redundant frames directly on CIS.
• article_number: 10658869
• pdf_url: https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10658869
• html_url: https://ieeexplore.ieee.org/document/10658869/
• abstract_url: https://ieeexplore.ieee.org/document/10658869/
• publication_title: 2024 IEEE 67th International Midwest Symposium on Circuits and Systems (MWSCAS)
• conference_location: Springfield, MA, USA
• conference_dates: 11-14 Aug. 2024
• publication_number: 10654782
• is_number: 10654792
• publication_year: 2024
• publication_date: 11-14 Aug. 2024
• start_page: 806
• end_page: 810
• citing_paper_count: 0
• citing_patent_count: 0
• download_count: 175
• insert_date: 20240916
• index_terms:
  • ieee_terms:
    • Image quality
    • Accuracy
    • Program processors
    • Sensitivity
    • Noise
    • Signal processing algorithms
    • Information filters
  • author_terms:
    • Autonomous driving
    • CIS
    • demosaicing
    • ISP
    • multi-object tracking
  • dynamic_index_terms:
    • Multi-object Tracking
    • Multi-Object Tracking Accuracy
    • Circuit Non-idealities
    • Impact Of Non-idealities
    • Image Quality
    • Input Image
    • Raw Images
    • Accuracy Drop
    • Small Amount Of Noise
    • Artificial Images
    • Neural Network
    • Convolutional Neural Network
    • Denoising
    • Noise Reduction
    • Dynamic Range
    • Additive Noise
    • Additive White Gaussian Noise
    • Thermal Noise
    • Current Noise
    • Gamma Correction
    • Gamma Value
    • Storage Nodes
• isbn_formats:
  • format: Print on Demand(PoD) ISBN, value: 979-8-3503-8718-6, isbnType: New-2005
  • format: USB ISBN, value: 979-8-3503-8716-2, isbnType: New-2005
  • format: Electronic ISBN, value: 979-8-3503-8717-9, isbnType: New-2005
• authors:
  • Author Name: Janak Sharda
    Affiliation: Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA, USA
    Author URL: https://ieeexplore.ieee.org/author/37087116232
    ID: 37087116232
    Order: 1
    Author Affiliations:
    • Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA, USA
  • Author Name: Qiucheng Wu
    Affiliation: Computer Science, University of California at Santa Barbara, Santa Barbara, CA, USA
    Author URL: https://ieeexplore.ieee.org/author/37089515051
    ID: 37089515051
    Order: 2
    Author Affiliations:
    • Computer Science, University of California at Santa Barbara, Santa Barbara, CA, USA
  • Author Name: Shiyu Chang
    Affiliation: Computer Science, University of California at Santa Barbara, Santa Barbara, CA, USA
    Author URL: https://ieeexplore.ieee.org/author/37069387600
    ID: 37069387600
    Order: 3
    Author Affiliations:
    • Computer Science, University of California at Santa Barbara, Santa Barbara, CA, USA
  • Author Name: Shimeng Yu
    Affiliation: Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA, USA
    Author URL: https://ieeexplore.ieee.org/author/37085352683
    ID: 37085352683
    Order: 4
    Author Affiliations:
    • Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA, USA

Image Sensor

• sensor_type: CMOS
• resolution: Not specified
• dynamic_range: 50 dB
• pixel_size: Not specified
• dark_current: Not specified

Optical Data

• focal_length: Not specified
• aperture: Not specified
• field_of_view: Not specified
• distortion: Not specified

Performance Metrics

• power_consumption: 140 mW
• noise: Noise from thermal and fixed-pattern sources specified as variations from simulations.

Applications & Benefits

• cell_imaging: Defined for autonomous driving applications
• benefits: ISP-less design saves at least 100 mW of power and reduces latency by 10 ms.

Supporting Organizations

• supported_by: DARPA, Semiconductor Research Corporation (SRC)

Manuscript Details

• publication_date: 11-14 Aug. 2024

Relevancy Score

• score: 10

Processed JSON Filename

request_189b4f88-a6d4-47d8-b019-0c88ca309ac2-impact_of_inpixel_processing_circuit_nonidealities_on_multiobject_tracking_accuracy_for_autonomous_driving.json