Crossspectral Gatedrgb Stereo Depth Estimation

• doi: 10.1109/CVPR52733.2024.02046
• title: Cross-spectral Gated-RGB Stereo Depth Estimation
• publisher: IEEE
• isbn: 979-8-3503-5301-3
• issn: 1063-6919
• rank: 2016
• access_type: LOCKED
• content_type: Conferences
• abstract: Gated cameras flood-illuminate a scene and capture the time-gated impulse response of a scene. By employing nanosecond-scale gates, existing sensors are capable of capturing mega-pixel gated images, delivering dense depth improving on today's LiDAR sensors in spatial resolution and depth precision. Although gated depth estimation methods deliver a million of depth estimates per frame, their res-olution is still an order below existing RGB imaging methods. In this work, we combine high-resolution stereo HDR RCCB cameras with gated imaging, allowing us to exploit depth cues from active gating, multi-view RGB and multi-view NIR sensing - multi-view and gated cues across the entire spectrum. The resulting capture system consists only of low-cost CMOS sensors and flood-illumination. We pro-pose a novel stereo-depth estimation method that is capa-ble of exploiting these multi-modal multi-view depth cues, including the active illumination that is measured by the RCCB camera when removing the IR-cut filter. The pro-posed method achieves accurate depth at long ranges, out-performing the next best existing method by 39% for ranges of 100 to 220 m in MAE on accumulated LiDAR ground-truth. Our code, models and datasets are available here11https://light.princeton.edu/gatedrccbstereo/.
• article_number: 10654986
• pdf_url: https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10654986
• html_url: https://ieeexplore.ieee.org/document/10654986/
• abstract_url: https://ieeexplore.ieee.org/document/10654986/
• publication_title: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
• conference_location: Seattle, WA, USA
• conference_dates: 16-22 June 2024
• publication_number: 10654794
• is_number: 10654797
• publication_year: 2024
• publication_date: 16-22 June 2024
• start_page: 21654
• end_page: 21665
• citing_paper_count: 0
• citing_patent_count: 0
• download_count: 79
• insert_date: 20240916
• index_terms:
  • ieee_terms:
    • Image sensors
    • Laser radar
    • Accuracy
    • Estimation
    • Lighting
    • Logic gates
    • Cameras
  • author_terms:
    • Cross-Spectral Imaging
    • Depth Estimation
    • Stereo Depth Estimation
    • Gated Imaging
  • dynamic_index_terms:
    • Depth Estimation
    • Stereo Depth Estimation
    • Depth From Stereo
    • Spatial Resolution
    • RGB Images
    • Depth Perception
    • Depth Cues
    • Dense Depth
    • Visible Light
    • Visible Region
    • Visible Spectrum
    • Convolutional Neural Network
    • Time-of-flight
    • Feature Maps
    • Mean Absolute Error
    • Low Light
    • Point Cloud
    • Small Objects
    • Depth Map
    • Function Of Depth
    • Self-driving
    • Self-driving Cars
    • Autonomous Cars
    • Camera View
    • Low Light Conditions
    • Feature Alignment
    • Stereo Images
    • High-resolution Depth
    • Stereo Matching
    • Stereo Camera
    • NIR Spectra
    • Stereo Pairs
    • Time-of-flight Sensors
    • Time Of Flight Camera
    • Monocular
    • Image Intensity
    • Light Sensitivity
    • Photophobic
    • Iterative Refinement
• isbn_formats:
  • format: Print on Demand(PoD) ISBN, value: 979-8-3503-5301-3, isbnType: New-2005
  • format: Electronic ISBN, value: 979-8-3503-5300-6, isbnType: New-2005
• authors:
  • Author Name: Samuel Brucker
    Affiliation: Torc Robotics
    Author URL: https://ieeexplore.ieee.org/author/355667046488159
    ID: 355667046488159
    Order: 1
    Author Affiliations:
    • Torc Robotics
  • Author Name: Stefanie Walz
    Affiliation: Mercedes-Benz
    Author URL: https://ieeexplore.ieee.org/author/37088594823
    ID: 37088594823
    Order: 2
    Author Affiliations:
    • Mercedes-Benz
  • Author Name: Mario Bijelic
    Affiliation: Torc Robotics
    Author URL: https://ieeexplore.ieee.org/author/37086488671
    ID: 37086488671
    Order: 3
    Author Affiliations:
    • Torc Robotics
    • Princeton University
  • Author Name: Felix Heide
    Affiliation: Torc Robotics
    Author URL: https://ieeexplore.ieee.org/author/37075809600
    ID: 37075809600
    Order: 4
    Author Affiliations:
    • Torc Robotics
    • Princeton University

Image Sensor

• sensor_type: CMOS
• resolution: 8 megapixels
• dynamic_range: 140 dB
• pixel_size: 2.1 µm
• dark_current: Dk v, dependent on gating settings

Optical Data

• focal_length: not specified
• aperture: not specified
• field_of_view: not specified
• distortion: not specified

Performance Metrics

Applications & Benefits

• cell_imaging: not specified
• benefits: High-resolution depth maps, effective in varied lighting conditions (day/night)

Supporting Organizations

• supported_by: Torc Robotics, Mercedes-Benz, Princeton University

Manuscript Details

• publication_date: 16-22 June 2024

Relevancy Score

• score: 9
• missing_fields:
  • focal_length
  • aperture
  • field_of_view
  • distortion
  • frame_rate
  • signal_to_noise_ratio
  • sensitivity
  • shutter_speed
  • power_consumption
  • noise

Processed JSON Filename

request_c75dd04d-ac7a-4986-bfbb-433190ab83be-crossspectral_gatedrgb_stereo_depth_estimation.json