Gated3d Monocular 3D Object Detection From Temporal Illumination Cues

• doi: 10.1109/ICCV48922.2021.00293
• title: Gated3D: Monocular 3D Object Detection From Temporal Illumination Cues
• publisher: IEEE
• isbn: 978-1-6654-2813-2
• issn: 1550-5499
• rank: 2540
• access_type: LOCKED
• content_type: Conferences
• abstract: Today’s state-of-the-art methods for 3D object detection are based on lidar, stereo, or monocular cameras. Lidar-based methods achieve the best accuracy, but have a large footprint, high cost, and mechanically-limited angular sampling rates, resulting in low spatial resolution at long ranges. Recent approaches using low-cost monocular or stereo cameras promise to overcome these limitations but struggle in low-light or low-contrast regions as they rely on passive CMOS sensors. We propose a novel 3D object detection modality that exploits temporal illumination cues from a low-cost monocular gated imager. We introduce a novel deep detection architecture, Gated3D, that is tailored to temporal illumination cues in gated images. This modality allows us to exploit mature 2D object feature extractors that guide the 3D predictions through a frustum segment estimation. We assess the proposed method experimentally on a 3D detection dataset that includes gated images captured over 10,000 km of driving data. We validate that our method outperforms state-of-the-art monocular and stereo methods, opening up a new sensor modality as an avenue to replace lidar in autonomous driving. https://light.princeton.edu/gated3d
• article_number: 9710811
• pdf_url: https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9710811
• html_url: https://ieeexplore.ieee.org/document/9710811/
• abstract_url: https://ieeexplore.ieee.org/document/9710811/
• publication_title: 2021 IEEE/CVF International Conference on Computer Vision (ICCV)
• conference_location: Montreal, QC, Canada
• conference_dates: 10-17 Oct. 2021
• publication_number: 9709627
• is_number: 9709628
• publication_year: 2021
• publication_date: 10-17 Oct. 2021
• start_page: 2918
• end_page: 2928
• citing_paper_count: 6
• citing_patent_count: 0
• download_count: 92
• insert_date: 20220228
• index_terms:
  • ieee_terms:
    • Image segmentation
    • Three-dimensional displays
    • Laser radar
    • Lighting
    • Object detection
    • Logic gates
    • Feature extraction
  • author_terms:
    • Detection and localization in 2D and 3D
    • 3D from a single image and shape-from-x
    • Stereo
    • 3D from multiview and other sensors
    • Vision for robotics and autonomous vehicles
  • dynamic_index_terms:
    • Object Detection
    • Temporal Cues
    • 3D Object Detection
    • Monocular 3D Object Detection
    • Temporal Illumination
    • Spatial Resolution
    • Detection Methods
    • Detection Method
    • Low Resolution
    • 3D Datasets
    • 3D Data Sets
    • Passive Sensors
    • Stereo Camera
    • 3D Detection
    • Pedestrian
    • Intersection Over Union
    • Jaccard Similarity
    • Jaccard Distance
    • Intersection-over-union
    • Jaccard Index
    • Point Cloud
    • Point Clouds
    • Bounding Box
    • RGB Images
    • Depth Map
    • Function Of Depth
    • 3D Position
    • 3D Location
    • Depth Estimation
    • 3D Bounding Box
    • Objective Dimensions
    • Low-cost Sensors
    • Backbone Feature
    • Passive Imaging
    • Stereo Images
    • Depth Perception
    • Depth Cues
    • Supplementary Document
    • Depth Of Use
• isbn_formats:
  • format: Print on Demand(PoD) ISBN, value: 978-1-6654-2813-2, isbnType: New-2005
  • format: Electronic ISBN, value: 978-1-6654-2812-5, isbnType: New-2005
• authors:
  • Author Name: Frank Julca-Aguilar
    Affiliation: Algolux
    Author URL: https://ieeexplore.ieee.org/author/37085386911
    ID: 37085386911
    Order: 1
    Author Affiliations:
    • Algolux
  • Author Name: Jason Taylor
    Affiliation: Algolux
    Author URL: https://ieeexplore.ieee.org/author/37089315621
    ID: 37089315621
    Order: 2
    Author Affiliations:
    • Algolux
  • Author Name: Mario Bijelic
    Affiliation: Mercedes-Benz AG
    Author URL: https://ieeexplore.ieee.org/author/37086488671
    ID: 37086488671
    Order: 3
    Author Affiliations:
    • Mercedes-Benz AG
    • Princeton University
  • Author Name: Fahim Mannan
    Affiliation: Algolux
    Author URL: https://ieeexplore.ieee.org/author/37706735000
    ID: 37706735000
    Order: 4
    Author Affiliations:
    • Algolux
  • Author Name: Ethan Tseng
    Affiliation: Princeton University
    Author URL: https://ieeexplore.ieee.org/author/37088454559
    ID: 37088454559
    Order: 5
    Author Affiliations:
    • Princeton University
  • Author Name: Felix Heide
    Affiliation: Algolux
    Author URL: https://ieeexplore.ieee.org/author/37075809600
    ID: 37075809600
    Order: 6
    Author Affiliations:
    • Algolux
    • Princeton University

Image Sensor

• sensor_type: CMOS
• resolution: 1280px×720px
• dynamic_range: 10 bit uint
• pixel_size: 10 µm
• dark_current: poissonian photon shot noise and gaussian read-out noise

Optical Data

• focal_length: 23 mm
• aperture: Not specified
• field_of_view: 31.1°H×17.8°V
• distortion: Not specified

Performance Metrics

• frame_rate: 120 Hz

Applications & Benefits

• cell_imaging: Not specified
• benefits: Enables digital imaging under low-light conditions and adverse weather scenarios.

Supporting Organizations

• supported_by: NSF CAREER Award, Sony Faculty Innovation Award

Manuscript Details

• publication_date: 10-17 Oct. 2021

Relevancy Score

• score: 9
• missing_fields:
  • aperture
  • distortion
  • signal_to_noise_ratio
  • sensitivity
  • shutter_speed
  • power_consumption
  • noise

Processed JSON Filename

request_32db4769-be42-49c9-996f-0e8e0664cbc2-gated3d_monocular_3d_object_detection_from_temporal_illumination_cues.json