Video From Stills Lensless Imaging With Rolling Shutter

• doi: 10.1109/ICCPHOT.2019.8747341
• title: Video from Stills: Lensless Imaging with Rolling Shutter
• publisher: IEEE
• isbn: 978-1-7281-3264-8
• issn: 2164-9774
• rank: 3965
• access_type: LOCKED
• content_type: Conferences
• abstract: Because image sensor chips have a finite bandwidth with which to read out pixels, recording video typically requires a trade-off between frame rate and pixel count. Compressed sensing techniques can circumvent this trade-off by assuming that the image is compressible. Here, we propose using multiplexing optics to spatially compress the scene, enabling information about the whole scene to be sampled from a row of sensor pixels, which can be read off quickly via a rolling shutter CMOS sensor. Conveniently, such multiplexing can be achieved with a simple lensless, diffuser-based imaging system. Using sparse recovery methods, we are able to recover 140 video frames at over 4,500 frames per second, all from a single captured image with a rolling shutter sensor. Our proof-of-concept system uses easily-fabricated diffusers paired with an off-the-shelf sensor. The resulting prototype enables compressive encoding of high frame rate video into a single rolling shutter exposure, and exceeds the sampling-limited performance of an equivalent global shutter system for sufficiently sparse objects.
• article_number: 8747341
• pdf_url: https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8747341
• html_url: https://ieeexplore.ieee.org/document/8747341/
• abstract_url: https://ieeexplore.ieee.org/document/8747341/
• publication_title: 2019 IEEE International Conference on Computational Photography (ICCP)
• conference_location: Tokyo, Japan
• conference_dates: 15-17 May 2019
• publication_number: 8743501
• is_number: 8747322
• publication_year: 2019
• publication_date: 15-17 May 2019
• start_page: 1
• end_page: 8
• citing_paper_count: 30
• citing_patent_count: 0
• download_count: 2320
• insert_date: 20190627
• index_terms:
  • ieee_terms:
    • Multiplexing
    • Cameras
    • Bandwidth
    • Optics
    • Image reconstruction
    • Image coding
  • author_terms:
    • optical imaging
    • video recording
    • compressed sensing
    • lensless imaging
    • video signal processing
    • CMOS image sensors
  • dynamic_index_terms:
    • Rolling Shutter
    • Lensless Imaging
    • Frame Rate
    • Video Frames
    • Image Sensor
    • Camera Sensor
    • Pixel Count
    • Sensor Pixel
    • Spatial Resolution
    • Exposure Time
    • Single Image
    • Focal Length
    • Focal Lengths
    • Focal Distance
    • Inverse Problem
    • Forward Problem
    • Square Wave
    • Square-wave
    • Forward Model
    • Point Spread Function
    • Bitrate
    • Current Sensor
    • High Dynamic Range
    • High-dynamic-range
    • Dynamic Scenes
    • Ball Bearings
    • Ball-bearing
    • Underdetermined Problem
    • Spatial Multiplexing
    • Spatial Division Multiplexing
    • Linear Convolution
    • Scene Point
    • Digital Micromirror Device
    • Subset Of Pixels
    • Compression Scheme
    • Compression Strategy
    • Point Source
    • Aperture
• isbn_formats:
  • format: Print on Demand(PoD) ISBN, value: 978-1-7281-3264-8, isbnType: New-2005
  • format: Electronic ISBN, value: 978-1-7281-3263-1, isbnType: New-2005
• authors:
  • Author Name: Nick Antipa
    Affiliation: Department of Electrical Engineering and Computer Sciences, University of California, Berkeley
    Author URL: https://ieeexplore.ieee.org/author/37085814188
    ID: 37085814188
    Order: 1
    Author Affiliations:
    • Department of Electrical Engineering and Computer Sciences, University of California, Berkeley
  • Author Name: Patrick Oare
    Affiliation: Department of Electrical Engineering and Computer Sciences, University of California, Berkeley
    Author URL: https://ieeexplore.ieee.org/author/37086874351
    ID: 37086874351
    Order: 2
    Author Affiliations:
    • Department of Electrical Engineering and Computer Sciences, University of California, Berkeley
  • Author Name: Emrah Bostan
    Affiliation: Department of Electrical Engineering and Computer Sciences, University of California, Berkeley
    Author URL: https://ieeexplore.ieee.org/author/38234557200
    ID: 38234557200
    Order: 3
    Author Affiliations:
    • Department of Electrical Engineering and Computer Sciences, University of California, Berkeley
  • Author Name: Ren Ng
    Affiliation: Department of Electrical Engineering and Computer Sciences, University of California, Berkeley
    Author URL: https://ieeexplore.ieee.org/author/37085689740
    ID: 37085689740
    Order: 4
    Author Affiliations:
    • Department of Electrical Engineering and Computer Sciences, University of California, Berkeley
  • Author Name: Laura Waller
    Affiliation: Department of Electrical Engineering and Computer Sciences, University of California, Berkeley
    Author URL: https://ieeexplore.ieee.org/author/38231000900
    ID: 38231000900
    Order: 5
    Author Affiliations:
    • Department of Electrical Engineering and Computer Sciences, University of California, Berkeley

Image Sensor

• sensor_type: CMOS
• resolution: unspecified
• dynamic_range: unspecified
• pixel_size: unspecified
• dark_current: unspecified

Optical Data

• focal_length: 12.7 mm
• aperture: unspecified
• field_of_view: 30° by 40°
• distortion: unspecified

Performance Metrics

• frame_rate: 4,545 fps
• signal_to_noise_ratio: unspecified
• sensitivity: unspecified
• shutter_speed: unspecified
• power_consumption: unspecified
• noise: unspecified

Applications & Benefits

• cell_imaging: unspecified
• benefits: Enables high frame rate video capture from a single rolling shutter exposure.

Supporting Organizations

• supported_by: The Moore Foundation, DARPA, National Science Foundation, Alfred P. Sloan Foundation, and Swiss National Science Foundation.

Manuscript Details

• publication_date: 15-17 May 2019

Relevancy Score

• score: 9
• missing_fields:
  • fill factor
  • quantum efficiency
  • fixed-pattern noise

Processed JSON Filename

request_c458d513-61b7-42ae-9125-b83db74a89ee-video_from_stills_lensless_imaging_with_rolling_shutter.json