Deep Homography Mixture For Single Image Rolling Shutter Correction

• doi: 10.1109/ICCV51070.2023.00905
• title: Deep Homography Mixture for Single Image Rolling Shutter Correction
• publisher: IEEE
• isbn: 979-8-3503-0719-1
• issn: 1550-5499
• rank: 2059
• access_type: LOCKED
• content_type: Conferences
• abstract: We present a deep homography mixture motion model for single image rolling shutter correction. Rolling shutter (RS) effects are often caused by row-wise exposure delay in the widely adopted CMOS sensor. Previous methods often require more than one frame for the correction, leading to data quality requirements. Few approaches address the more challenging task of single image RS correction, which often adopt designs like trajectory estimation or long rectangular kernels, to learn the camera motion parameters of an RS image, to restore the global shutter (GS) image. In this work, we adopt a more straightforward method to learn deep homography mixture motion between an RS image and its corresponding GS image, without large solution space or strict restrictions on image features. We show that dividing an image into blocks with a Gaussian weight of block scanlines fits well for the RS setting. Moreover, instead of directly learning the motion mapping, we learn coefficients that assemble several motion bases to produce the correction motion, where these bases are learned from the consecutive frames of natural videos beforehand. Experiments show that our method outperforms existing single RS methods statistically and visually, in both synthesized and real RS images. Our code and dataset are available at https://github.com/DavidYan2001/Deep_HM.
• article_number: 10378021
• pdf_url: https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10378021
• html_url: https://ieeexplore.ieee.org/document/10378021/
• abstract_url: https://ieeexplore.ieee.org/document/10378021/
• publication_title: 2023 IEEE/CVF International Conference on Computer Vision (ICCV)
• conference_location: Paris, France
• conference_dates: 1-6 Oct. 2023
• publication_number: 10376473
• is_number: 10376477
• publication_year: 2023
• publication_date: 1-6 Oct. 2023
• start_page: 9834
• end_page: 9843
• citing_paper_count: 2
• citing_patent_count: 0
• download_count: 46
• insert_date: 20240115
• index_terms:
  • ieee_terms:
    • Semiconductor device modeling
    • Pipelines
    • Estimation
    • Mixture models
    • Trajectory
    • Image restoration
    • Delays
  • dynamic_index_terms:
    • Single Image
    • Rolling Shutter
    • Deep Homography
    • Image Features
    • Imaging Characteristics
    • Line Scan
    • Motion Model
    • Consecutive Frames
    • Camera Motion
    • Nature Video
    • Gaussian Weighting
    • Deep Network
    • Types Of Images
    • Singular Value
    • Low Light
    • Learning-based Methods
    • Peak Signal-to-noise Ratio
    • PSNR
    • Optical Flow
    • Flow Map
    • Motion Vector
    • Structural Similarity Index Measure
    • Homography Matrix
    • Homography Matrices
    • Endpoint Error
    • Weight Map
    • Flow Motion
    • Scene Depth
    • Building Dataset
    • Low Texture
    • Rich Texture
    • Pair Of Frames
• isbn_formats:
  • format: Print on Demand(PoD) ISBN, value: 979-8-3503-0719-1, isbnType: New-2005
  • format: Electronic ISBN, value: 979-8-3503-0718-4, isbnType: New-2005
• authors:
  • Author Name: Weilong Yan
    Affiliation: University of Electronic Science and Technology of China
    Author URL: https://ieeexplore.ieee.org/author/37089886290
    ID: 37089886290
    Order: 1
    Author Affiliations:
    • University of Electronic Science and Technology of China
  • Author Name: Robby T. Tan
    Affiliation: National University of Singapore
    Author URL: https://ieeexplore.ieee.org/author/37085638421
    ID: 37085638421
    Order: 2
    Author Affiliations:
    • National University of Singapore
  • Author Name: Bing Zeng
    Affiliation: University of Electronic Science and Technology of China
    Author URL: https://ieeexplore.ieee.org/author/37270871000
    ID: 37270871000
    Order: 3
    Author Affiliations:
    • University of Electronic Science and Technology of China
  • Author Name: Shuaicheng Liu
    Affiliation: University of Electronic Science and Technology of China
    Author URL: https://ieeexplore.ieee.org/author/37085838967
    ID: 37085838967
    Order: 4
    Author Affiliations:
    • University of Electronic Science and Technology of China

Image Sensor

• sensor_type: CMOS
• resolution: not specified
• dynamic_range: not specified
• pixel_size: not specified
• dark_current: not specified

Optical Data

• focal_length: not specified
• aperture: not specified
• field_of_view: not specified
• distortion: not specified

Performance Metrics

Applications & Benefits

• cell_imaging: not specified
• benefits: not specified

Supporting Organizations

• supported_by: not specified

Manuscript Details

• publication_date: 1-6 Oct. 2023

Relevancy Score

• score: 9
• missing_fields:
  • resolution
  • dynamic_range
  • pixel_size
  • dark_current
  • focal_length
  • aperture
  • field_of_view
  • distortion
  • frame_rate
  • sensitivity
  • shutter_speed
  • power_consumption
  • noise
  • cell_imaging
  • benefits
  • supported_by

Processed JSON Filename

request_3d0785ef-b1d3-4d05-b8e3-cde4450b735a-deep_homography_mixture_for_single_image_rolling_shutter_correction.json