1 / 25

Spatio-Temporal Frequency Analysis for Removing Rain and Snow from Videos

This research paper explores a method for removing rain and snow from outdoor videos using spatio-temporal frequency analysis. The approach involves modeling precipitation patterns and refining estimates to enhance video quality.

hurstl
Download Presentation

Spatio-Temporal Frequency Analysis for Removing Rain and Snow from Videos

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Spatio-Temporal Frequency Analysis for Removing Rain and Snow from Videos Peter Barnum Takeo Kanade Srinivasa Narasimhan Carnegie Mellon University June 16, 2007

  2. Bad weather in outdoor videos Rain Snow

  3. Previous Work Image-based blurring Camera-based blurring Spikes due to rain Pixel Intensity Garg and Nayar ICCV ‘05 Streak detection Time Hase et al. ICIP ’98 Starik and Werman IWTAS ‘03 Zhang et al. ICME ‘06 Garg and Nayar CVPR ‘04

  4. Groups of streaks

  5. Length Imaging a falling particle Gaussians Snowflakes Raindrops Breadth

  6. Gaussian model of streak appearance Camera parameters are constant A Gaussian A blurred Gaussian streak Including streak orientation (just a coordinate space rotation)

  7. Where are the streaks?

  8. Fourier transform of the streaks

  9. Building a complete model For a given precipitation intensity For all common drop sizes Blurred Gaussian For all depths that are in-focus

  10. Model accuracy Original image 2D Fourier Transform Model Model with 50% randomly set to zero

  11. Large frame-to-frame difference Distributed evenly in frequency space Finding the precipitation rate Rain and snow have two useful properties Mailbox Building 100% 0% Snow

  12. Frame-to-frame differences t=1 t=3 t=2 w=-1 w=+1 w=0

  13. Frame-to-frame differences t=1 t=3 t=2 w=-1 w=+1 w=0

  14. Finding the precipitation rate For most objects But for rain and snow Because of these properties

  15. Estimating the model parameters Precipitation rate Streak orientation The precipitation rate is approximately: Estimating the streak orientation requires a spatial consistent estimate The orientation is found by:

  16. Frequency space examples Original image 2D FT Model

  17. = Computing per-frequency estimates At a given frequency:

  18. Computing per-pixel estimates

  19. t=1 t=2 t=3 Refining the single frame estimate Exactly the same model, constant in w

  20. Computing per-pixel estimates

  21. Conclusions and future work A global frequency method for rain and snow removal Refining global estimates with local features “Into each life some rain must fall.” -Henry Wadsworth Longfellow

  22. Extra slides

More Related