detection and removal of rain from videos l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Detection and Removal of Rain from Videos PowerPoint Presentation
Download Presentation
Detection and Removal of Rain from Videos

Loading in 2 Seconds...

play fullscreen
1 / 27

Detection and Removal of Rain from Videos - PowerPoint PPT Presentation


  • 491 Views
  • Uploaded on

Detection and Removal of Rain from Videos Kshitiz Garg and Shree K. Nayar Department of Computer Science Columbia University IEEE CVPR Conference June 2004, Washington DC, USA Sponsors: NSF, DARPA Outdoor Vision and Weather Fog Mist [Nayar and Narasimhan,99] [Narasimhan and Nayar,

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Detection and Removal of Rain from Videos' - ostinmannual


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
detection and removal of rain from videos

Detection and Removal of Rain from Videos

Kshitiz Garg and Shree K. Nayar

Department of Computer Science

Columbia University

IEEE CVPR Conference

June 2004, Washington DC, USA

Sponsors: NSF, DARPA

outdoor vision and weather
Outdoor Vision and Weather

Fog

Mist

[Nayar and Narasimhan,99]

[Narasimhan and Nayar,

00,01,02,03]

Rain

Snow

steady versus dynamic weather
Steady versus Dynamic Weather

Steady Weather (Fog, Mist)

Dynamic Weather (Rain, Snow)

Pixel

Pixel

Fog-droplet (1-10)

Raindrop (0.5-10)

Aggregate scattering

Individual scattering

Steady

Weather

Dynamic

Weather

Objects

previous work on rain
Previous Work on Rain
  • Atmospheric Sciences
    • Physical Properties of Rain
  • Communication
    • Signal Transmission through Rain
  • Graphics
    • Particle Systems
    • Heuristic Based Methods

[Beard and Chung 1987, Wang 1975, Marshall and Palmer 1948,Mason 1975]

[ Ting 1982,Manning 1993, Robert 1992 ]

[ Reeves 1983, Sims 1997 ]

[ Starik and Werman 2003 ]

image processing does not suffice

Median

Filtering

Image Processing Does Not Suffice

Rain with Scene Motion

Median Filtered Video

Courtesy © New Line Productions, Inc.

Difference (Original and Median Filtered)

modeling the visual effects of rain
Modeling the Visual Effects of Rain

Rain Drop

Rain Streak

Dynamics

Rain

Photometry

physical properties of rain drops

800

600

2.5 mm

Number Density

2

400

1.5

1

0.5

Falling Direction

200

4

1

2

3

a

Drop radius (mm)

Shapes of Rain Drop

Size Distribution of Rain Drop

Physical Properties of Rain Drops

Beard and Chung 1987

Marshall and Palmer 1948

(1/m)

N

Drop density

95% of Rain Drops are Spherical

visual appearance of a rain drop
Visual Appearance of a Rain Drop

Beard and Chung 1987

Marshall and Palmer 1948

Specular Reflection

N

Refraction

Internal Reflection

brightness properties of a rain drop
Brightness Properties of a Rain Drop

Refraction

Specular

1

0.8

FOV=1650

94%

0.6

0.4

0.2

Internal Reflection

Background

0

0.2

0.4

0.6

0.8

1

Distance from the center of the drop

Rain Drops are Brighter than Background

Brightness is Independent of Background

the world in a drop
The World in a Drop

A Drop

World in a Drop

Perspective Views of the World

rain and background intensities
Rain and Background Intensities

High Shutter Speed

( Exposure Time = 1/1000 s )

Normal Shutter Speed

( Exposure Time = 1/30 s )

motion blur model for rain drops

Pixel

Frame

Motion Blur Model for Rain Drops

Irradiance

Stationary

Rain Drop

Background

Pixel

Background

Time

Exposure Time

Time a drop stays over a pixel :

Change in Intensity :

Rain produces a “Delta Signal” in Time

motion blurred intensities of rain streaks
Motion Blurred Intensities of Rain Streaks

Irradiance

Intensity at a Pixel on a Rain Streak :

Stationary

Rain Drop

Intensity of the Background :

Background

Time

Exposure Time

Change in Intensity due to a Rain Streak :

Constants

Constants

Change in Intensity is Linearly related to Background

dynamics of rain
Dynamics of Rain

Background

Pin Hole

Frames

Time

Y-axis

X-axis

Discrete Space Time Volume

Continuous Space Time Domain

Rain Produces Strong Directional Correlations in Videos

rain detection and removal algorithm
Rain Detection and Removal Algorithm

Linear

Photometric

Constraint

Select Delta

in time

Estimated Rain Pixels

Candidate Rain Pixels

Original Video

Neighboring Frames

Compute

direction and

strength

Dynamic

Constraint

Rain Segmented

Correlation Map

Estimated Rain Field

slide21

Results: A clip from Magnolia

Original Clip

Courtesy © New Line Productions, Inc.

Rain Detection

slide22

Results: A clip from Magnolia

Original Video

Derained Video

Rain Component

results rain vs ripples
Results: Rain vs Ripples

Original Video

Rain Segmentation

results rain vs ripples24
Results: Rain vs Ripples

Original Video

Derained Video

Rain Component

results person walking in rain
Results: Person Walking in Rain

Original Video

Rain Detection

results person walking in rain26
Results: Person Walking in Rain

Derained Video

Original Video

Rain Component

summary
Summary
  • Visual Appearance of Rain

+

=

Dynamics

Rain

Photometry

  • Rain Detection and Removal
  • Future Work: Snow

Dynamics

Photometry