background vs foreground segmentation of video sequences n.
Download
Skip this Video
Download Presentation
Background vs. foreground segmentation of video sequences

Loading in 2 Seconds...

play fullscreen
1 / 49

Background vs. foreground segmentation of video sequences - PowerPoint PPT Presentation


  • 127 Views
  • Uploaded on

Background vs. foreground segmentation of video sequences. +. =. The Problem. Separate video into two layers: stationary background moving foreground Sequence is very noisy; reference image (background) is not given. Simple approach (1). background. temporal median. temporal mean.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Background vs. foreground segmentation of video sequences' - geordi


Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
the problem
The Problem
  • Separate video into two layers:
    • stationary background
    • moving foreground
  • Sequence is very noisy; reference image (background) is not given
simple approach 1
Simple approach (1)

background

temporal median

temporal mean

variational approach
Variational approach

Find the background and foregroundsimultaneously by minimizing energy functional

Bonus: remove noise

notations

[0,tmax]

Notations

given

need to find

C(x,t) background mask(1 on background, 0 on foreground)

N(x,t) original noisy sequence

B(x) background image

energy functional data term1
Energy functional: data term
  • Degeneracy:can be trivially minimized by
          • C 0 (everything is foreground)
          • B N (take original image as background)
energy functional data term3
Energy functional: data term

original images should be close to the restored background imagein the background areas

there should be enough of background

energy functional smoothness
Energy functional: smoothness

For background image B

For background mask C

edge preserving smoothness regularization term
Edge-preserving smoothnessRegularization term

Quadratic regularization [Tikhonov, Arsenin 1977]

ELE:

Known to produce very strong isotropic smoothing

edge preserving smoothness regularization term3

n

Edge-preserving smoothnessRegularization term

Change the coordinate system:

ELE:

across the edge

along the edge

Compare:

edge preserving smoothness regularization term4
Edge-preserving smoothnessRegularization term

Conditions on 

Weak edge (s +0)

(s)

Isotropic smoothing

(s) is quadratic at zero

s

edge preserving smoothness regularization term5
Edge-preserving smoothnessRegularization term

Conditions on 

Strong edge (s +)

  • no smoothing across the edge:
  • more smoothing along the edge:

(s)

Anisotropic smoothing

(s) does not grow too fast at infinity

s

edge preserving smoothness regularization term6
Edge-preserving smoothnessRegularization term

Conclusion

Using regularization term of the form:

we can achieve both isotropic smoothness in uniform regions and anisotropic smoothness on edges

with one function 

edge preserving smoothness regularization term7
Edge-preserving smoothnessRegularization term

Example of an edge-preserving function:

edge preserving smoothness space of bounded variations
Edge-preserving smoothnessSpace of Bounded Variations

Even if we have an edge-preserving functional:

if the space of solutions{u}contains only smooth functions, we may not achieve the desired minimum:

bounded variation nd case

Bounded Variation – ND case

bounded open subset, function

Variation of over

φ

where

edge preserving smoothness space of bounded variations2
Edge-preserving smoothnessSpace of Bounded Variations

integrable (absolute value) and with bounded variation

Functions are not required to have an integrable derivative …

What is the meaning of u in the regularization term?

Intuitively: norm of gradient |u|is replaced with variation |Du|

total variation
Total variation

Theorem (informally): if uBV() then

hausdorff measure
Hausdorff measure

area = 0

area > 0

How can we measure zero-measure sets?

hausdorff measure1
Hausdorff measure

1) cover with balls of diameter 

2) sum up diameters for optimal cover (do not waste balls)

3) refine:  0

hausdorff measure2
Hausdorff measure

Formally:

For ARNk-dimensional Hausdorff measure of A

up to normalization factor; covers are countable

  • HN is just the Lebesgue measure
  • curve in image: its length = H1 in R2
total variation1
Total variation

Theorem (more formally): if uBV() then

u(x)

u+

u-

x0

x

u+,u- - approximate upper and lower limits

Su = {x; u+>u-}

the jump set

energy functional1
Energy functional

data term

regularization for background image

regularization for background masks

total variation example
Total variation: example

= perimeter = 4

Divide each side into n parts

edge preserving smoothness space of bounded variations3
Edge-preserving smoothnessSpace of Bounded Variations

Small total variation(= sum of perimeters)

Large total variation

(= sum of perimeters)

edge preserving smoothness space of bounded variations4
Edge-preserving smoothnessSpace of Bounded Variations

Small total variation

Large total variation

edge preserving smoothness space of bounded variations5
Edge-preserving smoothnessSpace of Bounded Variations

BV informally: functions with discontinuities on curves

Edges are preserved, texture is not preserved:

energy minimization in BV

temporal median

original sequence

energy functional2
Energy functional

Time-discretized problem:

Find minimum of E subject to:

existence of solution
Existence of solution

Under usual assumptions

1,2: R+R+ strictly convex, nondecreasing, with linear growth at infinity

minimum of E exists in BV(B,C1,…,CT)

non uniqueness
(non-)Uniqueness

is not convex w.r.t. (B,C1,…,CT)! Solution may not be unique.

uniqueness
Uniqueness

But if c  3range2(Nt , t=1,…,T, x), then the functional is strictly convex, and solution is unique.

Interpretation: if we are allowed to say that everything is foreground, background image is not well-defined

finding solution
Finding solution

BV is a difficult space: you cannot write Euler-Lagrange equations, cannot work numerically with function in BV.

  • Strategy:
  • construct approximating functionals admitting solution in a more regular space
  • solve minimization problem for these functionals
  • find solution as limit of the approximate solutions
approximating functionals
Approximating functionals

Recall: 1,2(s) = s2 gives smooth solutions

Idea: replace i with i, which are quadratic at s  0 and s 

approximating problems
Approximating problems

has unique solution in the space

  • – convergence of functionals: ifE-converge to Ethen approximate solutions ofmin E

converge to min E