- 61 Views
- Uploaded on
- Presentation posted in: General

Background vs. foreground segmentation of video sequences

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Background vs. foreground segmentation of video sequences

+

=

- Separate video into two layers:
- stationary background
- moving foreground

- Sequence is very noisy; reference image (background) is not given

background

temporal median

temporal mean

threshold

Find the background and foregroundsimultaneously by minimizing energy functional

Bonus: remove noise

[0,tmax]

given

need to find

C(x,t) background mask(1 on background, 0 on foreground)

N(x,t) original noisy sequence

B(x) background image

B

N

B - N

C

- Degeneracy:can be trivially minimized by
- C 0 (everything is foreground)
- B N (take original image as background)

C

1

original images should be close to the restored background imagein the background areas

there should be enough of background

For background image B

For background mask C

Quadratic regularization [Tikhonov, Arsenin 1977]

ELE:

Known to produce very strong isotropic smoothing

Change regularization

ELE:

ELE:

n

Change the coordinate system:

ELE:

across the edge

along the edge

Compare:

Conditions on

Weak edge (s +0)

(s)

Isotropic smoothing

(s) is quadratic at zero

s

Conditions on

Strong edge (s +)

- no smoothing across the edge:

- more smoothing along the edge:

(s)

Anisotropic smoothing

(s) does not grow too fast at infinity

s

Conclusion

Using regularization term of the form:

we can achieve both isotropic smoothness in uniform regions and anisotropic smoothness on edges

with one function

Example of an edge-preserving function:

Even if we have an edge-preserving functional:

if the space of solutions{u}contains only smooth functions, we may not achieve the desired minimum:

which one is “better”?

Bounded Variation – ND case

bounded open subset, function

Variation of over

φ

where

integrable (absolute value) and with bounded variation

Functions are not required to have an integrable derivative …

What is the meaning of u in the regularization term?

Intuitively: norm of gradient |u|is replaced with variation |Du|

Theorem (informally): if uBV() then

area = 0

area > 0

How can we measure zero-measure sets?

1) cover with balls of diameter

2) sum up diameters for optimal cover (do not waste balls)

3) refine: 0

Formally:

For ARNk-dimensional Hausdorff measure of A

up to normalization factor; covers are countable

- HN is just the Lebesgue measure

- curve in image: its length = H1 in R2

Theorem (more formally): if uBV() then

u(x)

u+

u-

x0

x

u+,u- - approximate upper and lower limits

Su = {x; u+>u-}

the jump set

data term

regularization for background image

regularization for background masks

= perimeter = 4

Divide each side into n parts

Small total variation(= sum of perimeters)

Large total variation

(= sum of perimeters)

Small total variation

Large total variation

BV informally: functions with discontinuities on curves

Edges are preserved, texture is not preserved:

energy minimization in BV

temporal median

original sequence

Time-discretized problem:

Find minimum of E subject to:

Under usual assumptions

1,2: R+R+ strictly convex, nondecreasing, with linear growth at infinity

minimum of E exists in BV(B,C1,…,CT)

is not convex w.r.t. (B,C1,…,CT)! Solution may not be unique.

But if c 3range2(Nt , t=1,…,T, x), then the functional is strictly convex, and solution is unique.

Interpretation: if we are allowed to say that everything is foreground, background image is not well-defined

BV is a difficult space: you cannot write Euler-Lagrange equations, cannot work numerically with function in BV.

- Strategy:
- construct approximating functionals admitting solution in a more regular space
- solve minimization problem for these functionals
- find solution as limit of the approximate solutions

Recall: 1,2(s) = s2 gives smooth solutions

Idea: replace i with i, which are quadratic at s 0 and s

has unique solution in the space

- – convergence of functionals: ifE-converge to Ethen approximate solutions ofmin E
converge to min E

Thank You!