Loading in 5 sec....

Background vs. foreground segmentation of video sequencesPowerPoint Presentation

Background vs. foreground segmentation of video sequences

- 118 Views
- Uploaded on

Download Presentation
## PowerPoint Slideshow about ' Background vs. foreground segmentation of video sequences' - geordi

**An Image/Link below is provided (as is) to download presentation**

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

The Problem

- Separate video into two layers:
- stationary background
- moving foreground

- Sequence is very noisy; reference image (background) is not given

Simple approach (2)

Simple approach: noise can spoil everything

Variational approach

Find the background and foregroundsimultaneously by minimizing energy functional

Bonus: remove noise

[0,tmax]

Notationsgiven

need to find

C(x,t) background mask(1 on background, 0 on foreground)

N(x,t) original noisy sequence

B(x) background image

Energy functional: data term

- Degeneracy:can be trivially minimized by
- C 0 (everything is foreground)
- B N (take original image as background)

Energy functional: data term

original images should be close to the restored background imagein the background areas

there should be enough of background

Edge-preserving smoothnessRegularization term

Quadratic regularization [Tikhonov, Arsenin 1977]

ELE:

Known to produce very strong isotropic smoothing

Edge-preserving smoothnessRegularization term

ELE:

n

Edge-preserving smoothnessRegularization termChange the coordinate system:

ELE:

across the edge

along the edge

Compare:

Edge-preserving smoothnessRegularization term

Conditions on

Weak edge (s +0)

(s)

Isotropic smoothing

(s) is quadratic at zero

s

Edge-preserving smoothnessRegularization term

Conditions on

Strong edge (s +)

- no smoothing across the edge:

- more smoothing along the edge:

(s)

Anisotropic smoothing

(s) does not grow too fast at infinity

s

Edge-preserving smoothnessRegularization term

Conclusion

Using regularization term of the form:

we can achieve both isotropic smoothness in uniform regions and anisotropic smoothness on edges

with one function

Edge-preserving smoothnessRegularization term

Example of an edge-preserving function:

Edge-preserving smoothnessSpace of Bounded Variations

Even if we have an edge-preserving functional:

if the space of solutions{u}contains only smooth functions, we may not achieve the desired minimum:

Edge-preserving smoothnessSpace of Bounded Variations

which one is “better”?

Edge-preserving smoothnessSpace of Bounded Variations

integrable (absolute value) and with bounded variation

Functions are not required to have an integrable derivative …

What is the meaning of u in the regularization term?

Intuitively: norm of gradient |u|is replaced with variation |Du|

Total variation

Theorem (informally): if uBV() then

Hausdorff measure

1) cover with balls of diameter

2) sum up diameters for optimal cover (do not waste balls)

3) refine: 0

Hausdorff measure

Formally:

For ARNk-dimensional Hausdorff measure of A

up to normalization factor; covers are countable

- HN is just the Lebesgue measure

- curve in image: its length = H1 in R2

Total variation

Theorem (more formally): if uBV() then

u(x)

u+

u-

x0

x

u+,u- - approximate upper and lower limits

Su = {x; u+>u-}

the jump set

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation(= sum of perimeters)

Large total variation

(= sum of perimeters)

Edge-preserving smoothnessSpace of Bounded Variations

BV informally: functions with discontinuities on curves

Edges are preserved, texture is not preserved:

energy minimization in BV

temporal median

original sequence

Existence of solution

Under usual assumptions

1,2: R+R+ strictly convex, nondecreasing, with linear growth at infinity

minimum of E exists in BV(B,C1,…,CT)

(non-)Uniqueness

is not convex w.r.t. (B,C1,…,CT)! Solution may not be unique.

Uniqueness

But if c 3range2(Nt , t=1,…,T, x), then the functional is strictly convex, and solution is unique.

Interpretation: if we are allowed to say that everything is foreground, background image is not well-defined

Finding solution

BV is a difficult space: you cannot write Euler-Lagrange equations, cannot work numerically with function in BV.

- Strategy:
- construct approximating functionals admitting solution in a more regular space
- solve minimization problem for these functionals
- find solution as limit of the approximate solutions

Approximating functionals

Recall: 1,2(s) = s2 gives smooth solutions

Idea: replace i with i, which are quadratic at s 0 and s

Approximating problems

has unique solution in the space

- – convergence of functionals: ifE-converge to Ethen approximate solutions ofmin E
converge to min E

More results: INRIA_1Sequence restoration

More results: INRIA_2Sequence restoration

Download Presentation

Connecting to Server..