A duality based approach for realtime tv l 1 optical flow
This presentation is the property of its rightful owner.
Sponsored Links
1 / 16

A Duality Based Approach for Realtime TV-L 1 Optical Flow PowerPoint PPT Presentation


  • 172 Views
  • Uploaded on
  • Presentation posted in: General

A Duality Based Approach for Realtime TV-L 1 Optical Flow. Christopher Zach 1 , Thomas Pock 2 , and Horst Bischof 2. 1 VRVis Research Center, Graz 2 Institute for Computer Graphics and Vision, TU Graz E-mail: { zach, pock, bischof [email protected] time. Motivation.

Download Presentation

A Duality Based Approach for Realtime TV-L 1 Optical Flow

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


A duality based approach for realtime tv l 1 optical flow

A Duality Based Approach for Realtime TV-L1 Optical Flow

Christopher Zach1, Thomas Pock2, and Horst Bischof2

1 VRVis Research Center, Graz

2 Institute for Computer Graphics and Vision, TU Graz

E-mail: {zach, pock, [email protected]


Motivation

time

Motivation

  • Discontinuity preserving regularization of the flow field

  • Robustness to occulsions

  • Handle large displacements

  • Realtime (> 30 fps) for large images (512x512)


Outline

Outline

  • (I) Variational Optical Flow

  • (II) TV-L1 optical flow

  • (III) Duality Based Approach

  • (IV) Acceleration using the GPU

  • (V) Performance Evaluation

  • (VI) Conclusion & Demo


Optical flow

Optical Flow

  • Optical Flow (OF) is a major task of biological and artificial visual systems

  • Relates the motion of pixel intensities between consecutive image frames

  • Optical Flow Constraint:

  • Gives only the normal flow

  • No OFC in untextured areas

u1

u2

u


Variational optical flow

Variational Optical Flow

  • First studied by Horn and Schunck in 1981 [1]

  • Quadratic regularization does not allow for discontinuities and occlusions

  • Modifying the Horn and Schunck functional was pioneered by Black and Rangarajan [2]

[1] B.K. Horn and B.G. Schunck. Determinig Optical Flow. Artificial Intelligence, 1981

[2] M.J. Black and P. Rangarajan. On the Unification of Line Processes, Outlier Rejection and Robust Statistics with

Applications in Early Vision, IJCV, 1996


Tv l 1 optical flow

TV-L1 Optical Flow

  • We use a robust variant of the Horn-Schunck formulation

    • Total Variation (TV) of Rudin Osher and Fatemi (ROF) [3]

    • L1 penalization of the OF constraint

  • TV-L1 has been used in many approaches

  • Allows for discontinuities in the flow field and outliers in the optical flow constraint

  • Sophisticated optimization techniques are needed

  • This is the major goal of this paper

[3] L. Rudin and S. Osher and E. Fatemi. Nonlinear Total Variation Based Noise Removal Algorithms, Physica D, 1992


An approximative formulation

  • Eθ E as Θ 0

An Approximative Formulation

  • Main difficulty is induced by the TV term –> ROF model [3]

  • Simple pointwise optimization problem -> Thresholding

[3] L. Rudin and S. Osher and E. Fatemi. Nonlinear Total Variation Based Noise Removal Algorithms, Physica D, 1992


Primal formulation

Primal Formulation

  • Study of the ROF model

  • Primal Euler Lagrange equations

  • Degenerated if gradient vanishes

  • Simple solution: Replace by

  • Disadvantage: Large εsmoothes edges!


Dual formulation

|p| ≤ 1

p

Dual Formulation

  • Studied by Chan [4], and later by Chambolle [5]

  • One arrives at two new equations

  • Dual Euler Lagrange Equations

  • Advantage: No regularization is needed!

[4] T. Chan and G. Golub and P. Mulet, A Nonlinear Primal Dual Method for TV-based Image Restoration, 1999

[5] A. Chambolle, An Algorithm for Total Variation Minimization and Applications, 2004


Primal vs dual

Primal vs. Dual

  • Convergence of the Primal and Dual formulation

    • Primal: fixed-point scheme of Vogel & Oman [6]

    • Dual: fixed-point scheme of Chambolle [5]

ε=10

ε=10-15

ε=10-1

ε=1

EROF

iterations

[5] A. Chambolle, An Algorithm for Total Variation Minimization and Applications, 2004

[6] C. R. Vogel and M. E. Oman. Iterative Methods For Total Variation Denoising. 1996


Final algorithm

Final Algorithm

  • Energy minimization is embedded into a coarse-to-fine approach to handle large displacements

  • Solved via alternating optimization

  • Fix v, minimize wrt. u (Chambolle‘s algorithm)

  • Fix u, minimize wrt. v (Thresholding)

  • Goto 1 until convergence


Implementation on graphics hardware

G92

Nov

2007

Implementation on Graphics Hardware

  • Particularly well suited to compute variational methods

    • High degree of parallelism

    • High performance processing units

  • All features can be accessed via C-like languages

  • Performance of graphics cards is steadily increasing


Performance evaluation

Performance Evaluation

Frames per second

Error evaluation on the well known Yosemite without clouds sequence

[1] B.K. Horn and B.G. Schunck. Determinig Optical Flow. Artificial Intelligence, 1981

[7] T. Nir and A.M. Bruckstein and R. Kimmel, Over-Parameterized Variational Optical Flow, IJCV 2007


Conclusion future work

Conclusion & Future work

  • We have developed a duality based algorithm for TV-L1 optical flow computation

  • We have implemented this algorithm on state-of-the-art graphics hardware

  • In summary, we obtained an optical flow algorithm having a realtime performance of ~45 fps for 512x512 images

  • Implementation in CUDA should give an additional speedup

  • More sophisticated data terms for illumination changes

  • Multigrid techniques for the dual formulation


A duality based approach for realtime tv l 1 optical flow

Demo


Solution of the rof model

Solution of the ROF model

  • Compute the minimizer of ROF model

    • Solution of a huge sytem of non-linear equations

    • Leads to iterative algorithms

  • Primal formulation:

    • Fixed-point scheme of Vogel and Oman [6]

  • Dual formulation

    • Fixed-point scheme of Chambolle [5]

[5] A. Chambolle, An Algorithm for Total Variation Minimization and Applications, 2004

[6] C. R. Vogel and M. E. Oman. Iterative Methods For Total Variation Denoising. 1996


  • Login