1 / 28

Optimization & Learning for Registration of Moving Dynamic Textures

Optimization & Learning for Registration of Moving Dynamic Textures. Junzhou Huang 1 , Xiaolei Huang 2 , Dimitris Metaxas 1 Rutgers University 1 , Lehigh University 2. Outline. Background Goals & Problems Related Works Proposed Method Experiment Results Discussion & Conclusion.

joel-hewitt
Download Presentation

Optimization & Learning for Registration of Moving Dynamic Textures

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Optimization & Learning for Registration of Moving Dynamic Textures Junzhou Huang1, Xiaolei Huang2, Dimitris Metaxas1 Rutgers University1, Lehigh University2

  2. Outline • Background • Goals & Problems • Related Works • Proposed Method • Experiment Results • Discussion & Conclusion

  3. Background • Dynamic textures (DT) • static camera, exhibits a certain stationary • Moving Dynamic textures (MDT) • dynamic textures captured by a moving camera DT, [Kwatra et al. SIGGRAPH’03] MDT, [Fitzgibbon ICCV’01]

  4. Background • Video registration • Required by many video analysis applications • Traditional assumption • Static, rigid, brightness constancy • Bergen et al. ECCV’92, Black et al. ICCV’93 • Relaxing rigid assumption • Dynamic textures • Doretto et al. IJCV’03, Yuan at al. ECCV’04, Chan et al. NIPS’05, Lin et al. PAMI’07, Rav-Acha at al. Workshop at ICCV’05

  5. Our Goals • Registration of MDT • Recover the camera motion and register the image sequences including moving dynamic textures Left Translation Right Translation

  6. Complex Optimization Problems • Complex optimization • Camera motion, dynamic texture model • Chicken-and-Egg Problems • Challenges • About the mean images • About LDS model • About the camera motion?

  7. Related Works • Fitzgibbon, ICCV’01 • Pioneering attempt • Stochastic rigidity • Non-linear optimization • Vidal et al. CVPR’05 • Time varying LDS model • Static assumption in small time window • Simple and general framework but under estimation

  8. Formulation • Registration of MDT • I(t), the video frame • camera motion parameters • y0 , the desired average image of the video • y(t), related with appearance of DT • x(t), related with dynamics of DT

  9. Generative Model Generative image model for a MDT

  10. First Observation • Good registration • a good registration according to the accurate camera motion should simplify the dynamic texture model while preserving all useful information • Used by Fitzgibbon, ICCV’01, Minimizing the entropy function of an auto regressive process • Used by Vidal, CVPR’05, optimizing time varying LDS model by optimizing piecewise LDS model

  11. Second Observation • Good registration • A good registration according to the accurate camera motion should lead to a sharp average image whose statistics of derivative filters are similar to those of the input image frames. • Image statistics • Student-t distribution / heavy tailed image priors • Huang et al. CVPR’99, Roth et al. CVPR’05

  12. Prior Models • The Average image priors • The motion priors • The dynamic priors

  13. Average Image Priors • Student-t distribution • Model parameters / contrastive divergence method (a) Before registration, (b) in the middle of registration (c) after registration

  14. Motion / Dynamic Priors • Gaussian Perturbation (Motion) • Uncertainty in the motion modeled by a Gaussian perturbation about the mean estimation M0 / the covariance matrix S ( a diagonal matrix.) • Motivated by the work [Pickup et al. NIPS’06] • GPDM / MAR model (Dynamic) • Marginalizing over all possible mappings between appearance and dynamics • Motivated by the work [Wang et al. NIPS’05] [Moon et al. CVPR’06]

  15. Joint Optimization • Generative image model • Optimization • Final marginal likelihood • Scaled conjugate gradients algorithm (SCG)

  16. Procedures • Obtaining image derivative prior model • Dividing the long sequence into many short image sequences • Initialization for video registration • Performing model optimization with the proposed prior models until model convergence. • With estimated y0, Y and X, the camera motion is then obtained

  17. Obtaining Data • Three DT video sequences • DT data, [Kwatra et al. SIGGRAPH’03] • Synthesized MDT video sequence • 60 frames each, no motion from 1st to 20th frame and from 41st to 60th • Camera motions with speed [1, 0] from 21st to 40th

  18. Grass MDT Video • The average image (a) One frame, (b) the average image after registration, (c) before registration

  19. Grass MDT Video • The statistics of derivative filter responses

  20. Evaluation / Comparison • False Estimation Fraction • Comparison with two classical methods • Hybrid method, [Bergen et al. ECCV’92] [Black et al. ICCV’93] • Vidal’method, [Vidal et al. CVPR’05]

  21. Waterfall MDT Video • Motion estimation (a) Ground truth, (b) by hybrid method, (c) by Vidal’s, (d) proposed

  22. Waterfall MDT Video • The average Image and its statistics The average image and related distribution after registration by (a) proposed method, (b) Vidal’s method, (c) hybrid method

  23. FEF Comparisons • On three synthesized MDT video

  24. Real MDT Video • Moving flower bed video • Ours • 554 frames totally • Ground truth 110 pixels • Estimation 104.52 pixels ( FEF 4.98%) • Vidal’s • 250 frames • Ground truth 85 pixels • Estimation 60 pixels ( FEF 29.41%)

  25. Conclusions • What proposing: • Powerful priors for MDT registration • What getting out: • Camera motions • Average image • Dynamic texture model • What learning? • Registration simplify DT model while preserving useful information • Better registration lead to sharper average image

  26. Thanks !

  27. Thanks !

  28. Future Works • More complex camera motions • Different Metric functions for evaluation • Multiple dynamic texture segmentation

More Related