1 / 27

Video Summarization via D eterminantal P oint P rocesses (DPP)

Video Summarization via D eterminantal P oint P rocesses (DPP). Boqing Gong University of Southern California Joint work with Wei-Lun Chao, Kristen Grauman, and Fei Sha. Background Basic idea of DPP Sequential DPP ( NIPS 2014) Large-margin training of DPP Conclusion. Background.

patriciaperez
Download Presentation

Video Summarization via D eterminantal P oint P rocesses (DPP)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Video Summarization viaDeterminantal Point Processes (DPP) Boqing Gong University of Southern California Joint work with Wei-Lun Chao, Kristen Grauman, and Fei Sha

  2. Background • Basic idea of DPP • Sequential DPP (NIPS 2014) • Large-margin training of DPP • Conclusion

  3. Background • Motivation: Indispensable for fast video browsing and retrieval • Representation: • Key frames / segments extraction • Subset Selection problem

  4. Background • Video summarization is hard: • Individual selected frame: Representativeness • Selected frames as a whole: Diversity • Naïve solution:Clustering • Competing !

  5. Background • Clustering works?

  6. Video summarization: an overview • Video summarization is hard: • What criteria lead to user perspective? • What kind of models: • Supervised learning ! • Diverse subset with representative items

  7. Background • How to model subset selection problem? • Structured prediction, submodular functions • Determinantal Point Processes (DPPs) [Alex Kulesza and Ben Taskar, 2012]

  8. Background • Basic idea of DPP • Sequential DPP (NIPS 2014) • Large-margin training of DPP • Conclusion

  9. Basic idea of DPP • Idea: A point process based on matrix determinant. • Formulation:M discrete items (binary decision)

  10. Basic idea of DPP • Why diverse? • Extreme cases:

  11. Basic idea of DPP • Learning in DPP: • 11

  12. Background • Basic idea of DPP • Sequential DPP (NIPS 2014) • Large-margin training of DPP • Conclusion

  13. Sequential DPP • Motivation: • The temporal structure of video is missing • Proposed Idea: • Sequential DPP via Markov properties

  14. … • …

  15. Sequential DPP • Modeling the sequential structure: • Conditional DPP: still a DPP !

  16. Sequential DPP • Parameterization:

  17. Inference and Learning • Inference: • Allow brute-force search in small chunks • Optimization:

  18. Sequential DPP • Experimental setting: • 3 datasets: OVP (50), Youtube (39), Kodak (18) • Fisher vectors + Saliency + Contextual features • Evaluation: Recall, Precision, and F1 score • Comparison: unsupervised methods & vanilla DPP

  19. Sequential DPP • Experimental Results:

  20. Sequential DPP • Experimental Results:

  21. Sequential DPP • Experimental Results:

  22. Background • Basic idea of DPP • Sequential DPP (NIPS 2014) • Large-margin training of DPP • Conclusion

  23. Learning parameters in DPP • Maximum likehood estimation • Focuses on observed data only • Large-margin training • Maximizes margin between observed and undesired data • Discriminative learning • More flexible: incorporating evaluation metrics

  24. Large-margin training of DPP • More discriminative and flexible

  25. Conclusion • Supervised learning for video summarization • DPPs: modeling diversity subset selection • Video structure: Sequential DPP • Parameterization: Neural networks • Future work • Better inference algorithms • Models beyond DPP (submodular)

More Related