1 / 76

Coached Active Learning for Interactive Video Search

Coached Active Learning for Interactive Video Search. Xiao-Yong Wei , Zhen-Qun Yang Machine Intelligence Laboratory College of Computer Science Sichuan University, China. Coached Active Learning for Interactive Video Search. Xiao-Yong Wei, Zhen-Qun Yang Machine Intelligence Laboratory

oro
Download Presentation

Coached Active Learning for Interactive Video Search

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Coached Active Learning for Interactive Video Search Xiao-Yong Wei, Zhen-Qun Yang Machine Intelligence Laboratory College of Computer Science Sichuan University, China

  2. Coached Active Learning for Interactive Video Search Xiao-Yong Wei, Zhen-Qun Yang Machine Intelligence Laboratory College of Computer Science Sichuan University, China

  3. Coached Active Learning for Interactive Video Search Xiao-Yong Wei, Zhen-Qun Yang Machine Intelligence Laboratory College of Computer Science Sichuan University, China

  4. Coached Active Learning for Interactive Video Search Xiao-Yong Wei, Zhen-Qun Yang Machine Intelligence Laboratory College of Computer Science Sichuan University, China

  5. Search “car on road” with Google

  6. Search “car on road” with Google

  7. Similar to the Automatic Search task in TRECVID

  8. Interactive Search Relevance Feedback

  9. Interactive Search Query Modeling, i.e., to figure out what the searcher wants? Relevance Feedback

  10. Interactive Search Querying Strategy, i.e., how to keep the searcher’s patience on labeling? Query Modeling, i.e., what the searcher wants? Relevance Feedback

  11. Query Modeling is indeed a task of Space Exploration

  12. Search & Space Exploration

  13. Search & Space Exploration

  14. Feature Space with Unlabeled Instances

  15. Feature Space with Unlabeled Instances

  16. Space Exploration for Query Modeling How to explore effectively? Query Distribution Querying Strategy: which instances to query next (i.e., being presented to the searcher for labeling)?

  17. The Goal of Query Modeling in Terms of Space Exploration • A reasonable querying strategy • To find the Query Distribution (i.e., the distribution of the relevant instances) ASAP • To satisfy Searchers with Relevant Samples ASAP

  18. Active Learning with Uncertainty Sampling – A Popular and Effective Strategy

  19. Active Learning with Uncertainty Sampling The 1st Round – Training Classifier

  20. Active Learning with Uncertainty Sampling The 1st Round – Query the Searcher

  21. Active Learning with Uncertainty Sampling The 2nd Round – Training Classifier

  22. Active Learning with Uncertainty Sampling The 2nd Round – Query the Searcher

  23. Active Learning with Uncertainty Sampling The 3rd Round

  24. Active Learning with Uncertainty Sampling The 4th Round

  25. Active Learning with Uncertainty Sampling The 5th Round

  26. Active Learning with Uncertainty Sampling

  27. Active Learning with Uncertainty Sampling may not be a “kindred soul” with Video Search

  28. Active Learning with Uncertainty Sampling may not be a “kindred soul” with Video Search

  29. Active Learning with Uncertainty Sampling may not be a “kindred soul” with Video Search The spirit of Active Learning(to reduce labeling efforts) The goal of Query Modeling (to harvest relevant instances) Also found by A.G. Hauptmann and et al. [MM’06]

  30. The Dilemma of Exploration and Exploitation • Exploration: to exploremore unknown area and get more about the Query Distribution and to boost future gains • Exploitation: to harvestmore relevant instances and to obtain immediate rewards

  31. Our Idea – The Query Distribution, even Unknown, is Predictable

  32. The Idea of Coached Active Learning • Estimate the Query Distribution after each round to avoid the risk of learning on a completely unknown distribution

  33. The Idea of Coached Active Learning • Estimate the Query Distribution pdf of the underlying Query Distribution

  34. The Idea of Coached Active Learning • Estimate the Query Distribution • Query users with instances picked from both dense and uncertain areas of the distribution • Balance the proportion of the two types of instances

  35. The Idea of Coached Active Learning Modeling the Query Distribution Balancing the Exploration and Exploitation

  36. Modeling the Query Distribution

  37. Modeling the Query Distribution ? ? ? ? ? ? ? ? ? L+: An incomplete sampling of the Query Distribution

  38. Modeling the Query Distribution • An indirect way of estimating the pdf of the Query Distribution • Find training examples which are statistically from the same distribution as L+ • Use the pdf(s) of those training examples to estimate that of the query ? ? ? ? ? ? ? ? ?

  39. Modeling the Query Distribution • A practical implementation • Training: organize training examples into nonexclusive semantic groups • Testing: check L+with each group, see whether they are statistically from the same distribution using two-sample Hotelling T-Square Test • Testing: estimate Query Distribution with GMM ? ? ? ? ? ? ? ? ?

  40. Modeling the Query Distribution • Rationale of the implementation • Samples from the same distribution may carry similarsemantics • Semantic groups (which passed the test) then reflect the searcher’s query intention

  41. Modeling the Query Distribution

  42. Modeling the Query Distribution Distribution of the semantic group “road”

  43. Modeling the Query Distribution Distribution of the semantic group “road” Distribution of the semantic group “car” Distribution of the query “car on road”

  44. Balancing the Exploration and Exploitation

  45. Balancing the Exploration and Exploitation • The priority of selecting an instance x to query next (i.e., present to the searcher for labeling) Balancing Factor Exploitative Priority Explorative Priority

More Related