Commentary-based Video Categorization and Concept Discovery

1. Commentary-based Video Categorization and Concept Discovery By Janice Leung

2. Agenda • Introduction to Video Sharing Sites • Current Problem • Previous Works • Commentary-based Video Clustering • Conclusion • Future Works

3. Video Sharing Sites • Allows users to upload videos • Shares videos worldwide • Example: • Dailymotion • YouTube • MySpace

4. De Facto • YouTube • More than 65,000 new videos every day • 100 million videos views daily • 20 million unique visitors per month

5. Immense amount of videos Incredible growth of videos How to search for desired video? YouTube: Tags + simple Categorization

6. YouTube • Predefined categories • Videos • Title • Description • Tags • Category • Comments Provided by the one who uploads the video Provided by many users

9. Related Works • Classify videos: • Video features: color, grayscale histogram, pixel information • Keywords from description • Tags • Find user interests: • Object fetching information • Tags

10. Problems • Video features • Cannot tell exactly what the video is about • No users interest is considered • Keywords from description • Description provided by the one who uploaded the video • Not sufficient information

11. Problems (Cont.) • Tags • Not sufficient information • May reflect users feelings on videos but too brief to represent the complex idea of the videos • Object fetching information • Reflects users interests but no information about the videos at all

12. Video Categorization and Concept Discovery • Site: YouTube • Videos: involving Hong Kong singers

13. Comments Given by many users Can be large amount Express users opinions Rich words describe fine-grained level ideas Tags Given by only one person (the one who uploaded the video) Few tags Describe the video in a very brief way Singer name Song name Comment vs Tag

15. Comments • Include: • Video content • Music styles • Music ages • Singer description • Appearance • Style • News etc.

16. Commentary-based Video Categorization • Objective: Categories videos based on user interests and discover the concept of videos • Cluster videos by using comments • Group videos based on user interests • Find video concepts • Clustering algorithm: multi-assignment NMF

17. Video clustering • Bi-clustering: videos and words • Clusters videos and words into k groups by matrix factorization • Video-word matrix X as input • Video-word matrix X is derived by tf-idf

18. Tf-idf • Term frequency (tf) • Suppose there are t distinct terms in document j where fi,jis the number of occurrence of term i in document j

19. Tf-idf (Cont.) • Inverse document frequency where N is the total number of documents in dataset and ni is number of documents containing term i

20. Tf-idf (Cont.) • Importance weight of term i to document j • Matrix X as input to NMF is defined as

21. Video Clustering (Cont.) • Decompose X into non-negative matrices W and H by minimizing where Ref. : Document Clustering Based On Non-negative Matrix Factorization (Xu et al SIGIR’03)

22. Video Clustering (Cont.) NMF decomposition for video clustering

23. Video Clustering (Cont.) • Suppose • Number of videos: N • Number of distinct terms: M • Threshold: β • W in size M x K • wn,k: coefficient indicates how video n belongs to cluster k

24. Video-cluster assignment • Videos can belongs to multiple groups • Multi-cluster assignment • Video n belongs to cluster k if • Set of clusters that video n belongs to: where K is set if all clusters

25. Video-cluster assignment (Cont.) • Threshold, β • Many irrelevant videos for each cluster • Coefficient distribution varies for different clusters • Coefficient distribution dependant • Different for different clusters

26. Concept Discovery • Matrix H in size of K x M • hk,m: how likely term m belongs to cluster k • Term belongs to a cluster describes the videos in that cluster • Concept words of cluster k videos • Top 10 words of cluster k

27. Experiment • 19305 videos • 102 Hong Kong singers • 7271 users • Number of cluster, k: 20

28. Experiment (Cont.) • Threshold, β • Coefficient distribution dependant • Threshold for cluster i is defined as

29. Experiment (Cont.) • Video coefficients may distribute in an extremely uneven manner • Cause poor result • To compensate, threshold can be set as

30. Experiment (Cont.) C1 C2 C3 V1 0.65 0.65 0.23 0.23 0.12 0.12 V2 0.35 0.35 0.64 0.64 0.01 0.01 V3 0.65 0.65 0.22 0.22 0.13 0.13 V4 0.05 0.05 0.64 0.64 0.31 0.31 V5 0.00 0.00 0.30 0.30 0.70 0.70

31. Experiment (Cont.)

32. Experiment (Cont.)

33. Experiment (Cont.)

34. Concept Words vs Tags

35. Concept Words vs Tags Percentage of videos with tags covering concept words across groups

36. Singer Relationship Discovery • Comments on videos may talk about singers • Singer styles, appearance, news • Singer clustering using comments • Reveals relationships between singers • Discovers hidden phenomenon

37. Singer Relationship Discovery (Cont.)

38. Conclusion • Captures user interests more accurately and fairly than that of the human predefined categories • Categories can be changed dynamically, user interest changes from time to time • Obtain clusters with fine-grained level ideas • Ease the task of video search by categorizing videos and refining index

39. Future Works • Extend to user clustering • Obtain relationships videos, singers and users of the entire social network • Study the social culture • Ease the job of advertising to target customers • Connect people who share the same interests

40. Q & A Questions?