Online Manifold Regularization: A New Learning Setting and Empirical Study

1. Online Manifold Regularization: A New Learning Setting and Empirical Study Andrew B. Goldberg, Ming Li, and Xiaojin Zhu.(ECML PKDD, 2008). Hu EnLiang Friday, April 17, 2009

3. Standard online learning VS. Online Manifold Regularization • Both of them are long-life learning and learn non-iid sequentially; • Standard online learning: traditionally assumes that every input point is fully labeled, it cannot take advantage of unlabeled data; • Online MR: it learns even when the input point is unlabeled.

4. Online MR VS. batch MR (advantages) • Online MR scales better than batch MR in time and space; • Online MR achieves comparable performance to batch MR; • Online MR can handle concept drift; • Online MR is an “anytime classifier”, which continuously is being improved and its training is cheap.

5. The principle of online MR

8. The relationship of batch risk, instantaneous regularized risk and average instantaneous risk

11. How to accelerate online MR

20. Continue !!!

21. A BriefIntroduction to CBIR(Content-based Image Retrieval) Hu en liang Tuesday, April 08, 2008

22. Background:Content-based Image Retrieval • Properties: • Querying image according to user’s semantic-concepts. • Querying images according to image’s contents, such as: color, texture, shape, etc. • Hypothesis——similar contents means semantic affinity； • ‘Semantic gap’——semantic affinity doesn't means similar contents.

23. A prototype of feedback-based CBIR

27. Background: The Difficulty of ‘Semantic Gap’ • Key problems: • How to extract user’s semantic-concept (intention)? • How to bridge between content and semantic ? • Main methods: • Machine learning based RF (Relevance-Feedback); • The prior knowledge such as the historical logs.

28. How to Connect CBIR to ML? • (Semi-)supervised Metric Learning; • Manifold Learning, Dimension Reduction… • (Semi-)supervised Classification; • Active Learning; Co-training; • Assembling Classifier; • Ranking; …

29. Some Individual Characteristics for feedback-based CBIR • In contrast to typical ML, there are some special characteristics for RF-CBIR: • The problem of the small size sample; • The problem of asymmetrical training sample; • The online algorithm with real-time requirement;

30. Manifold Regularization (MR) Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. Mikhail Belkin, Partha Niyogi, Vikas Sindhwani. Journal of machine Learning Research 7, pp 2399-2434, 2006

31. To Modify MR for CBIR • There are some intrinsic characteristics for CBIR: • The problem of the small size sample; • The problem of asymmetrical training sample; • The online algorithm with real-time requirement; The (1+x)-manifolds hypothesis There only single submanifold for positive class, but multi-submanifolds for negative class!

32. positive manifold Negative manifold The Problem of MR for the Multi-Submanifolds Case

33. The Bias-MR Focusing on Single-Submanifold

34. A review of LapSVM

35. A review of LapSVM

36. A higher efficiency in BLapSVM

37. The BLapSVM Algorithm for CBIR

38. The ‘BEP’ Performance Chart

39. The ‘Efficiency’ Performance Chart

40. Thanks for Your Attention !