KDX: An indexer for support vector machines

1. KDX: An indexer for support vector machines Advisor : Dr. Hsu Presenter : Yu-San Hsieh Author : Navneet Panda, Edward Y. Chang 2006. TKDE.748-763

2. Outline • Motivation • Objective • Method • Experiments • Conclusions

3. Motivation • Many data mining and information-retrieval tasks query for the “top-k” best matches to a target concept. It would be to require a linear scan of the entire unlabeled pool.

4. Objective • To avoid a linear scan, we propose a kernel indexer (KDX) to work with SVM which could quickly converge on an approximate set of top-k instance of interest.

5. Method • KDX • KDX-Changing kernel parameters • KDX-Insertion and Deletion w hyperplane Fixed instance x2, x1 is any instances in dataset Replacing x2 by the central instance Changing parameter didn’t affect the order list in intra-reing index

6. Kernel function: Gaussian kernel Data Set: Corbis data set Result: change in kernel parameter and number of points in the ring don’t significantly affect KDX’s performance Experiments 65%↑ quality Discrepancy Dataset size Best performance K is large, the recall can approximate 100 percent.

7. Conclusions • We have presented KDX, a novel indexing strategy for speeding up top-k queries for SVMs. • The indexing structure was also shown to adapt to changing values of the parameter and number of points per ring.

8. My opinion • Advantage • KDX can speed up top-k queries without scanning all dataset • Changing kernel-parameter setting could not affect its performance • Drawback • Time complexity of computing inter-ring index is O(rg2) • Application • ……

9. Feature space Method KDX-create d b c a 1 2 4 1.Find the central instance φ(Xc). 2.Separate the instances. 3.Construct a intra-ring indexer for each ring. 4.Creating an inter-ring index. Central instance d X Intra-ring index a X X X c b Inter-ring index 1  a-max =1 a-inter-ring[1][0] = d 2 b-max =2 b-inter-ring[2][1] = a c-max = 4 c-inter-ring[3][1] = a d-max = 1 d-inter-ring[4][1] = a KDX-top_k 1.Computing θc. 2.Select the first identifying ring and a starting instance ψ(x). 3.Computing the angular separatio between ψ(x) and the farthest coordinate in the ring from the hyperplane ψ(x*). 4.Iteratively, replacing ψ(x) with a closer instance to ψ(x*). 5.Identifying a good starting instance ψ(x) for the next ring.

10. τ Method 5.Identifying a good starting instance ψ(x) for the next ring. Using the inter-ring index to find the next ring. 6.Until the top-k list is not improved after inspecting multiple ring 1.Computing θc. 3.Computing the angular separation between ψ(x) and the farthest coordinate in the ring from the hyperplane ψ(x*). 2.Select the first identifying ring randomly choose a starting instance ψ(x). w X r hyperplane 4.Iteratively, replacing ψ(x) with a closer instance to ψ(x*). Neighboring point ofψ(x) : x3, x1, x4, x5,x2, x6, x7, x8 Using PQ to pruning ψ(x7), ψ(x8) Resorted list of ψ(x) : x4, x5, x1, x3,x2, x6 Pruning ψ(x4), ψ(x5)  x1, x3,x2, x6 Using arc P’Q’ and anchor ψ(x1)  ψ(x3) ψ(x1) and ψ(x) are both selecting ψ(x3)

11. Method • KDX-Insertion and Deletion after before Insertion Symmetrically  No affect the result Asymmerically 1.The instance of interest does not lie in the most of the new instance lie, it does not affect the search for the most suitable instance. Deletion of instances 1.No affect the indexing scheme 2.Removing the corresponding row and column from the associative index  O(g) 3.Delection of instance ≠remove