Mining Social Images for Automated Tagging with Distance Metric Learning

1. Peilin Zhao¹ zhao0106@ntu.edu.sg Ying He¹ yhe@ntu.edu.sg Steven C.H. Hoi¹ chhoi@ntu.edu.sg ¹Nanyang Technological University, Singapore Introduction Algorithm CHARTS / GRAPHS / IMAGES Fig. the process of a retrieval-based annotation approach by mining social images with distance metric learning Mining Social Images with Distance Metric Learning for Automated Image Tagging Convergence Analysis Fig. Example of automatically tagging a novel image by UDML. UDML • Basic Ideas of UDML • Exploit both visual and textual contents of social images. • Unify both inductive and transductive metric learning techniques. Experimental Results Pengcheng Wu¹ wupe0003@ntu.edu.sg Fig. Average precision at top t annotated tags under 11 methods • Tagging Images with Optimized Metrics • Retrieve k-nearest neighbors of the novel unlabeled image. • Calculate the frequency of each candidate tag associated with the k-nearest social images. • Assign the unlabeled image with tags of high frequency and smallaverage distance. Fig. Average precision under different top k similar images used Fig. [Examples showing the tagging results by 11 different methods. Fourth ACM International Conference on Web Search and Data Mining(WSDM 2011)