Finding Celebrities in Billions of Web Images - PowerPoint PPT Presentation

lukas
finding celebrities in billions of web images n.
Skip this Video
Loading SlideShow in 5 Seconds..
Finding Celebrities in Billions of Web Images PowerPoint Presentation
Download Presentation
Finding Celebrities in Billions of Web Images

play fullscreen
1 / 18
Download Presentation
Presentation Description
203 Views
Download Presentation

Finding Celebrities in Billions of Web Images

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. Finding Celebrities in Billions of Web Images 云飞 2012-12-13

  2. Overview • 一、label an input image with a list of celebrities. • 二、the celebrity names are assigned to the faces by label propagation on a facial similarity graph.

  3. Overview • 本文的优点: • 1、the proposed image annotation system is capable of labeling names to general web images. • 2、our name assignment algorithm does not impose any assumption on the facial feature distribution. • 3、not only visual cues are used.

  4. Overview • 1. determine, by identifying celebrity names from surrounding text. • 2. given a set of names, assign the names to the faces in the input image.

  5. Overview • A. Image Annotation System • 1) construct a vocabulary; • 2) discover all webpages hosting its near-duplicates; • 3) use the vocabulary to filter the surrounding text. • Advances: • 1)effective; • 2)remove noise. • Annotated images: • 1)SFSN • 2)SFMN • 3)MF

  6. Overview • B. Multimodal Name Assignment • The context likelihood incorporates the information from surrounding text by using the confidence scores estimated by the image annotation system.

  7. IMAGE ANNOTATION SYSTEM • Goal: label an input image with a list of celebrities who may appear in the image. • A. Constructing a Large-Scale Celebrity Name Vocabulary • B. Discover Related Webpages by Near-Duplicate Image Retrieval • C. Annotating Images by Mining Surrounding Text of Related Webpages

  8. IMAGE ANNOTATION SYSTEM • Constructing a Large-Scale Celebrity Name Vocabulary 1)Wikipedia 首段 信息框 标签 2)Entitycube

  9. IMAGE ANNOTATION SYSTEM • B. Discover Related Webpages by Near-Duplicate Image Retrieval • divide and conquer strategy • 图片分成n×n • 降维 • 阈值化

  10. IMAGE ANNOTATION SYSTEM • C. Annotating Images by Mining Surrounding Text of Related Webpages • 1) Type of names; • 2) Type of surrounding text; • 3) Frequency versus ratio;

  11. MULTIMODAL NAME ASSIGNMENT • A. Notation • B. Overview of the Assignment Model • C. Label Propagation from SFSN Images p(Y|F) • D. Constrain the Propagation by a Context Likelihood p(Y|T; λ) • E. Normalization by Name Prior p(Y) • F. Implementation Detail: Face Representation

  12. A. Notation • faces in image In • denote the face labels as

  13. B. Overview of the Assignment Model • the confidence for label

  14. C. Label Propagation from SFSN Images p(Y|F) • how to propagate labels from SFSN images to SFMN and MF images

  15. D. Constrain the Propagation by a Context Likelihood p(Y|T; λ) • 1) For each image-level name vk, generate a binary variable zk from p(vk |T) as defined in (3) to indicate whether vk appears in image I. • 2) If zk=1, generate mk faces of name vk in image I from p(m|z; λ) as defined in (13).

  16. E. Normalization by Name Prior p(Y) • p(Y) represents the prior of names.

  17. F. Implementation Detail: Face Representation • the appearance of each face is described by local binary pattern (LBP). • the face image is divided into small regions from which the LBP features are extracted and concatenated into a single feature histogram. • pply PCA to reduce the dimension of face descriptor from over 3000 to 500 dimensions.

  18. Evaluation