Finding Celebrities in Billions of Web Images 云飞 2012-12-13
Overview • 一、label an input image with a list of celebrities. • 二、the celebrity names are assigned to the faces by label propagation on a facial similarity graph.
Overview • 本文的优点： • 1、the proposed image annotation system is capable of labeling names to general web images. • 2、our name assignment algorithm does not impose any assumption on the facial feature distribution. • 3、not only visual cues are used.
Overview • 1. determine, by identifying celebrity names from surrounding text. • 2. given a set of names, assign the names to the faces in the input image.
Overview • A. Image Annotation System • 1) construct a vocabulary; • 2) discover all webpages hosting its near-duplicates; • 3) use the vocabulary to filter the surrounding text. • Advances： • 1)effective; • 2)remove noise. • Annotated images: • 1)SFSN • 2)SFMN • 3)MF
Overview • B. Multimodal Name Assignment • The context likelihood incorporates the information from surrounding text by using the confidence scores estimated by the image annotation system.
IMAGE ANNOTATION SYSTEM • Goal: label an input image with a list of celebrities who may appear in the image. • A. Constructing a Large-Scale Celebrity Name Vocabulary • B. Discover Related Webpages by Near-Duplicate Image Retrieval • C. Annotating Images by Mining Surrounding Text of Related Webpages
IMAGE ANNOTATION SYSTEM • Constructing a Large-Scale Celebrity Name Vocabulary 1)Wikipedia 首段 信息框 标签 2)Entitycube
IMAGE ANNOTATION SYSTEM • B. Discover Related Webpages by Near-Duplicate Image Retrieval • divide and conquer strategy • 图片分成n×n • 降维 • 阈值化
IMAGE ANNOTATION SYSTEM • C. Annotating Images by Mining Surrounding Text of Related Webpages • 1) Type of names； • 2) Type of surrounding text； • 3) Frequency versus ratio；
MULTIMODAL NAME ASSIGNMENT • A. Notation • B. Overview of the Assignment Model • C. Label Propagation from SFSN Images p(Y|F) • D. Constrain the Propagation by a Context Likelihood p(Y|T; λ) • E. Normalization by Name Prior p(Y) • F. Implementation Detail: Face Representation
A. Notation • faces in image In • denote the face labels as
B. Overview of the Assignment Model • the confidence for label
C. Label Propagation from SFSN Images p(Y|F) • how to propagate labels from SFSN images to SFMN and MF images
D. Constrain the Propagation by a Context Likelihood p(Y|T; λ) • 1) For each image-level name vk, generate a binary variable zk from p(vk |T) as defined in (3) to indicate whether vk appears in image I. • 2) If zk=1, generate mk faces of name vk in image I from p(m|z; λ) as defined in (13).
E. Normalization by Name Prior p(Y) • p(Y) represents the prior of names.
F. Implementation Detail: Face Representation • the appearance of each face is described by local binary pattern (LBP). • the face image is divided into small regions from which the LBP features are extracted and concatenated into a single feature histogram. • pply PCA to reduce the dimension of face descriptor from over 3000 to 500 dimensions.