1 / 41

Relative Attributes

Relative Attributes. Mingxia Liu mingxialiu@nuaa.edu.cn. Outline. 1. Introduction 2. Relative Attributes [1] 3. Discussion 4. Our Intent Work. [1] Devi Parikh and Kristen Grauman.Relative Attributes. ICCV 2011 Best Paper. Traditional Recognition. Tiger. ???. Dog. Chimpanzee.

ursa
Download Presentation

Relative Attributes

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Relative Attributes Mingxia Liu mingxialiu@nuaa.edu.cn

  2. Outline • 1. Introduction • 2. Relative Attributes[1] • 3. Discussion • 4. Our Intent Work [1] Devi Parikh and Kristen Grauman.Relative Attributes. ICCV 2011 Best Paper.

  3. Traditional Recognition Tiger ??? Dog Chimpanzee Tiger

  4. Attributes-based Recognition Attributes provide a mode of communication between humans and machines! Striped Black White Big Furry White Black Big Striped Yellow Dog Chimpanzee Tiger [Lampert 2009] [Farhadi 2009] [Kumar 2009] [Berg 2010] [Parikh 2010] … Zero-shot learning Describing objects Face verification Attribute discovery Nameable attributes …

  5. Binary or Relative Attributes?

  6. Binary or Relative Attributes? “Smiling”=0 “Smiling”=1 ??? Smiling less than Gao, more than Ge

  7. Binary or Relative Attributes? Horse Donkey Mule

  8. Attributes for Mule Has four-legs Is furry Tail longer than donkeys’ Legs shorter than horses’ Has tail [Oliva 2001] [Ferrari 2007] [Lampert 2009] [Farhadi 2009] [Kumar 2009] [Wang 2009] [Wang 2010] [Berg 2010] [Branson 2010] [Parikh 2010] [ICCV 2011] …

  9. Binary Has four-legs Is furry Tail longer than donkeys’ Legs shorter than horses’ Has tail

  10. Relative Has four-legs Is furry Tail longer than donkeys’ Legs shorter than horses’ Has tail

  11. Relative Attributes • Enhanced human-machine communication • More informative • Natural for humans

  12. Outline • 1. Introduction • 2. Relative Attributes • 3. Discussion • 4. Our Intent Work

  13. Contents • Relative attributes • Allow relating images and categories to each other • Learn ranking function for each attribute • Novel applications • Zero-shot learning from attribute comparisons • Automatically generating relative image descriptions

  14. 1) Relative Attributes Annotation Attribute coast (C), forest (F), highway (H), inside-city (I), mountain (M), open-country(O), street (S) and tall-building (T) 8 categories, 6 attributes

  15. 2) Learning Relative Attributes For each attribute , supervision is “open”

  16. 2) Learning Relative Attributes Learn a scoring function Image features Learned parameters that best satisfies constraints:

  17. 2) Learning Relative Attributes Ranking SVM Image  Relative Attribute Score Based on [Joachims 2002]

  18. 2) Learning Relative Attributes Wm 1 2 3 Rank Margin 4 5 6

  19. 3) Relative Zero-shot Learning • N : total categories • N = S + U • S : ‘seen’ categories • training images are provided relative attribute relation such as “lions are larger than dogs, as large astigers, but less large than elephants” • U : ‘unseen’ categories • training images are not provided

  20. 3) Relative Zero-shot Learning • How to relate Seen categories and Unseen categories through relative attributes?

  21. 3) Relative Zero-shot Learning • For attribute m •  •  •  • If attribute m is not used to describe  to be the mean of all training image -

  22. 3) Relative Zero-shot Learning • Step1: Compute the attribute rank score: • Step2: Compute the mean and covariance of according to relative attribute description. • Step 3: Assign class label for using the Maximum Likelihood Method: 

  23. 3) Relative Zero-shot Learning Need not use all attributes, or all seen categories. Age: Scarlett Scarlett Hugh Clive Clive Hugh Miley Jared Miley Jared Smiling:

  24. 3) Relative Zero-shot Learning Age: Scarlett Hugh Clive S Miley Jared Miley Clive Smiling: Smiling Clive J Hugh Scarlett H Jared Miley Infer image category using max-likelihood Age Can predict new classes based on their relationships to existing classes – without training images

  25. 4) Automatic Relative Image Description Novel image Density Conventional binary description: not dense Dense: Not dense:

  26. 4) Automatic Relative Image Description Novel image Density less dense than more dense than

  27. Experiments

  28. Datasets Public Figures Face (PubFig) [Kumar 2009] 8 classes 800 images 11 attributes: white, chubby, etc. Outdoor Scene Recognition (OSR) [Oliva 2001] 8 classes 2700 images 6 attributes: open, natural, etc.

  29. Ranker vs. Classifier – – – + + +

  30. Baselines • Zero-shot learning • Binary attributes: Direct Attribute Prediction [Lampert 2009] • Relative attributes via classifier scores • Automatic image-description • Binary attributes

  31. Relative Zero-shot Learning Rel. att.(ranker) Rel. att. (classifier) Binary attributes An attribute is more discriminative when used relatively

  32. Relative zero-shot learning Classifier score Binary attributes Proposed

  33. Automatic Relative Image Description Binary (existing): Not natural Not open Has perspective Relative (ours): More natural than insidecity Less natural than highway More open than street Less open than coast Has more perspective than highway Has less perspective than insidecity

  34. Automatic Relative Image Description Binary (existing): Not natural Not open Has perspective Relative (ours): More natural than tallbuilding Less natural than forest More open than tallbuilding Less open than coast Has more perspective than tallbuilding

  35. Automatic Relative Image Description Binary (existing): Not Young BushyEyebrows RoundFace Relative (ours): More Young than CliveOwen Less Young than ScarlettJohansson More BushyEyebrows thanZacEfron Less BushyEyebrows than AlexRodriguez More RoundFace than CliveOwen Less RoundFace than ZacEfron (Viggo)

  36. Outline • 1. Introduction • 2. Relative Attributes • 3. Discussion • 4. Our Intent work

  37. Discussion • Relative attributes representation • Attribution relation learning

  38. Outline • 1. Introduction • 2. Relative Attributes • 3. Discussion • 4. Our Intent work

  39. Our Intent Work • Learning attributes’ relation automatically • Learning attributes’ relation with noise • Attribute selection • Instance selection for attributes

  40. Outline • 1. Introduction • 2. Relative Attributes • 3. Discussion • 4. Our Intent work

  41. Thank You

More Related