1 / 30

Interactively Discovery of Attributes Vocabulary

This study by Devi Parikh and Kristen Grauman explores traditional recognition of animals like dogs, chimpanzees, and tigers through attributes-based recognition. Attributes such as furry, white, black, big, stripped, yellow, help in applications like zero-shot learning. The research emphasizes the importance of discriminative and nameable attributes for effective communication between humans and machines. The study uses an interactive system to engage users in identifying candidate attributes and ensure proposals are both discriminative and nameable. Evaluation involves annotating candidates offline with thousands of responses, focusing on attributes like "black," "spotted," "green," and "smiling." Results show that a structured nameability space aids prediction, an active approach discovers more discriminative splits compared to baselines, and automatically generated descriptions enhance understanding. This interactive system aims to bridge the gap between humans and machines for attribute discovery.

cadee
Download Presentation

Interactively Discovery of Attributes Vocabulary

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Interactively Discovery of Attributes Vocabulary Devi Parikh and Kristen Grauman

  2. Traditional Recognition Dog Chimpanzee Tiger ???

  3. Attributes-based Recognition Furry White Black Big Stripped Yellow Stripped Black White Big Dog Chimpanzee Tiger

  4. Applications Zero-shot learning Attributes provide a mode of communication between humans and machines! A Zebra is… White Black Stripped Zebra Image description Stripped Black White Big

  5. Attributes Attributes are most useful if they are • Discriminative • Nameable

  6. Attributes Attributes are most useful if they are • Discriminative • Nameable

  7. Attributes Attributes are most useful if they are • Discriminative • Nameable

  8. Attributes Attributes are most useful if they are • Discriminative • Nameable

  9. Attributes Attributes are most useful if they are • Discriminative • Nameable

  10. Interactive System 1. Name: Fluffy 2. Name: x 3. Name: Metal … How do we show the user a candidate-attribute? How do we ensure proposals are discriminative? How do we ensure proposals are nameable?

  11. Attribute Visualization

  12. Attribute Visualization

  13. Ensure Discriminability Normalized cuts Max Margin Clustering

  14. Ensure Nameability 1. Name: Fluffy 2. Name: x 3. Name: Metal …

  15. Ensure Nameability 1. Name: Fluffy 2. Name: x 3. Name: Metal … Mixture of Probabilistic PCA

  16. Interactive System

  17. Evaluation • Outdoor Scenes • Animals with Attributes • Public Figures Face • Gist and Color features (LDA)

  18. Interactive System

  19. Evaluation • Annotate all candidates off-line “Black” … ~25000 responses

  20. Evaluation • Annotate all candidates off-line … ~25000 responses “Spotted”

  21. Evaluation • Annotate all candidates off-line … ~25000 responses Unnameable

  22. Evaluation • Annotate all candidates off-line … ~25000 responses “Green”

  23. Evaluation • Annotate all candidates off-line … ~25000 responses “Congested”

  24. Evaluation • Annotate all candidates off-line … ~25000 responses “Smiling”

  25. Results Structure exists in nameability space allowing for prediction Our active approach discovers more discriminative splits than baselines

  26. Results Comparing to discriminative-only baseline

  27. Results Comparing to descriptive-only baseline

  28. Results Automatically generated descriptions

  29. Summary • Machines need to understand us • Attributes need to be detectable & discriminative • We need to understand machines • Attributes need to be nameable • Interactive system for discovering attributes

  30. Thank you.

More Related