1 / 29

NLPainter “Text Analysis for picture/movie generation”

NLPainter “Text Analysis for picture/movie generation”. David Leoni Eduardo C á rdenas 11 /01/2012. Motivation for choosing the project:. The purpose of our project is to transform text in images trying that both express the same mining. More than 50% of human brain is devoted to vision

lis
Download Presentation

NLPainter “Text Analysis for picture/movie generation”

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. NLPainter “Text Analysis for picture/movie generation” David Leoni Eduardo Cárdenas 11/01/2012

  2. Motivation for choosing the project: • The purpose of our project is to transform text in images trying that both express the same mining. • More than 50% of human brain is devoted to vision • A fact mere text can't exploit, no matter how inspired and well written it is. • Adding illustrations to text can be of great help to memorize its contents • But searching images that represent the text is a time consuming task • Drawing entirely new images from scratch takes even longer.

  3. How the problem can be solve? In order to solve this problem we are going to use different techniques like text mining, natural language processing and semantic web: We obtained a big Image database. We have image withtags with the things that are inside of them. We selected the most representative picture in our database that describes a specific object.

  4. How the problem can be solve? We used some text mining techniques in order to obtain entities, attributes, etc. We usedthe PoS of the phrase that we want to convert to image. We associated the text with the images.

  5. Why an Ontology? • We need an ontology to hold implicit knowledge which might not be present in the text provided by the user • Examples: • an animal is in the tundra • We don’t have pictures tags that tell us animal, so we use an ontology in order to know that a lion is an animal. • Which are the usual animals living in the tundra?

  6. Databases The following databases of images was used forour project: • LabelMe • images are annotated with the shapes of the objects contained in the scene. • labeling was done by unpaid users • More than 70,000 shapes where obtained! • Animal Diversity Web • we fetched nearly 10000 pages. • 1545 were information about animals. • 3500 picture pages of animals (and for each picture page we extracted ~5 pics links) and 5000 were simply the pages about the hierarchy, needed to arrive to the information at the leaves • we fetched mammals,reptiles ,birds, bony fishes, insects, echinoderms, arthropods

  7. Databases • LabelMe

  8. Databases • Animal Diversity

  9. State of the Art • The Story Picturing Engine • A Text-to-Picture Synthesis System for Augmenting Communication • WordsEye

  10. Part of Speech STANFORD This software is a Java implementation of the log-linear part-of-speech taggers. The English taggers use the Penn Treebank tag set. Have been improved its speed, performance, usability, and support for other languages.

  11. Stanford Project

  12. Ontology:

  13. Ontology:

  14. Ontology:

  15. Ontology:

  16. General Diagram:

  17. Specific Diagram (Text): 17

  18. Specific Diagram (Images): 18

  19. Specific Diagram (Images): 19

  20. Technologies and algorithms(Text) • Programmation Language: • Java • Programming Environment: • Netbeans • Packages: • Stanford Parser • Additional Packages: • Image Generator 20

  21. Technologies and algorithms(Image) • Programmation Language: • MATLAB, Java • Programming Environment: • Netbeans • Packages: • LabelMe • XOM • Image Processing (Crop, class, merge class, etc. ) 21

  22. Technologies and algorithms(Ontology) • Editor: • Protégé 4.1 • RDF engine: • OWLimLite • Upper ontology: • Wordnet 22

  23. Technologies and algorithms(General project) • Programmation Language: • Java • Programming Environment: • NetBeans • RDF engine: • OWLIM lite • Packages: • XOM • Image Processing (awt) • Web server: • Apache Tomcat 7.0 • JSP 23

  24. Technologies and algorithms(General project) • Documentation: • Google Wiki • Versioning: • SVN • Project Web Page: • http://code.google.com/p/nlpainter/ 24

  25. WEB PAGE: • / 25

  26. Our Project working! • / 26

  27. Let se it works!!!

  28. Questions?

  29. References: • [LM] Bryan C. Russell and Antonio Torralba and Kevin P. Murphy and William T. Freeman}, Labelme: A database and web-based tool for image annotation, MIT AI Lab Memo, 2005 • [DBP] Christian Bizer, Jens Lehmann, GeorgiKobilarov, Sören Auer, Christian Becker, Richard Cyganiak, Sebastian Hellmann: DBpedia – A Crystallization Point for the We of Data. Journal of Web Semantics: Science, Services and Agents on the World Wide Web, Issue 7, Pages 154–165, 2009. • [TRA] Mihalcea, R., and Tarau, P. 2004. TextRank: Bringing order into texts. In Proc. Conf. Empirical Methods in Natural Language Processing, 404–411 • [CAPS] Ken Xu and James Stewart and Eugene Fiume , Constraint-Based Automatic Placement for Scene Composition, Proc. Graphics Interface, 2002,May, Calgary, Alberta, pp 25--34[ADW] Myers, P., R. Espinosa, C. S. Parr, T. Jones, G. S. Hammond, and T. A. Dewey. 2006. The Animal Diversity Web (online). Accessed November 01, 2011 at http://animaldiversity.org

More Related