1 / 11

Visiwords

Visiwords. John Tait Chief Scientific Officer. Warning. A few half formed ideas from the world of image and video indexing which may be of interest to MT people Not original ideas (apart from I think the link) In fact a line of work which derives originally from MT. The nub.

rosina
Download Presentation

Visiwords

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Visiwords John Tait Chief Scientific Officer

  2. Warning • A few half formed ideas from the world of image and video indexing which may be of interest to MT people • Not original ideas (apart from I think the link) • In fact a line of work which derives originally from MT

  3. The nub • Unsupervised Clustering of bundles of features • Colour, texture from image segments • Words, phrases from sentences or paragraphs ? • Associate these bundles with “translations” by supervised machine learning • Categorised images • Parallel texts

  4. Origins • “Matching Words and Pictures”: Barnard, Duygulu, Forsyth, de Freitas, Blei, Jordan. Journal of Machine Learning Research 3 (2003) 1107-1135 • “Image Classification Using Hybrid Neural Networks” Tsai, McGarry and Tait. Proceedings of the 26th ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2003), Toronto, July, 2003. pp 431-432.

  5. More or less general clusters General Concepts Specific Concepts

  6. Visiwords • Derived from Visiterms • These feature cluster nodes • A notion of an area of the “semantic field” • Remember these are colour, texture etc. for an area of an image …. No relation to language … or at least a very deep one

  7. Matching L2 General Concepts Examples L1 General Concepts L1 Specific Concepts L2 Specific Concepts

  8. Fast Forward to 2009 • Better statistical models tuned to the data • Much Bigger vocabularies of words categories • … and lots of other advances

  9. A question • Is there anything like this current MT research ?

  10. Concluding remarks • I’m surprised this worked at all • Why should image data be coherent and cohesive ? • But text is !!! • Is this a better way to deal with unknown and changing vocabulary

  11. Some other references • A Correlation Approach for Automatic Image Annotation Hardoon, D., Saunders, C., Szedmak, S. and Shawe-Taylor, J. (2006) A Correlation Approach for Automatic Image Annotation. In: Second International Conference on Advanced Data Mining and Applications, ADMA 2006, August, Xi'an, China. • Kucuktunc, O., Sevil, S. G., Tosun, A. B., Zitouni, H., Duygulu, P., and Can, F. 2008. Tag Suggestr: Automatic Photo Tag Expansion Using Visual Information for Photo Sharing Websites. In Proceedings of the 3rd international Conference on Semantic and Digital Media Technologies: Semantic Multimedia (Koblenz, Germany, December 03 - 05, 2008). D. Duke, L. Hardman, A. Hauptmann, D. Paulus, and S. Staab, Eds. Lecture Notes In Computer Science, vol. 5392. Springer-Verlag, Berlin, Heidelberg, 61-73. DOI= http://dx.doi.org/10.1007/978-3-540-92235-3_7

More Related