Image tagging
Download
1 / 41

image tagging - PowerPoint PPT Presentation


  • 271 Views
  • Updated On :

Image Tagging. Attaching textual meta-information or semantic linkages to images. By Perry Rajnovic. What is a Digital Image?. Digital Images are usually defined as an organized display of pixels, often called a bitmap.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'image tagging' - niveditha


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Image tagging

Image Tagging

Attaching textual meta-information or semantic linkages to images

By Perry Rajnovic


What is a digital image
What is a Digital Image?

  • Digital Images are usually defined as an organized display of pixels, often called a bitmap.

  • Each pixel is a numeric representation of the color intensities of that point.


What is a digital image1
What is a Digital Image?

  • Each pixel may be explicitly defined or be the result rendered by a vector or graphics package functionality.

  • These representations include no inherent textual elements or semantic description.

  • Due to the above, images are not easily machine-readable.


Why machine readability
Why machine-readability?

  • Most searches are done via textual queries, thus there must be a mechanism to link applicable keywords or phrases to images.

  • For blind persons, being able to convey information about the image in another medium would be good for accessibility.


Image contents
Image Contents

  • The contents of an image can be a full description written in prose (i.e. the adage “A picture is worth 1000 words”), or might simply have a few keywords describing spatial, temporal, or emotional aspects.

  • In many cases, accurately identifying the content of images requires human intervention.


Identifying image contents
Identifying Image Contents

  • Many good pattern recognition algorithms exist, however few are able to interpret the patterns extracted.

  • Artificial Intelligence algorithms can learn recognized patterns, but such a system’s flexibility is limited by its predefined knowledgebase.


Identifying text in images
Identifying Text in Images

  • CAPTCHAs (or Completely Automated Public Turing test to tell Computers and Humans Apart) are images which contain a distorted rendering of some text.


Identifying text in images1
Identifying Text in Images

  • Their goal is to provide an easy task for humans to do, but that is extremely hard for computer programs to perform equally.

  • For this task, OCR is generally not sufficient enough to extract the text.

  • This is a good example of why machine-readable information should be available.


Example tag contents

As an example of what might be provided to tag an image, to the right is a list of words and phrases to describe this slide’s header.

Navy Blue

Squares

Fade-out

Horizontal Bar

Minimalist

Decorative

Example Tag Contents


User applications
User Applications the right is a list of words and phrases to describe this slide’s header.

  • Many applications take advantage of image tagging, below are a few examples.

    • Apple iPhoto

    • Google Picasa

    • Adobe Photoshop Elements

  • Generally these programs use tagging for organization and user-defined searching.


Web applications
Web Applications the right is a list of words and phrases to describe this slide’s header.

  • Several Web-based applications are now including tagging for images, as well as other non-image based features.

    • Google ImageLabeler

    • Flickr.com

    • Facebook.com

    • 23hq.com

    • Fotki.com


Google images
Google Images the right is a list of words and phrases to describe this slide’s header.

  • Luis von Ahn developed the “ESP Game” which could be used to tag images.

  • He presented a Google tech talk about the game as a form of human computation.

  • Google later licensed the technology to create a similar web application called the Google Images ImageLabeler.


Google imagelabeler
Google ImageLabeler the right is a list of words and phrases to describe this slide’s header.

  • The ImageLabeler game allows to random users to generate tags that accurately describe images.

  • The tags should be accurate due to game constraints, and gain specificity after several rounds.

  • The computed tags can improve searches.


Flickr
Flickr the right is a list of words and phrases to describe this slide’s header.

  • Flickr is a “Web 2.0” photo hosting and sharing site.

  • Users are encouraged to upload photos, then to name, describe, tag, annotate, geotag, comment on, and group their photos in collaborative ways.


Flickr tags
Flickr - Tags the right is a list of words and phrases to describe this slide’s header.

  • Tags are words or phrases meant to act as keywords.

  • They are searchable within the site, and can show popular topics.

  • They improve search relevance.


Flickr geotagging
Flickr - Geotagging the right is a list of words and phrases to describe this slide’s header.

  • Geotagging is a term for adding geospatial metadata to images such as the latitude, longitude and other directional indications of where a photo was taken at.


What are annotations

Wikipedia defines them as: the right is a list of words and phrases to describe this slide’s header.

Extra information associated with a particular point in a document or other piece of information.

The US DoD defines them as:

A marking placed on imagery or drawings for explanatory purposes or to indicate items or areas of special importance.

What are annotations?


Annotating images
Annotating Images the right is a list of words and phrases to describe this slide’s header.

  • The use of annotations with images can provide several useful functions. Below are some examples:

    • Point out a specific piece of content.

    • Explain some icon or graphic.

    • Summarize the meaning of some region.

    • Provide additional information via text.


Flickr notes
Flickr - Notes the right is a list of words and phrases to describe this slide’s header.

  • Flickr provides a feature called Notes. It uses a Flash-based implementation of an annotation system.

  • You can dynamically size a rectangular region over a portion of the image, then attach a snippet of text to describe it.


Fotonotes
FotoNotes the right is a list of words and phrases to describe this slide’s header.

  • FotoNotes is a data format for annotating images.

  • Allows you to embed the metadata directly into the image files for portability.

  • Flickr’s Notes feature is inspired by this standard and accompanying visualization implementation.


Fotonotes more
FotoNotes - More the right is a list of words and phrases to describe this slide’s header.

  • It was developed by Greg Elin.

  • The homepage provides links to groups working with the standard.

  • Additionally, an implementation which works in most browsers is provided as-is for customization.


Facebook
Facebook the right is a list of words and phrases to describe this slide’s header.

  • Facebook.com has a tagging feature that is integrated with “My Photos”.

  • It allows you to add a textual descriptor (tag or person’s name) to a specific point in the image.

  • This allows the module to describe who or what are included in a specific album.


Facebook tag display
Facebook – Tag Display the right is a list of words and phrases to describe this slide’s header.

  • When the images are viewed, placing the mouse over a tag displays a fixed sized square indicating where the tag (person) is located within the image.

  • This enables users to identify objects by visual inspection or by matching the list of contained objects with their tag displays.


Facebook links
Facebook – Links the right is a list of words and phrases to describe this slide’s header.

  • Another capability incorporates the site’s concept of friends. If the person you tag is identified as your friend on the site, their name will link to their profile.

  • The site will also count this image in the “photos of” feature on their profile, allowing inclusion of photos added by other users.


Other image metadata mpeg
Other Image Metadata: MPEG the right is a list of words and phrases to describe this slide’s header.

  • The MPEG-7 standard is a “Multimedia Content Description Interface”

  • “MPEG-7 is not aimed at any one application in particular; rather, the elements that MPEG-7 standardises shall support as broad a range of applications as possible.”


Other image metadata adobe
Other Image Metadata: Adobe the right is a list of words and phrases to describe this slide’s header.

  • Adobe Systems created a new MetaData framework for images called XMP (Extensible Metadata Platform).

  • It is publicly documented, based on W3C standards, built on XML, and is designed to eliminate growing incompatibility for metadata storage.


Other image metadata iptc
Other Image Metadata: IPTC the right is a list of words and phrases to describe this slide’s header.

  • The International Press Telecommunications Council created standards for the interchange of news data over a decade ago.

  • These standards still persist in their IIM standard, as well as being usable in the newer XMP framework.


Improving clustering search interfaces

Improving Clustering Search Interfaces the right is a list of words and phrases to describe this slide’s header.

Joint Term Project

By Perry Rajnovic

and Mark Zalar


Term project
Term Project the right is a list of words and phrases to describe this slide’s header.

  • For my term project, I will be working with Mark Zalar to develop a new search engine interface

  • It will draw inspiration from all of the top search engines today, along with the enhancements now possible using emerging technologies.


Project goal
Project Goal the right is a list of words and phrases to describe this slide’s header.

  • The Goal of the project is to implement a search site that provides a highly usable interface for query refinement.

  • Our backend will use clustering mechanisms to allow for easy refocusing of search topics

  • Our frontend will use AJAX for flexibility.


Frontend design
Frontend Design the right is a list of words and phrases to describe this slide’s header.

  • The frontend will be designed with a technology known as Asynchronous JavaScript and XML (AJAX).

  • This technology allows the site designer to run unseen requests to the server and parse XML-based results in the scripting language for interactivity.


Frontend theory
Frontend Theory the right is a list of words and phrases to describe this slide’s header.

  • Most clustering based search solutions available today use minimally interactivity.

  • Our theory is that making the ability to harness the power of clustering dynamically as you refine your search will improve results as well as time necessary to finish a search.


Ajax functionality
AJAX Functionality the right is a list of words and phrases to describe this slide’s header.

  • Our site will use AJAX to dynamically reconfigure the clustering menu. This allows a quicker browsing of clusters to identify the optimal range of pages to search within.

  • The menu will also use a novel interface that shows sibling and parent clusters.


Ajax functionality 2
AJAX Functionality 2 the right is a list of words and phrases to describe this slide’s header.

  • The results will be displayed to the user with some animation.

  • This will help to alert users when changes are made to the order or set of results.

  • Another advantage of this is that users will be more aware of the difference between clusters as they browse them.


Search target
Search Target the right is a list of words and phrases to describe this slide’s header.

  • This search engine could target both websites and images.

  • Valid keywords improve content knowledge.

  • Clustering would be highly useful in finding an image with a desired scene or set of objects.


Example search creature
Example: search “creature” the right is a list of words and phrases to describe this slide’s header.

  • The engine might identify a general cluster of “animal” or “being”.

  • Animal might have more results, so the medium level clusters are shown for that.


Example search creature1
Example: search “creature” the right is a list of words and phrases to describe this slide’s header.

  • User wants a general discussion of mammals. Selects that cluster.

  • The results change to focus on those related to mammals as a group and in specific.


Results display
Results Display the right is a list of words and phrases to describe this slide’s header.

  • To provide animation, a similar technology to that found in “TiddlyWiki” will be used.

  • This interface allows topics to be added and removed dynamically with animation.

  • Additionally, extra links can be attached to each topic for more functionality (open in new window, similar items, etc.)


Future enhancements
Future Enhancements the right is a list of words and phrases to describe this slide’s header.

  • Our implementation will provide a basic mockup of the interface and refinement techniques made available.

  • Several enhancements could be made to this interface that would improve its usability or functionality.


Enhancements in search
Enhancements in Search the right is a list of words and phrases to describe this slide’s header.

  • Taking advantage of a meta-search would allow the clustering algorithm to have a higher volume of data with which to generate data topologies to be explored.

  • Using adaptive search (by userID or global optimization) would improve clustering by choosing ones more often used.


Enhancements in interface
Enhancements in Interface the right is a list of words and phrases to describe this slide’s header.

  • Because the site will be AJAX based, a large amount of flexibility is possible with respect to changes in the interface.

  • The browser window is similar to a canvas, with all of the site’s underlying Document Object Model available for addition, modification or deletion.


ad