Collaborative Image-based Tagging and Interactive Mapping on Tabletops - PowerPoint PPT Presentation

Collaborative image based tagging and interactive mapping on tabletops l.jpg
Download
1 / 58

  • 286 Views
  • Uploaded on
  • Presentation posted in: Internet / Web

Collaborative Image-based Tagging and Interactive Mapping on Tabletops. Presented By: Glen Whitaker Supervisor: Judy Kay. Research Question. How to Support Tabletop interaction for organisation of Content? Why is it important? Novelty of User Interface Organising is fundamental

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Download Presentation

Collaborative Image-based Tagging and Interactive Mapping on Tabletops

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Collaborative image based tagging and interactive mapping on tabletops l.jpg

Collaborative Image-based Tagging and Interactive Mapping on Tabletops

Presented By: Glen Whitaker

Supervisor: Judy Kay


Research question l.jpg

Research Question

  • How to Support Tabletop interaction for organisation of Content?

  • Why is it important?

    • Novelty of User Interface

    • Organising is fundamental

      • Especially file systems

  • Constraints?

    • Limited methods of input

  • Challenges?

    • No established toolkits, primitives, UI design

    • No guidelines

    • Research - Cruiser Tabletop as exploratory framework


Refined research question l.jpg

Refined Research Question

  • How to Support Tabletop interaction for organisation of Content using Tagging and Maps?

  • Why?

    • Tagging text is similar to associating file name and location to files in a file system

    • Maps similar concept to the folder structure in a file system


Sub problem how to support tagging at tabletop l.jpg

Sub-Problem: How to Support Tagging at Tabletop

  • Adding Tags to Images

  • Linking Images to maps

  • Single text/image tags

  • Multiple Tags


Refinement elements of tagging l.jpg

Refinement: Elements of Tagging

  • Way to organise and find digital artefacts

  • Key interface tasks for tagging

    • Defining tags

    • Adding tags to objects

    • Viewing tags

    • Searching tags

    • Deleting tags

  • How to achieve this on Tabletop Interfaces?


Approach 1 mytags l.jpg

Approach 1: MyTags

  • Inspired by TeamTag

    • Morris, M. R., A. Paepcke, T. Winograd, and J. Stamberger. "TeamTag: Exploring Centralized Versus Replicated Controls for Co-Located Tabletop Groupware." Proceedings of the SIGCHI conference on Human Factors in computing systems CHI: ACM Press, 2006. 1273 - 82.

  • Text-Based Tagging

  • Adding Text to Images

  • Adding Multiple Text to Images


Approach 2 mymaps l.jpg

Approach 2: MyMaps

  • Image-based Tagging in context of maps

  • Adding Images to Map Locations

    • Placing image on a single map location/region

  • Adding Images to Multiple Map Locations

    • Placing an image to multiple regions on map


Prototype l.jpg

Prototype


Defining tags l.jpg

Defining Tags

  • Tags are predefined (externally in a file/database)

  • Tags represented by special objects


Adding tags l.jpg

Adding Tags


Viewing tags l.jpg

Viewing Tags

  • MyTags

    • Flip image over to view list of text

  • MyMaps

    • Dwell on tag to view image

  • Viewing is same for single and multiple tags in both MyTags and MyMaps


Deleting tags l.jpg

Deleting Tags

  • Move tag to blackhole

  • Same for tags in both MyMaps and MyTags

  • Same for single and multiple tags


The framework l.jpg

The Framework

  • Cruiser

    • Images

    • Blackhole (garbage collection object)

  • Tagging Primitives

    • Adding Text Tags

    • Adding Image based tags

    • Adding Regions

    • View Tags

    • Delete Tags


Previous elements l.jpg

Previous Elements

Blackhole

Images

  • Previous Elements used by MyMaps:

    • Blackhole (garbage collection object)

    • Images (used to represent exhibits)

      • Audio can be associated with image

      • Images can have other images placed on back


New elements l.jpg

New Elements

  • “New” elements of the Cruiser Tabletop

    • Map Resources

    • Map Region Creation Tool

    • “Pin Images” similar to pins on a notice board

    • Image Region Identification

Map

Back of Image

Region

Pin Images

Text Attachment

Region Tool

Text Object


Architecture l.jpg

Architecture

= Trent’s Cruiser Elements

= New Elements


Evaluation l.jpg

Evaluation

  • Performance

    • Successful Tagging with MyTag

    • Successful Tagging with MyMaps

  • Other usability elements

    • Comparison of affordances

    • Learnability

    • Error Rate

    • Error Recovery

  • Collocated Collaboration UI aspects

    • Duplicated Maps

    • UI Facilitates Collaboration

  • Evaluate Effects of Short term memory load

    • Images

    • Maps


Evaluation design l.jpg

Evaluation Design

  • Qualitative Evaluations

  • 6 Pairs (User A, User B)

  • Asked to design their own museum

  • Double cross over condition (TeamTag and MyMaps + MyTags)

  • User A and B given different sets of text attached to 10 images

  • Questionnaire for effective issues:

    • Preference, attitudes, perspective


Evaluation steps l.jpg

Evaluation Steps

  • Tutorial

  • Task 1

    • Design virtual Museum

  • Task 2

    • Free-form virtual Museum

  • Tutorial of cross-over system

  • Repeat Tasks 1 and 2 for cross over system

  • Questionnaire

Encourage Collaboration in Tagging Context

Evaluate Collaboration and usability without design limitations

Double cross-over to control first system effect


Slide20 l.jpg

= Represents System used Second (always faster)


Slide21 l.jpg

= Represents System used Second

= Preferred Method

X


Slide22 l.jpg

10/12 Preferred Second System

2 preferred MyMaps even as first system

= Represents System used Second

= Preferred Method

X


Slide23 l.jpg

2 People (1 Pair) liked both

Others evenly split

Some commented regions suited task better and less planning needed

= Represents System used Second

= Preferred Method

X


Slide24 l.jpg

5 People (2 Pairs + 1) liked but hardware limitations prevented effective use

= Represents System used Second

= Preferred Method

X


Slide25 l.jpg

All participants liked Image Tagging and could use comfortably

= Represents System used Second

= Preferred Method

X


Slide26 l.jpg

Manly yes, participants lost tags creating objects

Pair 1 preferred flexibility to organise

= Represents System used Second

= Preferred Method

X


Performance results l.jpg

Performance Results

  • Successful Tagging with MyTag

    • Association of Text to objects performed with low error rate

    • Pairs completed tasks successfully using method

  • Successful Tagging with MyMaps

    • Association of Images was performed with low error rate

    • Pairs completed tasks successfully using method


Other usability elements l.jpg

Other usability elements

  • Swipe action for text tagging was liked and not really in the design

  • Learnability

    • Easy to learn tagging primitives

  • Error Rate/Recovery

    • Low error rate for Swipe/movement text association technique

      • Errors were due to having multiple images flipped within path of movement

    • Low error rate for image tags on maps

      • Errors were mainly a result of lack planning suitable location

    • Image Tags

      • Errors fixed by deletion of reference point and further planning

    • Text Tags

      • Errors fixed by deletion of attached text


Collocated collaboration l.jpg

Collocated Collaboration

  • Duplicate Maps

    • Participants did not use replicated maps as expected Hardware limitations - multiple use - affected this

    • 3 out of 12 participants used second map as a thumbnail view of the focus map

  • UI Facilitates Collaboration

    • Participants shared text objects

    • Participants collaboratively planned image tag locations

    • Participants collaboratively planned text tags for each image


Effects on short term memory load l.jpg

Effects on Short-Term Memory Load

  • Images

    • Images are flipped in tagging, causing short-term memory load but avoiding confusion

    • Images could not be semi-transparent since it would conflict with other functions

  • Maps

    • Dwelling needed to view images relating to tags

    • image tag design reduced clutter and confusion about map objects as the cost of the need to check what reference points represented

  • Participants were observed to have minor problems remembering tags with the small sample of images and text


Caveats l.jpg

Caveats

  • Hardware

    • Mimio capturing technology used for Cruiser only allows one interaction at a time

  • Software

    • Objects are 2-Dimensional representations

    • Text predefined in external file loaded at start up

    • Precision of interaction depends on accuracy of calibration


Future work l.jpg

Future Work

  • Handwriting for tag inputs

  • Backend database for efficiency (minimise I/O)

  • Personalised recommendations based on tags

  • Link tagging to file system – (Anthony Collins thesis)


Contribution l.jpg

Contribution

  • Significant progress in understanding how to support collocated collaborative planning at a tabletop

  • Insight into new gestures for image tagging and text tagging on tabletops

  • Swipe gesture discovered


Slide34 l.jpg

No more slides. Click to continue.


Results l.jpg

Results

  • Was 6 pairs enough?

    • Sample showed consistent patterns relating to Image Tagging

    • Provided substantial usability, performance, collaboration and affordance results

    • More detailed and extensive evaluations required for further results


Background virtual museum implementation l.jpg

Background: Virtual Museum Implementation

  • Virtual Museums

    • Digital Technology being used to display exhibits while preserving the contents of an exhibit in digital format

    • Augmented Reality and 3D Modelling advancements allow photorealistic exhibits

  • Huang, C., C. Chen, and P. Chung. "Projects in VR: Tangible Photorealistic Virtual Museum." IEEE Computer Graphics and Applications: IEEE, 2005. pp 15 - 17.


Direct touch interfaces l.jpg

Direct Touch Interfaces

  • DiamondTouch Tabletop

    • Direct-Touch Based Interaction

    • Unique identification based on circuits through chairs/pads

  • Other Touch based tables include: SmartSkin, Entertaible and Misto

  • Ryall, K., M.R. Morris, K. Everitt, C. Forlines, and C. Shen. "Experiences with and Observations of Direct-Touch Tabletops." First IEEE International Workshop on Horizontal Interactive Human-Computer Systems: IEEE, 2006.


Lumisight rear projection design l.jpg

Lumisight – Rear Projection Design

  • Lumisight Tabletop

    • Projectors inside the table project different images in different directions

    • Number of users limited to number of projectors

  • Kakehi, Y., T. Hosomi, M. Iida, T. Naemura, and M. Matsushita. "Transparent Tabletop Interface for Multiple Users on Lumisight Table." First IEEE International Workshop on Horizontal Interactive Human-Computer Systems: IEEE, 2006.


Teamsearch l.jpg

TeamSearch

  • TeamSearch

    • Use of special widgets to represent metadata shared by all users

    • Search images through use of widgets

    • Search through manually tagged images using widgets

  • Morris, M.R., A. Paepcke, and T. Winograd. "Teamsearch: Comparing Techniques for Co-Present Collaborative Search of Digital Media." First IEEE International Workshop on Horizontal Interactive Human-Computer Systems: IEEE, 2006. pp 97 - 104.

Widget

Personal Space

Query Objects

Metadata inside widgets


Teamsearch40 l.jpg

TeamSearch

  • Added Tags

    • Tags in the original implementation were added manually to each image

  • Searching

    • Users search the photos by placing query object onto the metadata values

    • Thumbnails of matching photo’s appear in that user’s personal space

  • Tags are

    • Metadata that matches the contents of the widgets


Teamtag l.jpg

TeamTag

  • TeamTag

    • Presentation and association of metadata

    • Centralised-control (TeamSearch approach) and Replicated-control design

  • Morris, M. R., A. Paepcke, T. Winograd, and J. Stamberger. "TeamTag: Exploring Centralized Versus Replicated Controls for Co-Located Tabletop Groupware." Proceedings of the SIGCHI conference on Human Factors in computing systems CHI: ACM Press, 2006. 1273 - 82.

Centralised-Controls Design

Replicated-Controls Design


Teamtag42 l.jpg

TeamTag

  • Centralised-controls design

    • Implementation of TeamSearch widgets

    • Users attach metadata by tapping image, then subsequently selecting a piece of metadata in a widget

  • Replicated-controls design

    • Presents metadata categorised into columns that are presented in front of each user

    • Users attach metadata by tapping image, then subsequently selecting a piece of metadata in the collection presented in front of them


Teamtag43 l.jpg

TeamTag

  • Added Tags

    • Tapping the image to be tagged and the subsequent piece of metadata selected by that user is attached

    • Multiple tags can be associated to images using this method

  • Tags are

    • Defined sets of metadata

  • Viewing Tags

    • Popup lists present what metadata is associated with an image

    • Popup lists activated by a user touching an image with two fingers


Centralised vs replicated l.jpg

Centralised Vs Replicated

  • Evaluated in context of zoo, labelling each image with appropriate metadata in groups of 4

  • 19 out of 24 subjects preferred replicated metadata interface

  • Collaborative evaluation of replicated interface showed on average 1.24 users contributed to the tagging of an image

  • Group collaboration and communication levels presented no significant difference


Google maps l.jpg

Google Maps

  • Google Maps

    • Interactively move around map of world

    • Zoom in on satellite and road maps of areas around the world

    • Identify businesses on the maps by providing business info to Google to tag based on street address

    • WikiMapia provides user defined region tagging

  • Google Maps Tour. Google. Available: http://www.google.com/help/maps/tour/. 10th of August 2006.


Google maps46 l.jpg

Google Maps

  • Added Tags

    • WikiMapia – Drawing of a box on the map to indicate a region

    • Google Maps – Entering of data with the location being labelled by the system based on data inputted about a business

  • Searching

    • Searches performed by comparing text inputted to the metadata that is associated with a tag

  • Tags

    • WikiMapia – regions are user defined boxes with a text tag identifying what the region is

    • Google Maps – identifies single addresses with business details associated with that address (limited to USA, Japan, UK, China and Canada)

  • Point Vs Region

    • WikiMapia allows user identification of regions

    • Google maps allows identification of single addresses on maps


The cruiser l.jpg

The Cruiser

  • Cruiser Tabletop

    • Uses Mimio® to Capture Interaction

    • Uses Pen stylus devices for interaction

    • Number of users limited to Pen Numbers

  • MyMaps and current software on the cruiser can be implemented using DiamondTouch interfaces

  • Trent Apted’s First Year of PhD presentation on Photo sharing with Sharepic system and the Cruiser Tabletop


Map resources l.jpg

Map Resources

  • Contribution

    • Explore how collaborative planning activities can be performed on tabletop interfaces

    • Provides a context for image based tagging

  • Transparency levels represent the interactive state

  • Maps are replicated for the number of users

Interactive Map

Non-Interactive Map

Map With some Tags

Walls in Museum


Regions l.jpg

Regions

  • Contribution

    • Categorisation through areas on map objects

    • Evaluate the effectiveness of planning through the identification of regions

  • Using a region creation tool to identify these regions

  • Image tagged to maps within regions inherit any text for those regions

Attached Regions

Region Creation Tool

E.g. Greek

E.g. Roman


Image tags l.jpg

Image Tags

  • Contribution

    • Evaluation of the effectiveness of image based tagging on collaborative interfaces

    • Evaluate how this assists in planning tasks on tabletop interfaces

  • Assigned by performing a gesture on the image to be attached while placed on top of the map

Map with Image Tags of 2 different Users

Images


Text association l.jpg

Text Association

  • Contribution

    • Compare swipe (MyMaps) to tapping (TeamTag)

    • Compare Map-based tagging and TeamTag approach to tag Images

  • Attach text by dragging over the back of images and regions

Back of Image

Text Attachment

Text Object


Image based tagging l.jpg

Image Based Tagging

  • Metadata stored in text format within a designated folder

  • Metadata stored for images and regions:

    • Maps

    • Regions

    • Rotations

    • Locations

    • Text

    • Scale

  • Each entry of a location in the tag files for each image and region represents a tag on a designated map

    ##LOCATION

    <map>: <x coordinate>, <y coordinate>, <z coordinate>

    <map>: <x coordinate>, <y coordinate>, <z coordinate>


Image based tagging53 l.jpg

Image Based Tagging

  • Tags are represented by special “Pin Images” which are small (at the moment) blue spherical-like images that resemble a pin

  • Each “Pin Image” relies on a designated rotation entry and location entry

    ##ROTATION

    <radian value of rotation>

    <radian value of rotation>

    ##LOCATION

    <map>: <x coordinate>, <y coordinate>, <z coordinate>

    <map>: <x coordinate>, <y coordinate>, <z coordinate>

  • Images that are linked to regions are also indirectly associated to the map that the region is linked to


More video l.jpg

More Video


Other tabletop research l.jpg

Other Tabletop Research

  • Benko, H., A.D. Wilson, and P. Baudisch. "Precise Selection Techniques for Multi-Touch Screens." SIGCHI conference on Human Factors in computing systems ACM Press, 2006. pp 1263 - 72.

  • Ha, V., K. Inkpen, R. Mandryk, and T. Whalen. "Direct Intentions: The Effect of Input Devices on Collaboration around a Tabletop Display." First IEEE International Workshop on Horizontal Interactive Human-Computer Systems: IEEE, 2006.

  • Ryall, K., K. Everitt, C. Forlines, and C. Shen. "Multispace: Enabling Electronic Document Micro-Mobility in Table-Centric Mult-Devie Environments." First IEEE International Workshop on Horizontal Interactive Human-Computer Systems: IEEE, 2006.

  • Scott, S., M. Sheelagh, T. Carpendale, and K. Inkpen. "Territoriality in Collaborative Tabletop Workspaces " Proceedings of the ACM conference on Computer supported cooperative work ACM Press, 2004. 294 - 303.

  • Tang, A., M. Tory, B. Po, P. Neumann, and S. Carpendale. "Collaborative Coupling over Tabletop Displays." SIGCHI conference on Human Factors in computing systems CHI: ACM Press, 2006. pp 1181 - 90.


More tags l.jpg

More Tags …

  • Wikimapia. Available: http://www.wikimapia.org. 10th of August 2006.

  • Toyama, K., R. Logan, and A. Roseway. "Geographic Location Tags on Digital Images." Proceedings of the eleventh ACM international conference on Multimedia ACM Press, 2003. pp 156 - 66.

  • Marlow, C., M. Naaman, D. Boyd, and M. Davis. "Ht06, Tagging Paper, Taxonomy, Flickr, Academic Article, to Read." The seventeenth conference on Hypertext and hypermedia ACM Press, 2006. pp 31 - 40.

  • Del.Icio.Us. Available: http://del.icio.us/. 3rd of September 2006.

  • Google Maps. Google. Available: http://maps.google.com/. 10th of August 2006.

  • Morris, M. R., A. Paepcke, T. Winograd, and J. Stamberger. "Teamtag: Exploring Centralized Versus Replicated Controls for Co-Located Tabletop Groupware." Proceedings of the SIGCHI conference on Human Factors in computing systems CHI: ACM Press, 2006. 1273 - 82.


References l.jpg

References

  • Koshizuka, N., and K. Sakamura. "Tokyo University Digital Museum." International Conference on Digital Libraries: Research and Practice Kyoto, 2000. pp 85 - 92.

  • WikiMapia. Google. Available: http://www.wikimapia.org. 10th of August 2006.

  • Kakehi, Y., T. Hosomi, M. Iida, T. Naemura, and M. Matsushita. "Transparent Tabletop Interface for Multiple Users on Lumisight Table." First IEEE International Workshop on Horizontal Interactive Human-Computer Systems: IEEE, 2006.

  • Huang, C., C. Chen, and P. Chung. "Projects in VR: Tangible Photorealistic Virtual Museum." IEEE Computer Graphics and Applications: IEEE, 2005. pp 15 - 17.

  • Morris, M. R., A. Paepcke, T. Winograd, and J. Stamberger. "Teamtag: Exploring Centralized Versus Replicated Controls for Co-Located Tabletop Groupware." Proceedings of the SIGCHI conference on Human Factors in computing systems CHI: ACM Press, 2006. 1273 - 82.


References58 l.jpg

References

  • Morris, M.R., A. Paepcke, and T. Winograd. "Teamsearch: Comparing Techniques for Co-Present Collaborative Search of Digital Media." First IEEE International Workshop on Horizontal Interactive Human-Computer Systems: IEEE, 2006. pp 97 - 104.

  • Trent Apted’s First Year of PhD presentation on Photo sharing with Sharepic system and the Cruiser Tabletop

  • Ryall, K., M.R. Morris, K. Everitt, C. Forlines, and C. Shen. "Experiences with and Observations of Direct-Touch Tabletops." First IEEE International Workshop on Horizontal Interactive Human-Computer Systems: IEEE, 2006.

  • Google Maps Tour. Google. Available: http://www.google.com/help/maps/tour/. 10th of August 2006.


  • Login