Novel Visualization and Interaction for Small to Jumbo Displays

2. Novel Visualization and Interaction for Small to Jumbo Displays Mary Czerwinski Microsoft Research

3. Who Contributed: VIBE Team

4. Large Display Surfaces Are Here Workstation in the real world

5. Overview • Initial large display research • Prototypes around usability issues observed • Visualization and interaction • New user experiences have to scale the wide continuum of displays • Future directions

6. Harris Poll Responses (7/02, N=1197)

7. use multimon no multimon 32% 30% plan to use multimon 38% Multimon Usage Trends Peddie Research, 2001 (N>6000)

8. Why A Larger Display Surface? • Productivity benefits 10-30% (despite sw usability issues) • Users prefer more display surface • Pricesdropping • Footprints getting smaller

9. 1st Prototype--dSharp Display • Triple projection • matrox parhelia card • 3028 x764 resolution • 42 in. across • Slightly curved • 120 degree FOV

10. Task Times – Significant

11. User Satisfaction - Significant

12. Windows Layout - Significant

13. Tenets--Large Display UX • User studies show large display surfaces fundamentally change user interaction • Designed tools to better understand/complement how work practice changes • Large display surfaces provide non-linear productivity increases • Additional space has different utility • e.g. focal/peripheral displays provide different cues

14. But…Usability Issues • Why click to bring a clearly visible window into focus? caused many errors • Where is my cursor? • Where is my start button? • Where is my taskbar? • Where are my dialogs? • The software doesn’t know where the bezel is…

15. Vibelogger • 1st activity repository for studying windows usage in aggregate • can profile users based on display size • can be extended to visualize workflow and capture context • single user: capture task contexts to surface pertinent ui or provide reminders

16. colored block for each time point and app amount of shading indicates percentage of visibility of the window tasks subtasks Multitask Visualization

17. Task Switching Visualization • switching tasks (red to blue) • how are email windows arranged and used? • compare to...

18. Windows and Task Management Issues Emerge • Larger displays = more open windows • Multimon users arrange windows spatially • Taskbar does not scale: • aggregation model not task-based • users can’t operate on groups of related windows

19. Changes in Window Access Patterns

20. Input: Drag-and-Pop problem • large displays create long distance mouse movement • touch & pen input has problems moving between screen units solution • drag-and-pop brings proxies of targets to the user from across display surfaces • the user can complete drag interactions locally—no need to deal with distances or to cross display borders

21. Table Cloth • Problem: • User wants to access content physically far away • Solution: • Pan the desktop to user • Compress content to the right of focus • Grab content you need and snap back

22. Multitasking Support • Projectbar, layoutbar, groupbar • Scalable fabric • Task flasher • Secret passwords • Table cloth • Wincuts

23. Task Management: Groupbar • Taskbar for lightweight grouping of windows • Allows for multiple bars, spatial placement of bars • Desktop snapshotting; task snapshots

24. Task management: Scalable Fabric • Configurable central focus + periphery • Easy task switch from periphery to focus • Leverages human spatial memory

25. Task Flasher • A more visual alt + tab • Uses 3d scaling and selection animation • Windows stay on the monitor on which they are positioned

26. Secret Passwords • Problem: many touchscreen systems (eg. SmartBoard, TabletPC) have no keyboard • Use software virtual keyboard • Hard to hide password on a virtual keyboard • Previous research showed users think anything on large display is considered public…they watch (Tan et al, CHI ‘03)

27. Solution: Secret Passwords

28. Meeting Support: Wincuts • What about co-located collaborative work? • People bring different expertise and information on personal devices to meetings and need to share • today’s model is broken—only one person gets to display entire desktop at a time (for better or worse) • or, must share applications and files with others • what about private information? • What about UI that should be scaled for the task at hand?

29. Visualization Research • DateLens with Ben Bederson • Facetmap • Team Tracks (SW Vis)

30. Datelens with Ben Bederson • Fisheye representation of dates • Compact overviews • User control • Integrated search • Integrated with outlook • Pen-enabled

31. FacetMap

32. FacetMap • Scalable Visualization for “all of your stuff” • Queries MyLifeBits SQL database interactively • Domain includes web pages, digital images, documents, email, SenseCam physical activity data, VibeLog PC activity data • Heterogeneous data, metadata, annotations, and relationships organized into “facets” • Browsing and searching accomplished by iterative selection from among available item attributes (and/or full text search) • Uses Piccolo.Net (on top of GDI+) for graphics & animation

33. Scalability • Scales in three ways: • Space: Recursive, space-filling algorithm generates useful views for any size screen • Items: Large numbers of items aggregated/abstracted into groups and counts at multiple levels • Facets: New item attributes can be added to the visualization dynamically • Fixed minimum node size to guarantee readability • Consistent information density • Greater levels of detail are collapsed until they are feasible to present, given display constraints

34. Medium Size (No Filters)

35. Medium Size (Filters Active)

36. Small Size

37. Large Size (Wall Display)

38. FacetMap for Mobile Phones • Quick searching through structured data • Scalable UI • Half visual and half text list • Prototyping stage (desktop) • Multiple visualizations (pick the best for a given results set)

39. SW Visualization: Team Tracks • Goal: help developers new to a code base familiarize themselves quickly • Based on logs of teams using the code base • Assumption: most frequently visited areas of the code are more important • We empirically verified this • Show developers most related areas of code based on where they currently are • Also give them previews of that code

40. Screenshot of Team Tracks

41. Evaluation Results • Better task completion rates • 9 / 9 completed tasks 1 and 2 (same) • 3 / 9 completed task 3 (versus 1 / 7) • dominated by algorithmic detail • 7 / 9 completed task 4 (versus 1 / 7) • dominated by finding relevant code fragments • Better quiz scores (t(16)=-2.04, p<.03, one-tailed) • Same importance ratings (r=0.45, p=.02)

42. Future VIBE Directions • Novel interaction and visualization techniques that scale from small to very large displays • Continued evaluation and iteration of designs from a user-centered perspective • Automatic task identification • For more information: http://research.microsoft.com/research/vibe

43. Thank You!http://research.microsoft.com/research/vibe