Web Standards and (the future of) User Interface Design

1. Seminario Martedì 22 aprile 2008, h. 15:00 Aula Seminari, seminterrato Dipartimento di Scienze dell'Informazione Bologna Web Standards and (the future of) User Interface Design Stéphane Sire (stephane.sire@epfl.ch) École Polytechnique Fédérale de Lausanne CGC/MEDIA

2. Plan Web Standards and (the future of) User Interface Design Part 1 Rich User Interaction Part 2 Iterative Design and supportive technologies Part 3 SVG and alike

3. Part 1 (the future) of User Interfaces Rich User Interaction

4. Evolution of User Graphical Interaction Styles Desktop Web Mobile Command Line WAP Point & Click (hypertext) Direct (1983) Manipulation HTML Forms WIMP Rich Desktop Application (RDA) Rich Internet Application (RIA) iPhone and alike (touch based) Rich User Interaction

5. Characteristics of Rich User Interaction • Better Graphics • Extensive use of Animations • Creative layout models • Borrowings from other interaction styles • Better feedback times (AJAX) A new vocabulary for design

6. Better Graphics (1) • Layers • Painter's model of rendering • Compositing operators • (Porter & Duff) • Clipping and masking • Applications • Layout based on layers (see later) • Non rectangular windows • Objects with holes • Layer's effects Source: Digistrip (CENA)

7. Better Graphics (2) • Color models • Alpha channel (RGBA) • Gradient colors • Texture filling • Applications • Light and shadows effects • Glossy effect • Reflection (cow.neondragon.net/stuff/reflection/) • etc

8. Better graphics (3) • Shapes & lines • Stroke styles • Joints • extremities • Bézier paths • Very efficient algorithm to draw it (De Casteljeau algorithm) • Applications • Rounded shapes

9. Better Graphics (4) • Pixel Effects • Anti-aliasing • Filters (blur, bump mapping, etc.) Source www.corzo.com Source www.treebuilder.de

10. Extensive use of animations Visual state transition User's guidance Source www.lemonde.fr (choose Journal Electronique) Source www.joehewitt.com/iui/samples/music.html System state change Source www.panic.com/goods/

11. Animation Tricks • "Exageration" • Pace • Application: Kinetic Scrolling (video on youTube) Author's made movie from www.laredoute.fr Author's made movie from www.youarethemnodel.com

12. New Layout Models (1) • Layered information Source maps.google.com Source www.natoora.com Source www.laredoute.fr

13. New Layout Models (2) • Contraction/dilatation • Accordion • Continuous surfaces • Linear (ribbon) : www.gucci.com • Two dimensions : www.goruneasy.com • Unconventional : www.etsy.com (choose Time Machine) Source www.yomiuri.co.jp Source www.goruneasy.com

14. New Layout Models (3) • Fixed layout + Animation • Scrolling Ribbon • more information with constant space within the page • Layers + Animation • Drawer windows • Docks Source www.tf1.fr

15. Borrowings from other styles • Direct manipulation • Drag & drop but still marginal • Post-WIMP techniques • Bifocal menus • Video-games • Rotative menus Source www.wat.tv Source www.amazon.com

16. Better Feedback Times with AJAX • No more page reload and full page refresh • Increase reactivity • Requires special tricks to get user's attention • Applications • Auto-suggestion input field (see Google Autosuggest) • Dynamical queries • allows some forms without no "Submit" button • or allows to pre-filter results before submitting Source www.trivop.com Source www.darty.com/nav/achat/telephonie/telephonie_mobile/telephone_portable/guide.html

17. Rich User Interaction (summary) • Graphical design • To increase intuitivity • To create emotions • Emotions increase perceived usability • See Don Norman last book "Emotional Design" • Requires multi-disciplinary teams • User experience (interaction) designer • Visual designer, Motion designer • Developers • etc (video on youTube)

18. Part 2 User Interface Design Iterative Design and supportive technologies

19. Product Development Process Reprinted from "Sketching User Experiences" (Buxton, 2007)

20. Iterative Design

21. Close to Agile Software Development • Agile Manigesto (source agilemanifesto.org) • Individuals and interactions over processes and tools • Working software over comprehensive documentation • Customer collaboration over contract negotiation • Responding to change over following a plan • See also Extreme Programming • Web version • the permanent Beta

22. Examples Source: Nielsen Source: Hiser Source: Alex Poole

23. Example from • Air Traffic Control Strip • DMAN project

24. Classical development cycle Executable Code Interaction, Visual, Motion, … Designers Sketches Storyboard Wireframes IA diagrams Users Programmers Browser Plugin Runtime Env. Feedback loop How to increase the number of iterations ?

25. Model Driven Development Cycle Graphical models Animation models Other models… Interaction, Visual, Motion, … Designers Users Executable code Browser, Plugin Runtime Env. Sketches Storyboard Wireframes IA diagrams Programmers

26. What is a model ? • Example from

27. How to represent a model ? • eXtensible Markup Language (XML) • <g id="Root"> • <g id="Back"> • <path id="Background"> • <g id="Zones"> • <g id="CatchZones"> • <text id="Callsign"> • <text id="RWY"> • <text id="Gate"> • <path id="Catch"> • <g id="DMAN"> • <text id="STA">

28. How to Edit a Model ? • Specialized editors • Example for graphics: Adobe illustrator SVG File

29. A short Example Interaction Designer Visual Designer Graphical Model File(s) Programmer: LET's DO IT ! Example from Adobe dev web site

30. Adding More Models…. Interaction Designer Finite State Machine Models Visual Designer Behavior Modelling with State Machines Graphical Model

31. Example from Paper Mockup Airnet Project Prototyping Visual Design Integration

32. Model Driven Summary • Models • Specialized editors • Easy to move from one platform to another • Code • Programming skills • Hard to maintain • Less portable Models (XML) Models Code Code

33. Part 3 Web Standards for User Interface Design SVG and alike

34. W3C standards (1) • Mainly "Models" • XML based languages • From "Draft" to "Recommendation" status • Models both for output and input modalities • HTML, XHTML, XForms, SMIL, MathML, VoiceXML, SCXML, InkML, EMMA, …, and SVG • Some "Code" too • Application Programming Interfaces (APIs) • DOM (3 levels) • API for manipulating models loaded into memory as trees • Many more APIs (events, load and save, progression, etc.) • XMLHttpRequest • Ajax programming

35. Web Standards and MDA Definition of Interactive Components Imperative Prog. Lang. XML Markup Only Today Web applications XML Future of applications Code Behind Instantiation of Interactive Components Today Desktop applications Imperative Prog. Lang. Markup Behind Code Only

36. W3C and Rich User Interaction (*) simplified with specialized toolkits (scriptaculous, jQuery, etc)

37. SVG History • Scalable Vector Graphic • Started in 1998 • 1st draft: February 1999 • SVG 1.1 recommendation 2003 • http://www.w3.org/TR/SVG11/ • SVG 1.2 Tiny candidate recommendation 2006 • http://www.w3.org/TR/SVGMobile12/ • SVG 1.2 Full to come later • Working Group with main software editors • Sun, Adobe, Apple, Microsoft, Corel, Ilog, HP, Canon, Autodesk, etc.

38. SVG vs. Flash (*) see http://www.w3.org/TR/SVG/minimize.html

39. Authoring Tools • Graphical Editors • Sodipodi, Illustrator, Corel Draw, OmniGraffle Pro, … • Specialized Editors • Ikivo Animator : animations • Automatic generation • Client-side Javascript generation • Examplefrom http://www.destatis.de/ • Client or server size XSLT generation • demo from www.treebuilder.de/default.asp?file=250484.xml • Automatic conversion from/to other formats • From MathML, AutoCAD, Visio, etc. • To PDF, PNG, etc. • Manual authoring :)

40. Players • Adobe to stop its plugin support in 2009… • Adobe to switch to Flash/AIR ? • … but becoming native in Web browsers • Opera, Safari, Firefox, … • open source rendering libraries (Cairo) • Not directly available in IE… • Microsoft to switch to XAML/WVG with silverlight plugin • Standalone players • Batik (Java) - contains a useful utility to generate SVG fonts • SVG Tiny 1.2 on mobile phones • www.bitflash.com/mobile_primer.html • www.ikivo.com/02player_svg.html • And more… • See list of implementations www.svgi.org/

41. Drawing Basic Shapes • Shapes • Rectangles <rect> • Cercles <circle> • Ellipses <ellipse> • Lines <line> • Polylines <polyline> • Polygons <polygon> • Text <text> • Attributes • x, y, width, height, rx, ry • fill, stroke, stroke-linecap, stroke-linejoin, stroke-miterlimit, stroke-width, … • opacity, stroke-opacity, fill-opacity • ... <rect x="200" y="300" width="90" height="40" fill="lime" stroke="blue" stroke-linejoin="bevel" stroke-width="3"/>

42. Drawing Arbitrary Shapes • Path <path d="path data" > • outline of a shape which can be filled, stroked, used as a clipping path • defined by connected lignes, arcs and curves. • Base attribute • "path data" : complex vocabulary • "," and EOL are allowed in "path data", spaces are optional between a command and a number (to compress data) <path d="M 100 100 L 300 100 L 200 300 z" fill="red" stroke="blue" stroke-width="1" />

43. Path data vocabulary • Each command is named by a letter (M: moveto L: lineto, etc.) followed by a coordinate, either • Absolute if capital (e.g. "M") • Relative to the previous point if lowercase (e.g. "m") • M and m: • Move the point to the given coordinate • Create a new sub-path starting at the origin • Syntax: M|m (x y) (x y)* • The second group (x y)* is interpreted as lineto commands • Ex: M100 100 200 200 • L and l: • Draw a line from the current point to the given coordinate and draw lines between subsequent given points • Syntax: L|l (x y) (x y)* • H and h (V and v): • Draw a horizontal line (vertical) from the current point to the current point incremented with the given number • Syntax: H|h (x) et V|v (y) • Z and z: • Close current subpath with the last point defined with M|m Cur point x M 100,100 L 200, 200

44. More path data vocabulary See details on http://www.w3.org/TR/SVG/paths.html

45. Defining reusable graphical components • Definition of a component • Group element: <g id="name"> • Container element for grouping graphics elements • Can be nested • Its attributes are inherited by its descendants • Instantiation of a component • <use xlink:href="#name"/> <g id="shape" stroke="blue"> <path width="1cm" d="M 0 50 35 25 70 50 Z" stroke-width="0.1cm" fill="none"/> <rect x="10" y="50" width="50" height="1cm" stroke-width="0.1cm"/> </g> <use xlink:href="#shape" x="140" y="20"/> <use xlink:href="#shape" x="70" y="60"/>

46. Declaring graphical components without drawing them • Everything that appears in a <defs> section will not be drawn • It will be available to be reused in other parts • This is a way to group resources (at the beginning of the file) <defs> <g id="shape" stroke="blue"> <path width="1cm" d="M 0 50 35 25 70 50 Z" stroke-width="0.1cm" fill="none"/> <rect x="10" y="50" width="50" height="1cm" stroke-width="0.1cm"/> </g> </defs> <use xlink:href="#shape" x="140" y="20"/> <use xlink:href="#shape" x="70" y="60"/>

47. Positioning graphics • Most elements can be positioned with x and y within their parent container • Groups <g> are also used to apply transformations • <g transform="translate(x,y)"> • Other transformations include rotate, skewX|Y, scale and matrix • This is equivalent to associating a transformation matrix with each group <g> (and to redefine a new coordinate system for the descendants) • Transformation are cumulative in nested groups • Current transformation matrix at each <g> • Obtained as the product of all transformation matrices up to the parent <g> <g transform="rotate(20, 200, 300)"> <rect x="200" y="300" width="90" height="40" fill="lime" stroke="blue" stroke-width="3"/> </g>

48. Painting and Coloring • Colors applies with different attributes to • Strokes (attribute "stroke") • Shapes interior (attribute "fill") • Colors are rendered by a paint server • The attribute URI reference selects a paint server • Solid colors (e.g. "orange" or "#50A619") • Gradients or Patterns (e.g. "url(#myGradient)") <defs> <linearGradient id="Gradient01"> <stop offset="20%" stop-color="#39F" /> <stop offset="90%" stop-color="#F3F" /> </linearGradient> </defs> <rect x="200" y="300" width="90" height="40" fill="url(#Gradient01)" stroke="blue" stroke-width="3"/>

49. Masking and Clipping • Masking • Attribute • mask="url(#mask)" • Draw #mask in an offscreen buffer • Convert it to gray level image • Composite with the masked layer Layer1 Layer2 Masked by Layer3 Layer3 Mask Not visible Result <g mask="url(#Mask)"> <use xlink:href="#logo" transform="scale(1 -1"/> </g>

50. Animating Graphics (2) • Each graphical attribute can be animated • animate element • Animation description based on SMIL2 (Synchronized Multimedia Integration Language level 2), W3C: http://www.w3.org/TR/smil20/ • Animation describing • Attribute value variation • Time variation • Including time effect (slow-in, slow-out, etc.) <rect x="10" y="10" width="200" height="20"> <animate attributeName="width" attributeType="XML" from="200" to="20" begin="0s" dur="5s" fill="freeze"/> </rect>