windows touch deep dive l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Windows Touch Deep Dive PowerPoint Presentation
Download Presentation
Windows Touch Deep Dive

Loading in 2 Seconds...

play fullscreen
1 / 45

Windows Touch Deep Dive - PowerPoint PPT Presentation


  • 632 Views
  • Uploaded on

CL13. Windows Touch Deep Dive. Reed Townsend Program Manager Microsoft Corporation. Agenda. Introduction Overview of Windows Touch API surface Real-world ISV questions and solutions Demo projects 3D manipulations Reimplementing Collage’s PhotoStrip control Wrap up.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Windows Touch Deep Dive' - aisha


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
windows touch deep dive
CL13

Windows Touch Deep Dive

Reed Townsend

Program Manager

Microsoft Corporation

agenda
Agenda
  • Introduction
  • Overview of Windows Touch API surface
  • Real-world ISV questions and solutions
  • Demo projects
    • 3D manipulations
    • Reimplementing Collage’s PhotoStrip control
  • Wrap up
touch framing the opportunity
Touch - Framing the Opportunity
  • What is touch?
    • It’s an input method
  • What does it mean to design for touch?
    • When designing for inputs, we’re really talking about user interface design (in the broad sense)
    • UI is often synonymous with visuals and the input and interaction half is forgotten
touch framing the opportunity4
Touch - Framing the Opportunity
  • 99% of design is done in the platform
    • Controls and APIs are the language of UI
    • “The limits of my language mean the limits of my world” – Ludwig Wittgenstein
    • Games (and sometimes web) follow a different model
  • What are the common controls for touch?

?

windows touch apis
Windows Touch APIs
  • Good/Better/Best platform model
    • Good:
      • What you get for free
      • Legacy, non-touch-aware apps
    • Better
      • Minimal investment
      • Typically WM_GESTURE, mouse message inspection
    • Best
      • Enables touch-optimized experiences!
      • WM_TOUCH, manipulations & inertia, and more
wm gesture
WM_GESTURE
  • Provides notifications when the user performs gestures over your window
  • Contains additional information like center of gesture and gesture-specific arguments

Typical sequence:

WM_TABLET_QUERYSYSTEMGESTURESTATUS

WM_GESTURENOTIFY

WM_GESTURE (GID_BEGIN)

WM_GESTURE (GID_PAN, GF_BEGIN)

WM_GESTURE (GID_PAN, no flags)

...

WM_GESTURE (GID_PAN, GF_INERTIA)

...

WM_GESTURE (GID_PAN, GF_END)

WM_GESTURE (GID_END)

real world questions and answers
Real-world Questions and Answers

WM_GESTURE

Q: I registered for WM_TOUCH but I no longer get WM_GESTURE. What gives?

A: Yes, WM_GESTURE and WM_TOUCH are exclusive in Win7.

Q: I’m implementing gestures in my app, but I stopped getting panning bounce feedback. Why?

A: Two possibilities: Panning messages are being accidentally eaten, or your app is providing custom handling of them but not calling the bounce feedback API.

getsystemmetrics
GetSystemMetrics()
  • GetSystemMetrics():
    • SM_DIGITIZER returns data about available digitizers on the system
    • SM_TABLETPC returns data about Tablet functionality
    • SM_MAXIMUMTOUCHES yields the largest number of contacts any of the available digitizers supports
mouse messages getmessageextrainfo
Mouse Messages & GetMessageExtraInfo()
  • GetMessageExtraInfo() returns the extra info associated with a message
  • Mouse up and down messages are tagged with a special signature indicating they came from touch or pen:
    • Mask extra info against 0xFFFFFF80
    • 0xFF515780 for touch, 0xFF515700 for pen
richer touch experiences

Richer Touch Experiences

demo

WM_TOUCH Applications

wm touch
WM_TOUCH
  • Provides raw touch input data, conceptually similar to mouse messages
  • Finger painting, custom gestures, feeding higher-level controls (e.g. Manipulations)

Typical sequence:

WM_TOUCH (first finger down)

WM_TOUCH (first finger moving)

...

WM_TOUCH (two fingers moving, third comes down)

WM_TOUCH (three fingers moving)

...

WM_TOUCH (one goes up, two fingers moving)

WM_TOUCH (two fingers moving)

...

WM_TOUCH (last finger goes up)

real world questions and answers12
Real-world Questions and Answers

WM_TOUCH

Q: How can I disable the press and hold gesture in my window?

A: WM_TABLET_QUERYSYSTEMGESTURESTATUS can be used to disable P&H, flicks, and tap feedback. (WM_TOUCH windows already disable flicks by default. SetGestureConfig() controls gestures available via WM_GESTURE.)

Q: I’m building a UI framework. How can I add touch support?

A: Start with WM_TOUCH. If you want pan/zoom/rotate, use manipulations on top of WM_TOUCH.

touch optimized experiences

Touch-Optimized Experiences

demo

WM_TOUCH, Manipulations, and More

manipulations inertia
Manipulations & Inertia
  • Manipulations:
    • Provides the full range of 2D affine transforms corresponding to user input
    • Enables multiple concurrent interactions
    • Can be fed by any source of raw data
    • Are a superset of WM_GESTURE support
  • E.g. Rotating and translating 3 photos simultaneously
typical flow
Typical Flow

On contact down:

  • Hit test against targets
  • (Add to map of contact ids and MPs)
  • MP->ProcessDown()

On contact moves:

  • (Look up mapping)
  • MP->ProcessMove()

On contact up:

  • (Look up mapping)
  • MP->ProcessUp()
  • (Kick off inertia)

Get callbacks on event interface

  • Apply transforms to targets
advanced manipulation topics
Advanced Manipulation Topics
  • Single-finger rotation
    • PivotPoint, PivotRadius properties
  • Rotation around a point
    • Generalization of single-finger rotation
  • Supporting mouse-based manipulations
  • All covered in the Platform SDK manipulations sample
inertia
Inertia
  • Grab initial state from manipulation processor in completed event
  • Set up deceleration properties
  • Set up borders/margins
  • Call Process() on a timer
demo projects
Demo Projects!
  • Demos:
    • 3D manipulations – An approach to mapping 2D onto 3D.
    • Reimplementing Collage’s PhotoStrip– Building custom touch controls
  • Examples of new ways to use the platform and new user interface options
3d manipulations
3D Manipulations
  • Set up 3D view, axes, objects
  • Add WM_TOUCH
  • Add manipulations
  • Map manipulations to 3D navigation
    • 2 finger camera operations, zoom
    • Spherical panning
  • Add inertia and other tweaks
3d manipulations20
3D Manipulations
  • Classes of note:
    • D3DXDriver
    • CComTouchDriver
    • Camera
      • Inherits from InertiaObj and _IManipulationEvents
  • Example data flow:
    • WM_TOUCH message arrives
    • CComTouchDriver::ProcessInputEvent()
    • Camera::ProcessInputEvent()
    • InertiaObj::ProcessInputEvent()
    • Manipulation Processor
    • Camera::ManipulationDelta()
    • Update the camera
building a photostrip control
Building a PhotoStrip Control
  • Part of a control framework
  • Respond to touch messages
  • Add constrained manipulations for panning
  • Add support for dragging in/out of control
  • Add API surface
  • Flesh out overlay and gallery
building a photostrip control22
Building a PhotoStrip Control
  • Window hierarchy:
  • Example data flow:
    • WM_TOUCH arrives
    • Hits overlay, routed to appropriate control
    • Photostrip gets message
    • Touch pans and triggers photo removal
    • Message to app window
    • Message to gallery
    • Photo is displayed in gallery

Overlay HWND

PhotoStrip1 HWND

Gallery HWND

PhotoStrip2 HWND

implementation notes
Implementation Notes
  • Smoothing input and manipulation data
  • Mixed input layers
    • 3D: manipulations + gestures
    • PhotoStrip: raw data + manipulations
  • Supporting multiple interactions across windows
    • Overlay + forwarding approach
    • Capture
    • Mediating between inputs
final thoughts
Final Thoughts
  • What are the common controls for touch?
    • We don’t know – they need to be defined
    • The existing controls were optimized for mouse and keyboard over decades
    • Help shape the future of touch user interfaces!
touch development roadmap
Touch Development Roadmap

.NET 4.0 & Surface Toolkit (Q1 2010)

Native

Win32

Application

WinFormsApplication

WPF Application

Surface Application

Surface SDK

1.0

Surface Multi-Touch Controls &API

Multi-Touch Controls & API

Multi-Touch Controls

Surface Toolkit for Windows 7

Managed Wrapper and Interop

WPF 3.5 SP1

WPF 4.0 w/ Multi Touch APIs

Surface Hardware

Windows Vista

Windows 7

Multi-Touch API

related content and resources
Related Content and Resources

Workshops and sessions:

  • PDC09-WKSP08 – Windows 7 Developer Boot Camp
  • PDC09-CL27 – Multitouch on Microsoft Surface and Windows 7 for .NET Developers

Contact us:

  • Forums: http://social.msdn.microsoft.com/Forums/en-US/tabletandtouch/threads/
  • Email:

wintouch@microsoft.com

wm gesture29
WM_GESTURE
  • Provides notifications when the user performs gestures over your window
  • Contains additional information like center of gesture and gesture-specific arguments

Typical sequence:

WM_TABLET_QUERYSYSTEMGESTURESTATUS

WM_GESTURENOTIFY

WM_GESTURE (GID_BEGIN)

WM_GESTURE (GID_PAN, GF_BEGIN)

WM_GESTURE (GID_PAN, no flags)

...

WM_GESTURE (GID_PAN, GF_INERTIA)

...

WM_GESTURE (GID_PAN, GF_END)

WM_GESTURE (GID_END)

controlling gesture behavior
Controlling Gesture Behavior
  • WM_GESTURENOTIFY – “Heads up, a gesture is coming”
  • SetGestureConfig()
    • Control how gestures work in your window
    • Examples: 2D panning, explicitly setting SFP flags
    • Can set in response to WM_GESTURENOTIFY, at window creation, or other times
single finger panning sfp enabling 2d panning
Single-Finger Panning (SFP)Enabling 2D panning

GESTURECONFIG gc = {

GID_PAN,

GC_PAN & GC_PAN_WITH_SINGLE_FINGER_VERTICALLY & GC_PAN_WITH_SINGLE_FINGER_HORIZONTALLY & GC_PAN_WITH_INERTIA,

0

};

BOOL bResult = SetGestureConfig(

hWnd, // window for which configuration is specified

0, // reserved, must be 0

1, // count of GESTURECONFIG structures

&gc, // array of GESTURECONFIG structures

sizeof(GESTURECONFIG) // sizeof(GESTURECONFIG)

);

legacy gesture handling
Legacy Gesture Handling
  • If an app doesn’t handle WM_GESTURE, it is treated as a gesture-unaware legacy app
  • Panning and zooming fallback behaviors are provided
  • If you want any default handling of gestures, make sure GID_BEGIN, GID_END, and the GID_ for the gesture in question are passed to DefWindowProc()
real world questions and answers33
Real-world Questions and Answers

WM_GESTURE

Q: I registered for WM_TOUCH but I no longer get WM_GESTURE. What gives?

A: Yes, WM_GESTURE and WM_TOUCH are exclusive in Win7.

Q: I’m implementing gestures in my app, but I stopped getting panning bounce feedback. Why?

A: Two possibilities: Panning messages are being accidentally eaten, or your app is providing custom handling of them but not calling the bounce feedback API.

getsystemmetrics34
GetSystemMetrics()
  • GetSystemMetrics():
    • SM_DIGITIZER returns data about available digitizers on the system
    • SM_TABLETPC returns data about Tablet functionality
    • SM_MAXIMUMTOUCHES yields the largest number of contacts any of the available digitizers supports
mouse messages getmessageextrainfo35
Mouse Messages & GetMessageExtraInfo()
  • GetMessageExtraInfo() returns the extra info associated with a message
  • Mouse up and down messages are tagged with a special signature indicating they came from touch or pen:
    • Mask extra info against 0xFFFFFF80
    • 0xFF515780 for touch, 0xFF515700 for pen
advanced wm gesture topics
Advanced WM_GESTURE Topics
  • Compound gestures:
    • Zoom and Pan simultaneously: both messages contain information about both gestures
  • Message bubbling
    • Unhandled gestures bubble up the parent window chain
    • If they aren’t handled, OS legacy support kicks in
  • Coalescing
    • WM_GESTURE message are coalesced
  • Capture
    • Similar to mouse, implicit capture to window on down
advanced wm touch topics
Advanced WM_TOUCH Topics
  • Primary contact
    • First finger down is the “primary contact” and determines the target window for capture
  • Capture
    • Similar to mouse, contacts are implicitly captured on down and limited to the primary contact’s window
  • Coalescing
    • WM_TOUCH is coalesced. Can disable coalescing explicitly w/ a flag in RegisterTouchWindow().
touch development roadmap39
Touch Development Roadmap

Windows 7 (2009)

Native

Win32

Application

WinFormsApplication

WPF Application

Surface Application

Surface SDK

1.0

Multi-Touch Controls

Multi-Touch Controls & API

WPF 3.5 SP1

Managed Wrapper and Interop

Surface Hardware

Windows Vista

Windows 7

Multi-Touch API

touch development roadmap40
Touch Development Roadmap

.NET 4.0 & Surface Toolkit (Q1 2010)

Native

Win32

Application

WinFormsApplication

WPF Application

Surface Application

Surface SDK

1.0

Surface Multi-Touch Controls &API

Multi-Touch Controls & API

Multi-Touch Controls

Surface Toolkit for Windows 7

Managed Wrapper and Interop

WPF 3.5 SP1

WPF 4.0 w/ Multi Touch APIs

Surface Hardware

Windows Vista

Windows 7

Multi-Touch API

touch development roadmap41
Touch Development Roadmap

Surface vNext (Future)

Native

Win32

Application

WinFormsApplication

WPF Application

Surface Application

Surface SDK

vNext

Surface Multi-Touch Controls &API

Multi-Touch Controls

Surface-specific

Controls & API

Surface Toolkit

Managed Wrapper and Interop

WPF 4.0 w/ Multi Touch APIs

Surface Hardware

Windows 7

Windows 7

Multi-Touch API

slide42

YOUR FEEDBACK IS IMPORTANT TO US!

Please fill out session evaluation forms online at

MicrosoftPDC.com

learn more on channel 9
Learn More On Channel 9
  • Expand your PDC experience through Channel 9
  • Explore videos, hands-on labs, sample code and demos through the new Channel 9 training courses

channel9.msdn.com/learn

Built by Developers for Developers….