developing for microsoft surface l.
Skip this Video
Loading SlideShow in 5 Seconds..
Developing For Microsoft Surface PowerPoint Presentation
Download Presentation
Developing For Microsoft Surface

Loading in 2 Seconds...

play fullscreen
1 / 47

Developing For Microsoft Surface - PowerPoint PPT Presentation

  • Uploaded on

PC17. Developing For Microsoft Surface.  Brad Carpenter General Manager Surface Platform. Agenda. Microsoft Surface and the future of computing How you can be part of this revolution How the Surface SDK enables rapid innovation Microsoft’s multi-touch roadmap Questions and Answer.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Developing For Microsoft Surface' - ull

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
developing for microsoft surface


Developing ForMicrosoft Surface

 Brad Carpenter

General Manager

Surface Platform

  • Microsoft Surface and the future of computing
  • How youcan be part of this revolution
  • How the Surface SDK enables rapid innovation
  • Microsoft’s multi-touch roadmap
  • Questions and Answer
microsoft surface and nui
Microsoft Surface And NUI

Microsoft Surface computing uses sensing and display technology to infuse everyday surfaces with digital content and is comprised of the following unique attributes:

Object recognition

Direct interaction



Object recognition

Direct interaction



how surface is being used today
How Surface Is Being Used Today
  • 5 Verticals
  • Automotive
  • Financial Services
  • Healthcare
  • Leisure and
  • Entertainment
  • Retail
developing for microsoft surface7

Developing ForMicrosoft Surface

 Robert Levy

Program Manager

Surface SDK

 Doug Kramer

Lead Developer

Surface SDK


You already know the basics of building applications for Surface

surface v1 architecture
Surface v1 Architecture

Surface Applications


Core API

Shell UI & APIs

Vision System

Windows Integration

Windows Vista SP1

Microsoft Surface Hardware & Drivers

surface and wpf
Surface And WPF
  • Expose Surface capabilities in a way consistent with WPF
    • Infrastructure integration for consistent behavior with mouse
    • Automatic hit testing to find the control being touched
    • Sets attached properties on controls (IsContactOver, etc.)
    • Raises routed events on controls (ContactDown, etc.)
  • Custom versions of most built-in WPF controls
  • Custom controls for Surface-specific UX
  • Base classes for building custom Surface controls
surface api usage with wpf
Surface API Usage With WPF



<Canvas Mouse.MouseDown=“OnMouseDown“>

<Button Click=“OnButtonClick”/>




void OnMouseDown(object sender, MouseEventArgs e)

{ // ...


void OnButtonClick(object sender,

RoutedEventArgs e)

{ // ...


  • XAML
  • <s:SurfaceWindow>
  • <Canvas s:Contacts.ContactDown=“OnContactDown“>
  • <s:SurfaceButton Click=“OnButtonClick”/>
  • </Canvas>
  • </s:SurfaceWindow>
  • Code
  • void OnContactDown(object sender, ContactEventArgs e)
  • { // ...
  • }
  • void OnButtonClick(object sender,
  • RoutedEventArgs e)
  • { // ...
  • }
surface classes for wpf input
Surface Classes For WPF Input









surface versions of common controls
Surface Versions Of Common Controls


Full screen on the Surface

Oriented towards the user




Multiple menus can be used at the same time





Only “click” when all contacts are removed

surface versions of common controls15
Surface Versions Of Common Controls
  • SurfaceListBox
  • SurfaceScrollViewer
  • Determines scrolling vs. select
  • Allows flicking
  • Provides “elastic” effect
  • SurfaceSlider
  • SurfaceScrollBar
  • Allows flicking
  • Track input results in smooth animation and capture
  • Subtle UX when not in use
  • SurfaceTextBox
  • SurfacePasswordBox
  • Invokes virtual keyboard oriented towards user when touched
neat but does it justify buying a surface
Neat, But Does It Justify Buying A Surface?
  • We’ve made it easy to port WPF and XNA apps to Surface
  • Great for developers
    • Re-use code, skills, and tools
    • Get started quickly
  • …but little value to users

Is this really just a big PC?

I don’t like having to share and wait my turn

Constantly reaching across the table is not fun

Reading upside down makes my head hurt

making surface shine
Making Surface Shine
  • Great apps leverage all of what makes Microsoft Surface unique
    • Multi-touch
    • Multi-user
    • 360° UI
    • Gestures and manipulations
    • Interaction with physical objects
  • WPF and the Surface SDK make this easy
introducing t he scatterview control
Introducing The ScatterView Control
  • Quickly enables 360°, multi-user apps
  • Gives users direct, shared control of layout
  • Standardizes manipulations for Resize, Move, and Rotate
  • Same familiar programming model as ItemsControls, ListBox, TreeView, Menu, etc.
recognizing tagged objects
Recognizing Tagged Objects

Byte Tags


256 unique values


>340,282,366,920,938,000,000,000,000,000,000,000,000unique values

Identity Tags

introducing the tagvisualizer control
Introducing The TagVisualizer Control

Robert Levy

Send Email

Simplifies common scenarios involving tagged objects

  • Displays interactive UI under, around, or next to objects
  • Keeps UI in sync with movements
  • Handles temporary lifting

TagVisualizer Control

UI for each object

Tag values to look for

Positioning of UI relative to tag

Max # of each object allowed

How to handle tag removal


Is this really just a big PC?

I don’t like having to share and wait my turn

Constantly reaching across the table is not fun

Reading upside down makes my head hurt

controls for making surface shine
Controls For Making Surface Shine

Robert Levy

Send Email


  • Multi-touch
  • Multi-user
  • 360° UI
  • Natural interactions with virtual objects


  • Object recognition
  • Natural interactions with physical objects
touch development roadmap
Touch Development Roadmap

Windows 7 Release

NET 3.5 / Surface 1.0 Release

NET 4.0 / Surface 2.0 Release

WPF Application





Surface Application

Surface SDK 1.0

Surface SDK 2.0

Multi-Touch Controls


Multi-Touch Controls & API

Multi-Touch API

Managed Wrapper and Interop

WPF 3.5 x

WPF 4.0

WPF 3.5

Multi-Touch API

Multi-Touch API and Controls

Surface Hardware

Windows Vista

Surface Hardware

Windows 7

Windows 7

Multi-Touch API

  • Microsoft Surface is shaping the future
    • Multi-touch, multi-user, 360° UI, object recognition, …
  • UI design requires creativity and innovation
  • Coding builds upon what you already know
  • Microsoft is creating a unified API for touch
call to action
Call To Action
  • Help us change the world
    • Create apps using Surface’s unique capabilities
    • Get Microsoft Surface developer hardware
      • Order forms on
      • $1,500 discount until November 15th
    • Swipe your badge to receive an e-mail invitation to join
      • Download the Microsoft Surface SDK & Simulator
      • Participate in online forums and training
related content
Related Content
  • Hands on Labs
    • PCHOL19: Using the Microsoft Surface Simulator
    • PCHOL20: A Lap Around the Microsoft Surface SDK
    • Surface HOL: Exploring Vision Based Object Recognition with Microsoft Surface
  • Demo applications from partners
  • Scavenger Hunt game
  • PC03: Developing multi-touch applications with Windows 7
    • Wednesday, 1:15
evals recordings
Evals & Recordings

Please fill out your evaluation for this session at:

This session will be available as a recording at:


© 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.

The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

simulator for off table dev test
Simulator for Off-Table Dev & Test
  • Replicates input
    • With multiple mice
    • With one mouse
      • Click to place multiple contacts
      • Click and drag to group them together
      • Move, rotate, and stretch the group
  • Supports all types of contacts
    • Finger, resizable blobs, and tags
  • Supports input automation for testing
    • Recording and playback UI
    • Simulator APIs


manipulation inertia processor apis
Manipulation & Inertia Processor APIs
  • Low level APIs for building controls with consistent behavior
    • Reduces the cost of building new controls
    • Increases consistency for end users
  • Interpreting Surface input as 2D transformations
    • Input: list of contacts to track
    • Output: scale, rotation, movement, velocity data (~60 FPS)
    • Used by: ScatterView, ScrollViewer, Slider, Concierge map
  • Two modes of inertia
    • Initial velocity + deceleration rate
      • Realistic
      • Used by ScatterView and Concierge map
    • Initial velocity + final position/orientation
      • Goal-based
      • Used by ScrollViewer and Slider
ups downs
Ups & Downs


  • Contacts.ContactDownContacts.ContactUp
up down event comparison
Up & Down Event Comparison
  • Similarities:
    • Naming convention & meaning
    • Routing strategy
      • Tunneling (“preview”) events & bubbling events
      • Events stop once marked as handled
      • Hit testing & capture logic
  • Differences:
    • Need to find out which contact the event is for
    • New Contact object is created with each Down event

Mouse.MouseMove … Mouse.MouseMove …

Contacts.ContactChanged … Contacts.ContactChanged …

mousemoved contactchanged
MouseMoved == ContactChanged?

ContactChanged includes:

  • Position
  • Orientation
  • Recognized type
  • Size
  • Shape
  • Area

Wait a sec… he said this was supposed to be the same as mouse.

Why are these events named


movement event comparison
Movement Event Comparison
  • Similarities:
    • Routing strategy
      • Tunneling (“preview”) events & bubbling events
      • Events stop once marked as handled
      • Hit testing & capture logic
    • Event for changes in position
  • Differences:
    • ContactChanged is not just for movements
    • Change events only happen between Down & Up
    • Need to find out which contact the event is for


  • Relative to specified UI element
    • GetPosition(window) = {50, 100}
    • GetPosition(button) = {10, 10}
  • To track position changes across multiplecontrols, get position relative to a common parent



  • Degrees clockwise from relative east
  • Range and usefulness varies
    • Tagged objects: 360°
    • Generic blobs: 180°
    • Fingers: 360° but…unreliable when straight down



ellipse bounds
Ellipse & Bounds


Ellipse Contact.GetEllipse(UIElement)

void Contact.UpdateEllipse(Ellipse, UIElement)

  • Ellipse is returned with transforms that counteract transforms on the specified element
    • Adding the shape as a child element results in UI matching the physical object
  • Bounding rectangle is always screen-aligned
  • UpdateEllipse method can improve performance