natural user interfaces microsoft surface development n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Natural User Interfaces & Microsoft Surface Development PowerPoint Presentation
Download Presentation
Natural User Interfaces & Microsoft Surface Development

Loading in 2 Seconds...

play fullscreen
1 / 42

Natural User Interfaces & Microsoft Surface Development - PowerPoint PPT Presentation


  • 143 Views
  • Uploaded on

Natural User Interfaces & Microsoft Surface Development. Why are we here?. Obligatory corporate plug Obligatory personal plug The real content: What is a Natural User Interface (NUI)? What is the Microsoft Surface? MS Surface development environment Demos and code: Hello, Surface!

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Natural User Interfaces & Microsoft Surface Development' - balin


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
why are we here
Why are we here?
  • Obligatory corporate plug
  • Obligatory personal plug
  • The real content:
    • What is a Natural User Interface (NUI)?
    • What is the Microsoft Surface?
    • MS Surface development environment
    • Demos and code:
      • Hello, Surface!
      • Raw Images in WPF
      • Contact Data Visualization
      • Surface Controls
      • Tag Visualizations
    • The Future: Windows Touch and Surface 2.0
our company culture and success

Based on our philosophy of The Intelligent Application of TechnologyTM, we design, develop, and implement technologysolutions that help our customers run their businesses better.

Our Company, Culture and Success

t4g at a glance
T4G at a glance

1996

Year T4G was

established

3

  • # of consecutive years on the Great Place to Work® list

260+

80

# of employees in 7 locations across North America

% of customers in 2010 who invited us back for multiple projects

slide5

OUR VALUES

Integrity

Passion

Communication

Teamwork

Adaptive

Innovation

Entrepreneurialism

Value Conscious

Social Responsibility

delivering leadership and expertise
Delivering leadership and expertise

Our Vertical Expertise

  • Our Supporting Services
  • Business Intelligence
  • Infrastructure & Security
  • Software Development & Integration
  • Content Delivery
  • Mobile Solutions
  • Social Media Strategy & Measurement
  • Strategic Planning
  • Hosting & Application Management
  • Portals & Collaboration
  • User Experience & Design

Travel and Hospitality

Retail

Energy

Financial Services

Professional Services

Healthcare

Government

Communications

who am i
Who am I?
  • Principal Consultant at T4G
  • Focused on Portal Technologies (SharePoint)
  • Also work on NUIs and Multi-touch technologies, among other things
  • 25+ years on software and various technology things
what is a natural user interface
What is a Natural User Interface?

A natural user interface is a user interface designed to use natural human behaviours for interacting directly with content.

a new way of interacting
A New Way of Interacting

Speech

Social Interaction

Eye Tracking

Augmented Reality

NUIis much more than Touch

Gestures

Natural Learning

User eXperience

Object Recognition

a new way of interacting1
A New Way of Interacting

Social

Immersive

“The content is the application”

implications
Implications

PervasiveUIs

Don’t replace human interaction, enhance it

Extend the real world, blur the lines between UI and real life

what is the surface
What is the Surface?

“Microsoft Surface is a revolutionary multi-touch computer that responds to natural hand gestures and real-world objects, helping people interact with digital content in a simple and intuitive way.”

surface is not just multi touch
Surface is not just multi-touch
  • Windows 7 & WPF 4 are a great baseline for touch
  • Surface creates unique opportunities with specialized HW
    • Robust top
    • Horizontal form factor
    • Infrared camera array
surface beyond multi touch
Surface: Beyond Multi-Touch
  • Massive multi-touch
  • Multi-user experiences
  • Object recognition
  • Optical engineering
surface multi user
Surface: Multi-User
  • Trend: Software continues to reduce the need for face-to-face human interactions
  • Problem: Happiness and creativity depend on social interactions
  • Surface brings people together
    • Industrial design encourages users to gather
    • Input capabilities allow simultaneous use
    • Apps facilitate collaboration / competition
surface massive multi touch
Surface: Massive Multi-Touch
  • Surface can efficiently track 50+ simultaneous fingers and objects
surface objects
Surface: Objects
  • Uniquely identify & react to billions of tagged objects

Byte Tags

256 unique values

Identity Tags

>340,282,366,920,938,000,000,000,000,000,000,000,000unique values

  • Tags include orientation indictors dots
surface optics
Surface: Optics
  • Determine the size and shape of every input
  • See into and project onto physical objects
what makes a good surface application
What makes a GOOD Surface Application?
  • The app must be right for the Surface paradigm
  • It should be social
  • Should not involve large amounts of “traditional” content
  • Needs to be implemented thinking about touch
who is using it
Who is using it?
  • Retailers (Microsoft, Wind Mobile, etc.)
  • Hospitality (Hard Rock Café, Rio iBar, Sheraton, etc.)
  • Museums (Smithsonian)
  • Media (MSNBC)
  • Healthcare, Education, Others
development environment
Development Environment

Architecture of the Surface Development Platform

development environment1
Development Environment

All stuff you know…

  • Windows Vista 32-bit
  • .NET 3.5
  • WPF or XNA
development environment2
Development Environment

Get your own Surface…

On a unit, you can use the Microsoft Surface SDK to develop and test Microsoft Surface applications directly. You can run the unit in two modes.

In administrator mode, you can access all of the Windows Vista functionality.

User mode demonstrates how the unit appears in a commercial venue. That is, the unit suppresses the Windows Vista user interface, automatically starts required applications, and makes sure that those applications are restarted if they need to. Typically, you develop applications in administrator mode.

development environment3
Development Environment

If you do not have a Surface or you share one)…

On a separate workstation, you can install the Microsoft Surface SDK and use the Surface Simulator tool to simulate a Microsoft Surface test environment.

  • Runs the same Surface Shell as the Surface does
  • Use 1 or more mice to simulate multi-touch
  • Tools for simulating Tags or Blobs
  • Record and Replay interactions to support testing and debugging

Limitations:

  • No cameras (obviously) so if you use raw image data, cannot test it on simulator
  • Should have a pretty good sized screen
  • The SDK is only supported on Windows Vista Business, 32-bit! But, there are ways around that! (http://www.brianpeek.com/blog/archive/2009/03/10/install-the-surface-sdk-on-windows-7-and-or-x64.aspx)
surface sdk features for multi touch
Surface SDK Features for Multi-Touch
  • Common controls optimized for Multi-Touch
  • Controls design primarily for Multi-Touch
  • Essential Multi-Touch UX Functionality
lessons learned
Lessons Learned
  • There is a lot to learn about NUIs
  • There is just as much to unlearn
  • Forget most of what you know about what makes a “good” UI
  • Visuals/graphics/interaction design more important than code
  • Less is more
  • Development is easier on a workstation
hello world
Hello, World!

Plain Old WPF XAML

<Window x:Class="WpfApplication1.MainWindow“

xmlns=http://schemas.microsoft.com/winfx/2006/xaml/presentation

xmlns:x=http://schemas.microsoft.com/winfx/2006/xaml

Title="MainWindow" Height="350" Width="525">

<Grid>

</Grid>

</Window>

hello world1
Hello, World!

Surface XAML

<s:SurfaceWindow x:Class="HelloSurface.SurfaceWindow1“

xmlns=http://schemas.microsoft.com/winfx/2006/xaml/presentation

xmlns:x=http://schemas.microsoft.com/winfx/2006/xaml

xmlns:s=http://schemas.microsoft.com/surface/2008

Title="Hello Surface">

<Grid>

</Grid>

</s:SurfaceWindow>

code sample 1
Code Sample 1

Hello, Surface!

  • Basic Surface Application
  • Responding to Touch Events
  • Responding to multiple Touch Events
  • Handling orientation information
code sample 2
Code Sample 2

Contact Data Visualization

  • Types of Contact events Surface sees (fingers, tags, and blobs)
  • Information about contacts
code sample 3
Code Sample 3

Surface Controls

  • Common controls optimized for Multi-Touch
  • Controls design primarily for Multi-Touch
code sample 31
Code Sample 3

Surface Controls

  • Common controls optimized for Multi-Touch
  • Controls design primarily for Multi-Touch
code sample 32
Code Sample 3

Surface Controls

  • ScatterView Control
  • Adding ScatterViewItems in XAML
  • Adding ScatterViewItems in code (in response to Contact Events)
  • ScatterViewItems through Binding (with simple item template)
  • Handling contact events when ScatterViewItem content is “active”
surface objects1
Surface: Objects
  • Uniquely identify & react to billions of tagged objects

Byte Tags

256 unique values

Identity Tags

>340,282,366,920,938,000,000,000,000,000,000,000,000unique values

  • Tags include orientation indictors dots
code sample 4
Code Sample 4

Tag Visualizations

  • Adding a Tag Visualizer to recognize a tag
  • Extracting the information in the tag
  • Displaying a Tag Visualization
touch development roadmap
Touch Development Roadmap

Windows 7 (2009)

Native

Win32

Application

WinFormsApplication

WPF Application

Surface Application

Surface SDK

1.0

Multi-Touch Controls

Multi-Touch Controls & API

WPF 3.5 SP1

Managed Wrapper and Interop

Surface Hardware

Windows Vista

Windows 7

Multi-Touch API

touch development roadmap1
Touch Development Roadmap

.NET 4.0 & Surface Toolkit (Q1 2010)

Native

Win32

Application

WinFormsApplication

WPF Application

Surface Application

Surface SDK

1.0

Surface Multi-Touch Controls &API

Multi-Touch Controls & API

Multi-Touch Controls

Surface Toolkit for Windows Touch

Managed Wrapper and Interop

WPF 3.5 SP1

WPF 4.0 w/ Multi Touch APIs

Surface Hardware

Windows Vista

Windows 7

Multi-Touch API

touch development roadmap2
Touch Development Roadmap

Surface 2.0

Native

Win32

Application

WinFormsApplication

WPF Application

Surface Application

Surface SDK

vNext

Surface Multi-Touch Controls &API

Multi-Touch Controls

Surface-specific

Controls & API

Surface Toolkit for Windows Touch

Managed Wrapper and Interop

WPF 4.0 w/ Multi Touch APIs

Surface Hardware

Windows 7

Windows 7

Multi-Touch API

surface 2 0 integration extension of wpf 4
Surface 2.0 Integration & Extension of WPF 4
  • All input is routed through the WPF InputManager
  • Surface input provider can send to multiple windows
  • Surface extension methods provide access to additional input data (tag values, orientation, shape, etc)

Windows Touch input provider

Manipulation & Touch Events

WPF InputManager

Apps

& Controls

Surface input provider

Surface extension methods

surface 2 0
Surface 2.0
  • Announced at CES 2011
  • Available later this year
  • SDK Available April 2011 (?)
  • Embedded Windows 7 Professional 64-bit
  • .NET Framework 4.0
  • Windows Presentation Foundation (WPF) 4.0
  • Microsoft XNA® Framework 4.0
  • Windows PowerShell and DMTF DASH support, and enhanced administrator tools
resources
Resources

On the web:

Surface Web Site (http://www.surface.com)

Deconstructing the NUI (Joshua Blake, Surface MVP)

My Blog (http://fyeomans.com)

On Twitter:

@Surface – the official MS Surface twitter identity

#Surface

@fyeomans