ui issues neural nets rts n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
UI Issues, Neural Nets, RTS PowerPoint Presentation
Download Presentation
UI Issues, Neural Nets, RTS

Loading in 2 Seconds...

play fullscreen
1 / 26

UI Issues, Neural Nets, RTS - PowerPoint PPT Presentation


  • 75 Views
  • Uploaded on

UI Issues, Neural Nets, RTS. CSE 490RA January 27, 2005. Lecture outline. Leftovers from UI Lecture Neural Networks RTS. Uses of the stylus. Direct writing Abstract writing Pointing Selecting Gesture Direct manipulation Control. Mode issues in pen computing.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'UI Issues, Neural Nets, RTS' - feivel


Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
ui issues neural nets rts

UI Issues, Neural Nets, RTS

CSE 490RA

January 27, 2005

lecture outline
Lecture outline
  • Leftovers from UI Lecture
  • Neural Networks
  • RTS
uses of the stylus
Uses of the stylus
  • Direct writing
  • Abstract writing
  • Pointing
  • Selecting
  • Gesture
  • Direct manipulation
  • Control
mode issues in pen computing
Mode issues in pen computing
  • Adding modes to the pen
    • Barrel button
    • Secondary button with non-dominant hand
    • Eraser tip
    • Pressure
    • Explicit mode buttons
  • Area based modes (writing area, gesture area, control area, etc.)
    • Cursor feedback
    • Importance of visual cues for informing user
    • Errors in crossing mode boundaries
control
Control
  • Widgets design for stylus use
flow menu
Flow Menu
  • Use movement through octants for control information

Move

Item

100%

100%

Text

Item

Highlight

Zoom

66.6%

200%

66.6%

200%

Shape

50%

75.0

400%

50%

100.0

400%

25%

800%

25%

800%

Custom

Custom

crossy crossing based ui
CrossY: Crossing based UI
  • Specify operations by drawing through
gestures
Gestures
  • Commands issued with a single stroke
  • May be drawn or invisible
  • Support from SDK
    • Register gestures to be recognized
  • UI Issues
    • Similar to keyboard short cuts
      • Speed up for experts
      • Hard to learn / remember
gestures1
Gestures
  • Ambiguity
    • Distinction between gestures
    • Distinction between gesture and other ink
  • Robustness
    • Handling misrecognized gestures
      • False positive
      • False negative
    • Gesture initiated actions should be undoable
neural networks
Neural Networks
  • Fundamentals for Handwriting Reco Lecture (Jay Pittman)
  • Recognition algorithm
  • Learning based recognition algorithm
general considerations for learning algorithms
General considerations for learning algorithms
  • Training sets
    • Collection
    • Evaluation
  • Training cost
    • Time and space
  • Algorithm cost
    • Time and space
  • Robustness to error
neural networks perceptrons
Neural networksPerceptrons
  • Motivated by considerations of the brain
single layer neural networks
Single layer neural networks
  • Bias weights
  • Threshold activation function
    • Step function
    • Sigmoid function: 1/(1 + e-x)
what you can do with single layer networks
What you can do with single layer networks
  • Any linearly separable dataset can be recognized with a single layer neural network
gradient descent algorithm
Gradient descent algorithm
  • Choose initial weights
  • While not at optimum
    • Compute derivative
    • Move along derivative
  • It can be proved this converges
multilayer networks with hidden nodes
Multilayer networks with hidden nodes
  • Can recognized much wider range of data set
  • The gradient descent algorithm generalizes to this case
real time stylus
Real Time Stylus
  • Allow for user computation on the ink thread
architecture overview
Architecture (Overview)

RealTime Event Sink

RealTime Event Sink

RealTime Event Sink

RealTimeStylus

Ink Collecting

Object

queue

storage

Pen

Service

InkCollector

“Inking” Thread

UI Thread

substroke operations
Substroke operations
  • Examples
    • Custom Inking
    • Multiple Ink
    • Distributed Ink
  • Rendering
    • Dynamic – draw on ink packet
    • Static – draw on paint event
custom inking
Custom Inking
  • Create plugin to listen for packets
  • Registor for Packets
  • Draw triangle on each packet
slide23

public void Packets(RealTimeStylus sender, PacketsData data){

for (int i = 0; i < data.Count; i += data.PacketPropertyCount){

Point point = new Point(data[i], data[i+1]); // Packet data always has x, y

// followed by the rest

Point convertedPoint = new Point(); // We need to convert to Pixels...

convertedPoint.X = (int)Math.Round((float)point.X * (float)myGraphics.DpiX/2540.0F);

convertedPoint.Y = (int)Math.Round((float)point.Y * (float)myGraphics.DpiY/2540.0F);

if (this.pointCount == 0)

this.firstPoint = convertedPoint;

if (this.pointCount > 1){

Color color = pointCount % 2 == 0 ? Color.Red : Color.Yellow;

Point[] triangle = new Point[3];

triangle[0] = this.firstPoint;

triangle[1] = this.previousPoint;

triangle[2] = convertedPoint;

myGraphics.FillPolygon(new SolidBrush(color), triangle);

}

this.pointCount++;

this.previousPoint = convertedPoint;

}

}

distributed ink
Distributed Ink
  • Capture ink packets on machine 1
  • Send packets to machine 2
  • Reconstruct ink stroke by inserting packets
distributed ink1
Distributed Ink
  • Sender:
    • Collect packets in packet, when count is above threshold, send message
  • Receiver:
    • Insert packets as custom data
    • Render custom data as it is received
    • Assemble custom data as an ink stroke