1 / 26

UI Issues, Neural Nets, RTS

UI Issues, Neural Nets, RTS. CSE 490RA January 27, 2005. Lecture outline. Leftovers from UI Lecture Neural Networks RTS. Uses of the stylus. Direct writing Abstract writing Pointing Selecting Gesture Direct manipulation Control. Mode issues in pen computing.

feivel
Download Presentation

UI Issues, Neural Nets, RTS

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. UI Issues, Neural Nets, RTS CSE 490RA January 27, 2005

  2. Lecture outline • Leftovers from UI Lecture • Neural Networks • RTS

  3. Uses of the stylus • Direct writing • Abstract writing • Pointing • Selecting • Gesture • Direct manipulation • Control

  4. Mode issues in pen computing • Adding modes to the pen • Barrel button • Secondary button with non-dominant hand • Eraser tip • Pressure • Explicit mode buttons • Area based modes (writing area, gesture area, control area, etc.) • Cursor feedback • Importance of visual cues for informing user • Errors in crossing mode boundaries

  5. Control • Widgets design for stylus use

  6. Flow Menu • Use movement through octants for control information Move Item 100% 100% Text Item Highlight Zoom 66.6% 200% 66.6% 200% Shape 50% 75.0 400% 50% 100.0 400% 25% 800% 25% 800% Custom Custom

  7. Interaction with direct manipulation Move Item Highlight Zoom

  8. CrossY: Crossing based UI • Specify operations by drawing through

  9. Gestures • Commands issued with a single stroke • May be drawn or invisible • Support from SDK • Register gestures to be recognized • UI Issues • Similar to keyboard short cuts • Speed up for experts • Hard to learn / remember

  10. Gestures • Ambiguity • Distinction between gestures • Distinction between gesture and other ink • Robustness • Handling misrecognized gestures • False positive • False negative • Gesture initiated actions should be undoable

  11. Neural Networks • Fundamentals for Handwriting Reco Lecture (Jay Pittman) • Recognition algorithm • Learning based recognition algorithm

  12. General considerations for learning algorithms • Training sets • Collection • Evaluation • Training cost • Time and space • Algorithm cost • Time and space • Robustness to error

  13. Neural networksPerceptrons • Motivated by considerations of the brain

  14. Single layer neural networks • Bias weights • Threshold activation function • Step function • Sigmoid function: 1/(1 + e-x)

  15. What you can do with single layer networks • Any linearly separable dataset can be recognized with a single layer neural network

  16. Gradient descent algorithm • Choose initial weights • While not at optimum • Compute derivative • Move along derivative • It can be proved this converges

  17. However, single layer networks are very limited

  18. Multilayer networks with hidden nodes • Can recognized much wider range of data set • The gradient descent algorithm generalizes to this case

  19. Real Time Stylus • Allow for user computation on the ink thread

  20. Architecture (Overview) RealTime Event Sink RealTime Event Sink RealTime Event Sink RealTimeStylus Ink Collecting Object queue storage Pen Service InkCollector “Inking” Thread UI Thread

  21. Substroke operations • Examples • Custom Inking • Multiple Ink • Distributed Ink • Rendering • Dynamic – draw on ink packet • Static – draw on paint event

  22. Custom Inking • Create plugin to listen for packets • Registor for Packets • Draw triangle on each packet

  23. public void Packets(RealTimeStylus sender, PacketsData data){ for (int i = 0; i < data.Count; i += data.PacketPropertyCount){ Point point = new Point(data[i], data[i+1]); // Packet data always has x, y // followed by the rest Point convertedPoint = new Point(); // We need to convert to Pixels... convertedPoint.X = (int)Math.Round((float)point.X * (float)myGraphics.DpiX/2540.0F); convertedPoint.Y = (int)Math.Round((float)point.Y * (float)myGraphics.DpiY/2540.0F); if (this.pointCount == 0) this.firstPoint = convertedPoint; if (this.pointCount > 1){ Color color = pointCount % 2 == 0 ? Color.Red : Color.Yellow; Point[] triangle = new Point[3]; triangle[0] = this.firstPoint; triangle[1] = this.previousPoint; triangle[2] = convertedPoint; myGraphics.FillPolygon(new SolidBrush(color), triangle); } this.pointCount++; this.previousPoint = convertedPoint; } }

  24. Distributed Ink • Capture ink packets on machine 1 • Send packets to machine 2 • Reconstruct ink stroke by inserting packets

  25. Sample application

  26. Distributed Ink • Sender: • Collect packets in packet, when count is above threshold, send message • Receiver: • Insert packets as custom data • Render custom data as it is received • Assemble custom data as an ink stroke

More Related