1 / 44

Tools for Programming in the Real World

Tools for Programming in the Real World. Self Introduction for MSRA Fellowship 2012 Jun Kato , The University of Tokyo. Jun Kato. 2 nd - year PhD candidate Igarashi Laboratory, The University of Tokyo Concentration: Human-Computer Interaction Especially,

ismail
Download Presentation

Tools for Programming in the Real World

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Tools forProgramming in the Real World Self Introduction for MSRA Fellowship 2012 Jun Kato, The University of Tokyo

  2. Jun Kato • 2nd- year PhD candidate • Igarashi Laboratory, The University of Tokyo • Concentration: Human-Computer Interaction • Especially, Tools for Programming in the Real World

  3. Agenda • My motivation for the research • Introduction of the projects • Ideas for new projects

  4. Tools for Programming in the Real World My motivationfor the research

  5. Tools for Programmingin the Real World Pressing [UIST’09 SIC] IntelliViewer [UIST’10 SIC] Multi-touch [CHI’09 SRC] Surfboard [UIST’10 Demo] CRISTAL [SIGGRAPH’09 E-Tech] DejaVu [UIST’12 (to appear)] Phybots [DIS’12] Sharedo [Ro-MAN’12 (submitted)] Roboko [ CHI’13 (to be submitted)]

  6. In the Real WorldFrom Robot to Human Pressing [UIST’09 SIC] IntelliViewer [UIST’10 SIC] Multi-touch [CHI’09 SRC] Multi-touch [CHI’09 SRC] Surfboard [UIST’10 Demo] CRISTAL [SIGGRAPH’09 E-Tech] CRISTAL [SIGGRAPH’09 E-Tech] DejaVu [UIST’12 (to appear)] Phybots [DIS’12] Phybots [DIS’12] Sharedo [Ro-MAN’12 (submitted)] Roboko [ CHI’13 (to be submitted)]

  7. In the Real WorldFrom Robot to Human Pressing [UIST’09 SIC] IntelliViewer [UIST’10 SIC] Multi-touch [CHI’09 SRC] Surfboard [UIST’10 Demo] CRISTAL [SIGGRAPH’09 E-Tech] DejaVu [UIST’12 (to appear)] DejaVu [UIST’12 (to appear)] Phybots [DIS’12] Sharedo [Ro-MAN’12 (submitted)] Roboko [ CHI’13 (to be submitted)] Roboko [ CHI’13 (to be submitted)]

  8. Tools for ProgrammingFrom Keyboard to IDE Pressing [UIST’09 SIC] IntelliViewer [UIST’10 SIC] Multi-touch [CHI’09 SRC] Multi-touch [CHI’09 SRC] Surfboard [UIST’10 Demo] CRISTAL [SIGGRAPH’09 E-Tech] DejaVu [UIST’12 (to appear)] DejaVu [UIST’12 (to appear)] Phybots [DIS’12] Phybots [DIS’12] Sharedo [Ro-MAN’12 (submitted)] Roboko [ CHI’13 (to be submitted)] Roboko [ CHI’13 (to be submitted)]

  9. Tools for ProgrammingFrom Keyboard to IDE Pressing [UIST’09 SIC] Pressing [UIST’09 SIC] IntelliViewer [UIST’10 SIC] IntelliViewer [UIST’10 SIC] Multi-touch [CHI’09 SRC] Surfboard [UIST’10 Demo] Surfboard [UIST’10 Demo] CRISTAL [SIGGRAPH’09 E-Tech] DejaVu [UIST’12 (to appear)] Phybots [DIS’12] Sharedo [Ro-MAN’12 (submitted)] Roboko [ CHI’13 (to be submitted)]

  10. Tools for Programming in the Real World Introduction ofthe (Selected) Projects

  11. Projects • Phybots • DejaVu • Roboko

  12. Motivation • Add mobility to our daily objects

  13. Gulf of HCI and Robotics Robots Physical UIs Toolkit for: HCI researchers & Interaction designers Robotics people Target users: Prototyping Reliability Focus: Low-level&Static High-level&Extensible Software API: Small&Cheap Medium-Large&Expensive Hardware: K-Junior, $938 Phidget Kit, $200

  14. Goal of Phybots • Add mobility to physical objects • In a cheap and easy way • Through high-level and extendible API • With support for the whole prototyping process

  15. Prototyping with Phybots Test with runtime debug tool Hardware construction Programming with built-in API (Extend APIwhen needed)

  16. Localization and Locomotion API • Hardware setup: easy and cheap • Camera • PC or Mac • Robotic things From our user study: Miniature drive recorder

  17. Localization and Locomotion API • Navigation by global coordinates • Move • Push • TracePath X-axis Y-axis Taskmove = newMove( mouseX, mouseY); move.assign(robot); move.start(); From our user study: Beach flags with obstacles

  18. Localization and Locomotion API • Vector field navigation • Easy design of new behavior ex) Follow another robot • Combination of existing fields ex) Move + collision avoidance

  19. Runtime Debug Tool Workflow Monitor Service Monitor Entity Monitor

  20. User Studies • Alpha version deployment • To graduate students • Current version deployment • To HCI students • To robotics students

  21. Alpha version deployment • Lessons learned: • Mere mobility was not enough. • Most apps only used a single API call. • Parameter configuration was painful. • Lessons learned: • Extendibility is important. • Higher-level task management is desired. • Support for testing phase is needed.

  22. Current version deployment

  23. Phybots • A toolkit for prototyping “robotic things” • Localization and Locomotion API • Extensible Architecture • Runtime Debug Tool • Open-source software available at http://phybots.com

  24. Projects • Phybots • DejaVu • Roboko

  25. Motivation • Input by the user’s hands, body, or physical objects • Captured by cameras • Processed in real-time • Support for computer-vision-based programming is needed Wilson, 2005 Wang, 2006 Cao, 2003

  26. Existing approaches • Hardware, Software API • Becomes more and more accessible • Webcam, Kinect, OpenCV, etc. • Development Environments • Do not provide explicit support… Ex) Breakpoint-based variable monitoring OpenCV Visual Studio Eclipse …

  27. DejaVu

  28. DejaVu Canvas Frame data visualization Freehand sketch

  29. DejaVu Canvas • Capable of showing variable values at the different positions in the code

  30. DejaVu Timeline

  31. Conclusion • DejaVu provides integrated support for the development of interactive camera-based programs that tightly matches • their nature: Visual/Continuous/Non-reproducible • the workflow:Record input/Change code/Replay Canvas Workflow

  32. Projects • Phybots • DejaVu • Roboko

  33. Motivation: Programming robot actionsis difficult. Dance Clearing dishes Cooking

  34. Typical approach: Text-based Programming arm.setPosition(4.3, 2.4); robot1.setPose(pose32); joint[0].setAngle(0.6); joint[1].setAngle(0.7); joint[2].setAngle(-0.24); … Problem:We cannot imagine the actual pose of the robot…

  35. Typical approach: Visual Editing Problem:We cannot design flexible robot behavior such as “repeat this for 3 times when …”

  36. Overview of the IDE

  37. Workflow of the Programmer 2. Drag and Drop 1. Capture PC Robot Camera

  38. Supported APIs: Getting and setting pose

  39. Supported APIs: Execute a sequential action

  40. Project outcome: new application Not only a robot but a human (Kinect) This work is an outcome of collaboration with a MSR researcher, Xiang Cao

  41. Integrating visual representations into text-based programming environment

  42. Tools for Programming in the Real World Ideas for new projects

  43. Ideas for New Projects • (CENSORED) • (CENSORED) • (CENSORED)

  44. Thank you for listening… • I’ve designed various tools for programming in the real world. • I am looking forward to further collaboration with MSRA researchers.

More Related