Intelligent MultiModal Interfaces

1. Intelligent MultiModal Interfaces Manuel J. Fonseca Joaquim A. Jorge http://sketch.inesc-id.pt

2. Presentation Outline • Problem • Tools/Solution • Objectives • Project Overview • Task Analysis • 2D Sketch Editor • Sketch-Based Retrieval • Current Status

3. Problem • CAD systems allow very complicated models • “Bad” Usability at early stages of product design (pencil/paper reign) • Early design phase dissociated from product definition • Large conceptual gaps Mental model vs Geometric models • Longer product cycles • Inflexible separation - design and engineering • Raise manufacturing costs and Time-to-market

7. 9 Weeks to get the Virtual model

8. Tools/Solution • Alternative input modalities and new interface technologies • New sketching applications supporting what we call Calligraphic Interfaces. • Remove artificial dialogue constraints (WIMP Uis) • Allow designers to work as they would with more traditional media (pencil/paper)

9. Objectives • Rethinking Usability of design • Research into better input modalities • RISC Interfaces, Drawing as metaphor • User-centered approach • Computer-based tools in initial stages of product design

10. Constraint Based Personal Design Assistants Model Assembly Retrieval by Example Immersive Surface Modeling

11. Task Analysis How ? • We visited Industrial partners • CENTIMFE • ItalDesign • ELASIS • Observed/Videotaped users in action • Interviewed them and presented questionnaires • They answered to 11 questions • Reports for each site

12. What did we ask? Task Analysis • Who is going to use the system? • What tasks do they perform? • What tasks are desired? • How are the tasks learned? • Where are the tasks performed? • How often are the tasks performed? • What are the time constraints? • etc.

13. Task Analysis Why ? • Identify critical tasks • Define user requirements • Recommendations and guidelines for the project • Assure that our work is based on Real Data not on our imagination!

14. Results/Conclusions Task Analysis • User Requirements • Larger drawing area • Bigger and easier to identify icons • New tools for the left hand • No mouse and no keyboard • Access to past projects • Possibility to use the touch sense for modelling • Software integration (2D+3D+presentation+...) • ... • Final TA meeting (Sept 9-10, Lisbon) • Final report almost done

15. On Going WPs • API Specification • 2D Sketch Editor for Modelling • 3D Surface sketching in Immersive Environments • Constraint-Based Scene Modelling • Sketch-Based Retrieval

16. 2D Sketch Editor • Recognition-based Interfaces • Reduced Instruction Set Interfaces • Handle Imprecision and Ambiguity in uniform manner • Less (More expressive) Commands

17. Current Status 2D Editor • Simple prototype using IRIT (geometric kernel) • Make it more robust using better geometric kernel such as ACIS or ICEM • Add more functionality

18. Sketch-Based Retrieval • Designer re-use drawings • CAD Systems do not support retrieval • Text indexing not sufficient for CAD • Need Innovative methods for CAD databases • Simple geometric features + Spatial relations • Indexing + Matching algorithms

19. Our proposal Sketched Query Suggested Results Query by Sketch Retrieval • How do designers retrieve?(Library Component, Past project) – Videos from TA

20. Retrieval System Architecture Classification Matching Query

21. Current Status Retrieval • Description using Topology Graphs • Graph Matching using Eigenvalues • New Indexing Structure, NB-Tree • High-Dimensional Data points • Outperforms recent structures • SR-Tree [Katayama 97] • Pyramid Technique [Berchtold 98] • A-Tree [Sakurai 00]

22. Retrieval Performance Evaluation • 100,000 points • PII@233MHz • 384MB RAM • Knn = 10 • Uniform Data

23. Project Outline and Status User Requeriments & Task Analysis API Specification Almost Done Done Constraint-Based Scene Modelling 2D Sketch Editor for Modelling Field and User Tests Working On It Sketch-Based Retrieval 3D Surface Sketching Prototypes Integration and Final Prototype Final User Evaluation