SKETCH-BASED USER INTERFACE STUDY
Download
1 / 28

SKETCH-BASED USER INTERFACE STUDY - PowerPoint PPT Presentation


  • 112 Views
  • Uploaded on

SKETCH-BASED USER INTERFACE STUDY. Presented By Jin Xiangyu Department of Computer Science and Technology Nanjing University  June 2002. PART I: INTRODUCTION. The rise of the research issue of Human-Computer Interaction (HCI). Computer-Oriented. Human-Oriented.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' SKETCH-BASED USER INTERFACE STUDY' - amy-oliver


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

SKETCH-BASED USER INTERFACE STUDY

Presented ByJin Xiangyu

Department of Computer Science and TechnologyNanjing University June 2002



The rise of the research issue of Human-Computer Interaction (HCI)

Computer-Oriented

Human-Oriented

1.1. A Revolution is Undertaking

Computers are becoming more and more powerful and easily available today

Human and computer, which one should be the center of computer assisted tasks?

The idea of this revolution is to bend computers to people’s way of interacting, not the other way around (Landay 2001)


1.2. Why Sketch-based User Interface? (HCI)

To write down designer’s improvisatory ideas by diagrams is very important for creative tasks.

Traditional menu/toolbar button-based user interface Demo

(1) Inefficient

A three-step process.

(2) Unnatural

Leaving a sketch uninterrupted, or at least in its rough state, is key to preserving this fluidity (Hearst 1998).

(3) Unsuitable for handled devices

No area to accommodate so many stencils and buttons.

The Solution is “By Sketch”.

Sketching with a pen is a mode of informal, perceptual interaction that has been shown to be especially valuable for creative design tasks (Gross 1996) .


Three application level (HCI)

Level

Symbol Set

Drawing Approach

Example

1

Strictly defined

Generally agreed among users

Handwriting recognition

2

Not very strictly defined

Stroke-number and stroke order free

Sketchy symbol recognition

3

Undefined

Totally free

Sketch-based image retrieval

1.3. Research focus: What Kind of Sketch-based User Interface We are Interested in?

On-line VS Off-line

Our research focus


1.4. Three Designing Principles: (HCI) Humanistic, Intelligent, Individualized

How to make the UI to be humanistic?

An graphics inputting user scenario is proposed, which employs an interactive sketching-recognition-rectification process in one-fluent-step.

How to make the UI to be intelligent?

On-line graphics recognition is employed to predict user’s original intention.

How to make the UI to be individualized?

SVM-based incremental learning is employed to adapt different user in shape classification.

These three characteristics are harmoniously combined in our prototype system

—Smart Sketchpad.



Step 1: Sketching (HCI)

Recognize and Regularize

The suggested candidate objects

Step 2: Sketching

Recognize and Regularize

Step 3: Clicking on the intended object and replace the strokes with the very object with proper parameters

2.1. Critical Technique 1: User Scenario for Graphics Inputting

Employing one interactive, fluent sketching-recognition-rectification process instead of three split ones.


Strokes (HCI)

Primitive Shape

Primitive Shape

Composite Graphic Object

Primitive Shape Classification and Regularization

Composite Shape Recognition

2.2. Critical Technique 2: On-line Graphics Recognition

Sketching (user) : Decompose

Composite Graphic Object

Recognition (computer) : Assemble

Strokes


The User Sketchy Shape (HCI)

The User Intended Shape

Preprocessing

Shape Classification

Shape Fitting

Shape Regularization

The input stroke

Quadrangle

The fitted shape

The regularized shape

2.2.1. Primitive Shape Classification and Regularization

By Vertex Combination

Primitive Shape Classification & Regularization

Strokes

Primitive Shapes


User2 (HCI)

0.35

User1

0.75

The optimal thresholds are different

Experimental Results

Shape Classification Precision for 1367 samples


Inner-Shape Regularization (HCI)

The results of primitive shape classification and regularization

Experimental Results

Shape Regularization Results


In order to suggest the user in an early stage, the system should recognize graphic object in an incomplete form.

2.2.2. Composite Graphic Object Recognition

A “Partial”“Structural” Similarity Assessment Strategy is Proposed


The similarity assessment strategy should not only invariant to shifting, rotation, mirroring, but also should invariant to inner distortion.

2.2.2. Composite Graphic Object Recognition

A “Partial”“Structural” Similarity Assessment Strategy is Proposed


The Source Object to shifting, rotation, mirroring, but also should invariant to inner distortion.

The Candidate Object

L1

P1

Graphic Primitive Extraction

L2

L3

P2

P5

P3

Line-segments, arc-segments,

and ellipses/circles

L4

P4

Spatial Relation Graph (SRGs)

The Proposed Approach

The computational complicity is Pnm.

Conditioned Partial Permutation Algorithm


The Original Object to shifting, rotation, mirroring, but also should invariant to inner distortion.

Adding Noises

Eliminating some parts

The Generated Query

Performance Evaluation

Query Generating by adding noises and eliminating some parts.

When the user draws 80% of his/her intended object (for users may miss some parts of the object inadvertently) with 10% distortion (this is similar to noises in real user drawing situation), R6 is nearly 90%(averagely of 304 graphic objects).

Experimental results show that our approach can achieve good performance with noises for incomplete objects, and our approach is also invariant to shifting, rotation, mirroring, and inner distortions.


An ambiguous case to shifting, rotation, mirroring, but also should invariant to inner distortion.

Question: A triangle or a quadrangle?

2.3. Critical Technique 3:

User Adaptation

User adaptation is a classical problem in user interface study. Many pattern recognition problems are user specific, for users’ handwritings, drawing styles, and accents are different.

Rule-based feedback may yield “conflict” results due to its intrinsic deficiency, which may lose its general performance when it adapts to a specific user further.

SVM-incremental learning are introduced into the user adaptation problem of shape classification.


2.3.1. Questions to shifting, rotation, mirroring, but also should invariant to inner distortion.

  • Four questions need to be solved:

  • Whether SVM-based Incremental learning can overcome “conflicts”?

  • What is the advantage of Incremental leaning compared with repetitive learning?

  • Which one is better, Syed’s or Xiao’s?

  • Which structure is better, one-against-one and one-against-all?

2.3.2. Experiments

Experimental Environments:

Feature Extraction (20-dimensional vector) by turning function

Virtual Sample Generation (with 40 samples each)

40 incremental training sets and two test sets are created

Training time, open-test precision, closed-test precision are tested for different algorithms and structures.


2.3.3. Answers to shifting, rotation, mirroring, but also should invariant to inner distortion.

Theoretical analysis and experimental results both show

(1) SVM-based incremental leaning can overcome “conflict”

(2) Incremental learning is much faster than repetitive learning without loss of precision

(3) Syed’s algorithm is better than Xiao’s

(4) One-against-one structure is much faster than one-against-all in our environments


PART III: THE SMART SKETCHPAD SYSTEM to shifting, rotation, mirroring, but also should invariant to inner distortion.


3.1. System architecture of Smart Sketchpad to shifting, rotation, mirroring, but also should invariant to inner distortion.


The Sketching Area to shifting, rotation, mirroring, but also should invariant to inner distortion.

The just inputted shape are recognized and regularized

Part of the intended object are sketched

The candidate shape are shown

Candidate object name and their similarities are shown

Candidate object list

3.2. The Sketch-based User Interface of Smart Sketchpad

Inputting two graphic objects and then delete them

Demo


3.3. UI Evaluation to shifting, rotation, mirroring, but also should invariant to inner distortion.

10 different subjects are required to draw the following two diagrams with traditional UI and the sketch-based UI. There are 304 objects listed in 26 stencils (with 12 each) for traditional UI. There are 6 objects can be shown in the Smart Toolbox for sketch-based UI.

Diagram 1

Diagram 2

A demo of inputting Diagram 2 by sketch


Drawing Time for Different Sketches Under Different UIs to shifting, rotation, mirroring, but also should invariant to inner distortion.

Averagely, the sketch-based UI is 22.4% and 42.9% more efficient than the traditional toolbar button-based UI for sketch1 and sketch2, respectively.

The ultimate comments of all users unanimous agree that they’d like to choose the sketch-based UI instead of the traditional one.


PART IV: SUMMAY to shifting, rotation, mirroring, but also should invariant to inner distortion.


Algorithm Level to shifting, rotation, mirroring, but also should invariant to inner distortion.

(1) Agglomerate Point Filtering, Vertex Combination, Shape Fitting, Shape Regularization

(2) Conditioned Partial Permutation

(3) Comparison of SVM-based Incremental Learning Algorithms and Structures

Solution Level

(1) Interactive graphics inputting user scenario

(2) On-line graphics recognition: primitive shape classification and regularization; composite shape recognition.

(3) SVM-incremental learning for user adaptation in shape classification problem.

System Level

(1) A prototype system for conceptual/schematic designing tasks is implemented.

(2) User evaluation between the traditional and sketch-based UIs is performed.


Future Works to shifting, rotation, mirroring, but also should invariant to inner distortion.

(1) How to perform stroke segmentation?

(2) How to cut down the computational cost and improve the recognition precision?

(3) How to make the system learn aggressively?


THANKS to shifting, rotation, mirroring, but also should invariant to inner distortion.


ad