sign language recognition l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Sign Language Recognition PowerPoint Presentation
Download Presentation
Sign Language Recognition

Loading in 2 Seconds...

play fullscreen
1 / 13

Sign Language Recognition - PowerPoint PPT Presentation


  • 403 Views
  • Uploaded on

Sign Language Recognition Overview Average person typing speed Composing: 19 words per minute Transcribing: 33 words per minute Sign speaker Full sign language: 200—220 wpm Spelling out: est. 50-55 wpm Faster by 1.5x to 3x Purpose and Scope Native signers can input faster Benefits:

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Sign Language Recognition' - jaden


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
sign language recognition
Byron Hood | version 0.3

Computer Systems Lab Project 2007-2008

Sign Language Recognition
overview
Byron Hood | version 0.3

Computer Systems Lab Project 2007-2008

Overview
  • Average person typing speed
    • Composing: 19 words per minute
    • Transcribing: 33 words per minute
  • Sign speaker
    • Full sign language: 200—220 wpm
    • Spelling out: est. 50-55 wpm
    • Faster by 1.5x to 3x
purpose and scope
Byron Hood | version 0.3

Computer Systems Lab Project 2007-2008

Purpose and Scope
  • Native signers can input faster
  • Benefits:
    • Hearing & speaking disabled
    • Sign interpreters
  • Just letters & numbers for now
research
Byron Hood | version 0.3

Computer Systems Lab Project 2007-2008

Research
  • Related projects
    • Using mechanical gloves, colored gloves
    • Tracking body parts
    • Neural network-based application
      • Still images: 92% accuracy
      • Motion: less than 50% accuracy
    • Feature vector-based application
      • Also about 90% accuracy on stills
      • No motion tests
more research
Byron Hood | version 0.3

Computer Systems Lab Project 2007-2008

More Research
  • Image techniques:
    • Edge detection (Robert’s Cross)‏
    • Line detection (Hough transform)‏
    • Line interpretation methods
      • Chaining groups of lines
      • Macro-scale templates
program architecture
Byron Hood | version 0.3

Computer Systems Lab Project 2007-2008

Program Architecture
  • Webcam interface
  • Edge detection
  • Line detection
  • Line interpretation
  • Hand attribute matching
testing model
Byron Hood | version 0.3

Computer Systems Lab Project 2007-2008

Testing Model
  • Human interaction necessary
  • General testing model:

bhood@bulusan:~/syslab-tech$ \

> ./main images/hand.pgm in.pgm

[DEBUG] Edge detect time: 486 ms

[DEBUG] Wrote image file to `in.pgm’

Errors:0Warnings:0

code sample
Byron Hood | version 0.3

Computer Systems Lab Project 2007-2008

Code sample
  • Robert’s Cross edge detection:

int row,col;

// loop through the image

for(row=0; row<rows; row++) {

for(col=0; col<cols; col++) {

// avoid the problems from trying to calculate on an edge

if(row==0 || row==rows-1 || col==0 || col==cols-1) {

image2[row][col]=0; // so set value to 0

}

else {

int left = image1[row][col-1]; // some variables

int right = image1[row][col+1]; // to make the final

int top = image1[row-1][col]; // equation seem a

int bottom = image1[row+1][col]; // little bit neater

image2[row][col]=(int)(sqrt(pow(left – right , 2) +

pow(top - bottom, 2)));

}

}

}

edge detection results
Byron Hood | version 0.3

Computer Systems Lab Project 2007-2008

Edge Detection Results

Original image

(800 x 703)‏

  • Results:
    • Shows the important edges but little else
    • Robert’s Cross method the optimal balance

Final image

(800 x 703)‏

cropping results
Byron Hood | version 0.3

Computer Systems Lab Project 2007-2008

Cropping Results
  • Remove useless rows with no features
  • Very large optimization
    • Area difference
    • Input/output

Original

(800 x 703)‏

Result

(633 x 645)‏

line detection
Byron Hood | version 0.3

Computer Systems Lab Project 2007-2008

Line detection
  • Mostly complete
    • Still some bugs to iron out
    • Shows nonexistent lines occasionally (see image!)‏
line interpretation
Byron Hood | version 0.3

Computer Systems Lab Project 2007-2008

Line Interpretation
  • Chaining groups of lines
  • Templates
    • Generation
    • Template-based comparison
  • Hand matching
the mysterious future
Byron Hood | version 0.3

Computer Systems Lab Project 2007-2008

The Mysterious Future
  • Finish line interpretation
    • Building method for interpreting lines
    • Working on template-based recognition
    • Write XML template files
  • Finish camera-computer interaction
    • Device control must be precise, picky