multimedia and human computer interfaces l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Multimedia and Human Computer Interfaces PowerPoint Presentation
Download Presentation
Multimedia and Human Computer Interfaces

Loading in 2 Seconds...

play fullscreen
1 / 21

Multimedia and Human Computer Interfaces - PowerPoint PPT Presentation


  • 162 Views
  • Uploaded on

Multimedia and Human Computer Interfaces. Hao Jiang Computer Science Department Boston College Nov. 20, 2007. Outline. Human Computer Interface. Multimedia interfaces and applications. Vision based interfaces Systems using vision based interfaces. Some basic methods in vision interfaces

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Multimedia and Human Computer Interfaces' - braima


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
multimedia and human computer interfaces

Multimedia and Human Computer Interfaces

Hao Jiang

Computer Science Department

Boston College

Nov. 20, 2007

CS335 Principles of Multimedia Systems

outline
Outline
  • Human Computer Interface.
  • Multimedia interfaces and applications.
  • Vision based interfaces
    • Systems using vision based interfaces.
    • Some basic methods in vision interfaces
      • Background subtraction.
      • Color blob detection and tracking.
      • Face detection (Viola and Jones face detector).

CS335 Principles of Multimedia Systems

human computer interface
Human Computer Interface
  • Different meanings of HCI
    • The study of people, computer technology and the ways they interactive with each other.
    • The design, evaluation and implementation of computer systems for human use.
    • The research to study how to make computer systems more useable.

CS335 Principles of Multimedia Systems

aspects of hci
Aspects of HCI
  • Design
    • Intuition, experiences and design principles.
  • Human factors
    • Experiments testing the perception, memory and cognition models of human being.
  • Devices
    • Physical devices such as mouse, keyboards and other fancy HCI hardware.
  • Software and systems
    • Such as windows GUI, shell commands in Unix etc.

CS335 Principles of Multimedia Systems

hci studies
HCI Studies
  • Computers
    • What are they good at and bad at?
    • Input, output, processing, networking, etc.
  • People
    • What are people good at?
    • What is the limitation of human abilities?
    • Input, output, memory, perception and cognition model.
    • No average answer. Must accommodate diversities.
  • Usability
    • More than the size of widgets, where are the buttons and syntax of commands.
  • Context
    • Environment, physical constraints, cognition constraints, task, etc.

CS335 Principles of Multimedia Systems

the quality of hci
The Quality of HCI
  • There are fives factors that affect the design and implementation of HCI
    • Time to learn.
    • Speed of performance.
    • Rate of errors.
    • Retention over time.
    • Subjective satisfaction.

CS335 Principles of Multimedia Systems

multimedia interfaces
Multimedia Interfaces
  • Multimedia interface combines different sorts of media modalities to achieve more natural and more efficient human computer communication.
    • Speech, audio, touch, video input (patterns, posture, gesture) etc.
  • “Put that there” is an example multimedia HCI that combines speech, gestures and graphics.

Put that there, MIT 1981

http://www.media.mit.edu/speech/sig_videos.html

CS335 Principles of Multimedia Systems

early and late integration
Early and Late Integration

Mode 1

Classification

Output

Features

Mode 2

Early integration

Mode 1

Classification

Features

Output

Mode 2

Classification

Late integration

CS335 Principles of Multimedia Systems

advantages of multimedia interface
Advantages of Multimedia Interface
  • More alternative methods.
  • The flexibility of combining different schemes.
  • More efficient for specific tasks.
  • Greater precision of information.
  • Help handicapped people to use computers.

Virtual Reality

Pen gesture interface

http://www.youtube.com/watch?v=d7eGypGOlOc&feature=related

http://www.youtube.com/watch?v=Gnq-8iaOcXk

CS335 Principles of Multimedia Systems

vision based interfaces
Vision Based Interfaces
  • Vision based interfaces uses computer vision techniques to achieve more effective HCI.
  • Vision based interfaces are based on techniques such as:

Hand tracking,

Hand gesture,

Arm gesture

Hand Pointing

Face detection

Head tracking

Gaze

Expression

Lip reading

Posture, body gesture and

action recognition

CS335 Principles of Multimedia Systems

example systems
Example Systems

Motion

Capturing

CS335 Principles of Multimedia Systems

example systems12
Example Systems
  • The MIT Kidsroom

Posture recognition

Action

detection

CS335 Principles of Multimedia Systems

methods in vision interfaces
Methods in Vision Interfaces
  • Background subtraction.
  • Estimating the background

B(n) = B(n)*alpha + I(n)*(1-alpha)

Foreground Map = |Image – Background|>threshold

alpha is a positive floating point number close to 1.

CS335 Principles of Multimedia Systems

color blob detection
Color Blob Detection

We would like to compare the template color histogram with

each location in the image.

h(x,y) is the color histogram

a

b

(x,y)

c

d

We first convert each image pixel by its bin number.

s(x,y) = s(x,y-1) + i(x,y)

h(x,y) = h(x-1,y)+ s(x,y)

CS335 Principles of Multimedia Systems

blob tracking filtering scheme
Blob Tracking (Filtering Scheme)

t3

t2

t1

t4

Search window

: Predicted location.

: detected object.

p

y

: Final estimation.

x

x = p * alpha + y * (1-alpha)

CS335 Principles of Multimedia Systems

blob tracking dp approach
Blob Tracking (DP Approach)

C

F

B

A

E

D

t3

t2

t4

t1

cost(F) + alpha ||2C-B-F||

CF

const

BC

CE

AB

t

DF

BD

DE

CS335 Principles of Multimedia Systems

mean shift tracking
Mean Shift Tracking
  • Instead of exhaustive search in a window, if object moves slowly, we can use a more efficient location updating scheme – Mean shift.
  • In the following equation, x(i) are points in the search window and y is the location of object, w(i) are weights.

CS335 Principles of Multimedia Systems

mean shift tracking result
Mean Shift Tracking Result

http://www.caip.rutgers.edu/~comanici/Papers/KernelTracking.pdf

CS335 Principles of Multimedia Systems

face detection
Face Detection
  • Viola and Jones face detector

These features can be computed using integral images similarly

to the color histograms discussed before.

CS335 Principles of Multimedia Systems

face detection20
Face Detection
  • It uses boosting scheme to automatically choose the optimal weight for the classifier based on each kind of feature.
  • To further improve the efficiency, a cascaded classifier is used.

CS335 Principles of Multimedia Systems

summary
Summary
  • We only touch the very surface of multimedia interfaces.
  • Currently, the dominant HCI is still GUI with keyboard and mouse.
  • Multimedia interfaces are beginning to attract more and more interests.
  • We need more robust and efficient media recognition schemes to support effective multimedia interfaces.

CS335 Principles of Multimedia Systems