Multi
This presentation is the property of its rightful owner.
Sponsored Links
1 / 58

David Nguyen Professor John Canny PowerPoint PPT Presentation


  • 89 Views
  • Uploaded on
  • Presentation posted in: General

Multi V iew. A Spatially Faithful Video-Conferencing System. David Nguyen Professor John Canny. Talk Outline. Motivation Prior Work Our Approach Experiment Discussion Conclusions.

Download Presentation

David Nguyen Professor John Canny

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


David nguyen professor john canny

MultiView

A Spatially Faithful

Video-Conferencing System

David Nguyen

Professor John Canny


Talk outline

Talk Outline

Motivation

Prior Work

Our Approach

Experiment

Discussion

Conclusions


David nguyen professor john canny

MultiView is the first practical video conferencing system that preserves many non-verbal cues in group-to-groupmeetings by improving spatial faithfulness


David nguyen professor john canny

Spatial Faithfulness describes how attention cues (e.g., gaze and gesture) are preserved or distorted across a virtual space.


Motivation

Motivation

You’re Fired!


Spatial distortions collapsed viewer effect

Spatial DistortionsCollapsed Viewer Effect

With only one camera, remote participants take on a shared and incorrect perspective warping nonverbal cues


Mona lisa effect

0

10

20

35

50

Mona Lisa Effect


Spatial faithfulness

Spatial Faithfulness

  • Different levels of Spatial Faithfulness

    • Mutual – Correctly perceive that attention is or is not being directed at you when it actually is or is not

    • Partial – Correctly perceive the general direction of attention when it is away from you

    • Full – Correctly perceive the specific object of attention

increasing

Dave

Donald


Talk outline1

Talk Outline

Motivation

Prior Work

Our Approach

Experiment

Discussion

Conclusions


Hydra sellen et al 1992

HYDRA(Sellen et al., 1992)

Group Sites

Mutual Sp.Fa.

Partial Sp.Fa.

Full Sp.Fa.


Gaze 2 vertegaal et al 2003

GAZE-2 (Vertegaal et al., 2003)

Group Sites

Mutual Sp.Fa.

Partial Sp.Fa.

Full Sp.Fa.


Majic okada et al 1994

MAJIC(Okada et al., 1994)

Group Sites

Mutual Sp.Fa.

Partial Sp.Fa.

Full Sp.Fa.


Talk outline2

Talk Outline

Motivation

Prior Work

Our Approach

Experiment

Discussion

Conclusions


Unique and correct views

Unique and Correct Views

“Virtually Here”

Multiple Viewpoint Display

Preserving a geometric relationships

between virtualand actualpositions

provides full spatial faithfulness


Gaze parallax

Gaze Parallax


Camera placement vs gaze parallax

Camera Placement vs. Gaze Parallax

  • Ideally, cameras should be placed at the position of the eyes

    • Occlusion (either the camera or the image)

  • Cameras can be up to 5o before gaze parallax is perceived (Chen, 2002)

[5°-7°]


Multiview directional display

MultiView Directional Display

  • Big, Bright, High Resolution Display

  • Each view is provided by a projector

  • The projected image is reflected directly back in the direction of the projector

  • The image can be seen at varying heights only behind the projector


Construction

Construction

  • Retroreflective Layer Reflects image back in direction of source

  • Vertical Diffuser Diffuses image vertically to accommodate varying viewing heights

  • Antireflective Sheet Reduces distracting glares due to glossy surface and front projection setup


Brightness vs viewing angle

Brightness vs. Viewing Angle

α

-10O

0O


Multiview

MultiView

Cameras

Projectors

MultiView

Screen


David nguyen professor john canny

Cost

  • Small fixed cost

  • Variable costs increase linearly

  • Alternatives increase quadratically


Easy to join

Easy to Join


Talk outline3

Talk Outline

Motivation

Prior Work

Our Approach

Experiment

Discussion

Conclusions


Experimental setup

Experimental Setup

3O


Experimental setup1

Experimental Setup

  • 23 participants: UC Berkeley graduate and undergraduate students

  • Each Paid $10

  • Experiments

    • Exp 1: 10 Trials (230 Total)

    • Exp 2: 10 Trials (230 Total)

    • Exp 3: 30 Trials (690 Total)


Experiment 1 2 mona lisa effect

Experiment 1 & 2:Mona Lisa Effect

  • Each researcher was asked to look or point at one of the five targets

  • Each participant was asked to circle which target each researcher appeared to be looking/pointing at

  • Repeated for 10 trials


Experiment 1 accuracy of gaze perception

Experiment 1:Accuracy of Gaze Perception

  • Confusion Matrix

    • Each column representsactual target stimulus

    • Each Row represents perceived target

    • Ideally, this would be a diagonal matrix.

  • Participants were accurate in determining target of gaze

    • 90% were at most one target off


Experiment 2 accuracy of gesture perception

Experiment 2:Accuracy of Gesture Perception

  • Confusion Matrix

    • Each column represents actual target stimulus

    • Each Row represents perceived target

    • Ideally, this would be a diagonal matrix.

  • Participants were accurate in determining target of gaze

    • 94% were at most one target off


Experiment 3 eye contact

Experiment 3: Eye Contact

  • Researchers and Participants were paired off.

  • Researchers were asked to look at the eyes, at the cameras, below the eyes, to the left of the eyes or to the right of the eyes.

  • Each participant was asked if they felt like the partner was looking at them in the eyes.


Mutual gaze

Mutual Gaze

  • No significant difference in positive responses between different gaze positions

  • “At Cam” yielded <100% positive response

  • Participants noted a strong sense of eye contact in the context of conversation

Yes/No

“Are they looking at me?”


Virtual viewing distance

Virtual Viewing Distance

Viewers sat 12’ from screen

  • projector throw distance

Image needed to be scaled down to 66%

  • Scene:Screen size

Resulting in a virtual viewing distance of 18’


Future work

Future Work

  • New MultiView design iteration

    • 8’ Screen

    • Short throw projectors for closer virtual distance (from 18’ to 8’)

    • True Life Size Images


Future work shared workspace

Future Work:Shared Workspace

VirtualObject

Object

Projector

Glass Table


Future work1

Future Work

  • Using a new digital pipeline, we can control for aspects such as video quality and latency for low level experiments

  • New series of higher level experiments using new screen design

    • Trust formation

    • Turn Taking

    • Speech Patterns


Conclusion

?

Conclusion

  • Standard video conference systems have spatial distortions which can adversely effect communication

  • Our perceptual experiments show that attention target cues can be accurately determined using our system

  • Our solution, MultiView, is the first practical video conferencing system to support spatially faithful group-to-group meetings


Additional slides

Additional Slides


David nguyen professor john canny

MultiView

David Nguyen

Professor John Canny

A Spatially Faithful Video-Conferencing System


Position dependant views

View From Left

View From Right

View From Center

Position Dependant Views

Participants are facing the right camera


Current technologies

Current Technologies


Functions of gaze kendon 1967

Functions of Gaze(Kendon, 1967)

  • Monitoring

    • “Is my partner listening to me?”

    • “Does my partner want to take over?”

    • Planning, Current Control, Checking

  • Regulating

    • Floor Control (Turn Taking)

    • Forestall/Demand a Response

  • Expressing – feelings or attitudes

    • Power, interest, point-granting, attention, agreement


First impressions 7 38 55 rule mehrabian 1971

First Impressions: 7-38-55 Rule(Mehrabian, 1971)

  • In forming first impressions…

    • 7% is based on what you actually say

    • 38% is based on the way you say it

    • 55% is based on your appearance

  • Suggests that a lot is at risk in forming first impressions if visual nonverbal cues are not appropriately preserved


Pop quiz first impressions 7 38 55 rule mehrabian 1971

Pop Quiz: First Impressions7-38-55 Rule(Mehrabian, 1971)

In forming first impressions…

  • is based on content

  • is based on voice

  • is based on appearance

Fill in the blank with [7%, 38%, or 55%]

7%

38%

55%


The video tunnel

The Video Tunnel

  • Variants Include…

    • Gaze-2 (Vertegaal et al., 1999, 2003)

    • GA Display (Monk and Gale, 2002)

    • The Teleprompter (Kahn, 1994)

    • Clearboard (Ishii and Kobayashi, 1992)

    • Reciprocal Video Tunnel (Buxton and Moran, 1990)


Computer vision graphics approach gemmel et al 2000

Computer Vision/Graphics Approach(Gemmel et al., 2000)


Human accuity chen 2002

Human Accuity (Chen, 2002)


Next design iteration

Next Design Iteration

  • 8ft screen for life size images

  • Short throw projector for closer viewing distances (8’wide @ 9’6”) essentially halving virtual distance

Ghosting

  • Multiple elements and uncontrollable diffusion causes ghosting effects


Error by viewing position

Error by Viewing Position

  • In standard video conferencing systems, Mona Lisa Effect would cause greater error as viewing position deviates from “virtual camera position”

  • Finding: Viewing position had no significant effect on error

  • MultiView does not exhibit the Mona Lisa Effect


3 site setup

3 Site Setup


Vision more motion freedom

Vision: More Motion Freedom

  • Three evolving technologies

    • Synthesize viewpoints from multiple cameras

    • Shrinking Projector Size and Power Consumption

    • Position Tracking

Virtual

Camera


Prior work

GAZE-2

(Vertigaal et al., 2003)

Hydra

(Sellen et al., 1992)

MAJIC

(Okada et al., 1994)

Prior Work


Feature comparison

Feature Comparison


Multi v iew nguyen and canny 2005

MultiView(Nguyen and Canny, 2005)

A Spatially Faithful Video Conferencing System


Trust formation bos et al 2002

Trust development was delayed in audio/video

Defections were more likely with video/audio than FTF communication.

Little difference between video and audio

Trust Formation(Bos et al., 2002)


Errors by stimulus direction

Mean Error By Gaze Direction

Mean Error By Gesture Direction

0.79

0.73

0.76

0.68

0.65

0.59

0.55

0.28

0.23

0.43

1

2

3

4

5

1

2

3

4

5

Direction

Direction

Errors By Stimulus Direction

  • Edge Effects

  • Extension of Chen’s Snap To Contact Theory

    • Includes other sources attention (not just gaze)

    • Includes third party viewers viewing two others in discussion


Acknowledgments

Acknowledgments


  • Login