slide1
Download
Skip this Video
Download Presentation
Depth Matters: Influence of Depth Cues on Visual Saliency

Loading in 2 Seconds...

play fullscreen
1 / 42

Depth Matters: Influence of Depth Cues on Visual Saliency - PowerPoint PPT Presentation


  • 169 Views
  • Uploaded on

Depth Matters: Influence of Depth Cues on Visual Saliency. Congyan Lang , Tam V. Nguyen, Harish Katti , Karthik Yadati , Mohan Kankanhalli , and Shuicheng Yan Todays ’ Presenter : Daniel Segal Computer Vision – ECCV 2012

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Depth Matters: Influence of Depth Cues on Visual Saliency' - oki


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

Depth Matters: Influence of Depth Cues on

Visual Saliency

Congyan Lang , Tam V. Nguyen, Harish Katti , KarthikYadati ,

Mohan Kankanhalli , and Shuicheng Yan

Todays’ Presenter :

Daniel Segal

Computer Vision – ECCV 2012

12th European Conference on Computer Vision, Florence, Italy, October 7-13, 2012, Proceedings, Part II

slide2

Authors

Congyan Lang Tam V. Nguye Harish Katti

slide3

Seminar outline

  • Problem motivation
  • Challenges
  • Related work
  • Proposed solution
  • Pros & Cons
  • Limitations
  • Future ideas
slide4

Problem motivation

  • Problem presentation
  • Does depth affects saliency?
  • If so How to incorporate depth data?
slide5

Problem motivation

  • Why saliency?
  • Surveillance
  • Search and Rescue
  • Medical application
  • Image automated cropping
  • Advertising
  • Target recognition
  • Video summarizations
  • Preprocess for other algorithms
slide6

Problem motivation

  • Why incorporate depth?
  • Are observers fixation different when viewing 3D images?
  • Human natural visual attention evolved in 3D environment
  • Absent of 3D fixation data
slide7

Challenges

  • Finding efficient saliency models
  • Difficult to model top-down processes effect
  • Integrating depth additional information
  • Absent datasets for 3D stimulus
  • Absent of images dataset with corresponding depth maps
  • Subjective

Which one is more

conspicuous?

slide8

Related work

Computing Visual Attention from Scene Depth (2000) in ICPR

Authors :NabilOuerhani and Heinz Hiigli

  • Intro
  • Based on Itti & Koch saliency algorithm
  • Easy to compute in parallel
  • Easy to incorporate depth features
  • Only visual evaluation method
slide9

Top-Down overview

Integration

Feature extraction

Feature Map 1

Center Surround

Conspicuity Map 1

Range Finder

Conspicuity Map i

Saliency

Map

Feature Map i

Center Surround

Video

Camera

Feature Map n+m

Center Surround

Conspicuity Map n+m

slide10

Integration step

  • Assign weights that promotes conspicuity

M

Wi

Σ

m

Wi

Conspicuity map i

slide12

Limitation

  • No statistical analysis to prove improvements
  • No comparison with other methods
  • No quantities evaluation method
  • Proposed solution advantages
  • A new 3D dataset was created for statitical analysis
  • Evaluation and comparison with different methods
  • Experiments to investigate 3D saliency
slide13

Related work

Pre-Attentive detection of depth saliency using stereo vision(2010) in AIPR

Authors :M. Zaheer Aziz and BarbelMertsching

  • Intro
  • Depth approximation using stereo images
  • Relates only to depth saliency
slide14

Top-Down overview

IR

Algorithm

IL

Depth saliency magnitude

slide15

Preprocessing

Clipping

Smoothing

Segmentation

IR

IR

1 1

1 1 1

1 1 1

1/9

IL

Depth saliency magnitude

slide16

Key insight

IR

IL

IR

IL

slide17

Main Algorithm explained/1

Segmentation

IR

Δ(x, y)

Remove extra stripes

-

IL

slide18

Main Algorithm explained/2

Remove occluded stripes

Assign region depth

slide19

Experiments and results

  • Human subjects marked depth saliency labels
  • Efficiency factor defined
  • Capability factor defined

Nf

Nf

Nf<Ns

Ns

Ns

Nf>Ns

Penalty

Nf =# of labels found

Ns =# of labels

slide21

Related work

  • Limitation
  • Requires stereo images
  • Approximated depth calculation
  • Emphasis on runtime
  • Experiments somehow dubious
  • Self invented evaluation index
  • Hard to compare with other methods
  • Ignoring fusion with contrast saliency algo.
slide22

Related work

  • Proposed solution advantages
  • Messured depth data
  • Statistical analysis and comparison
  • Applicable on all 2D saliency algorithms
  • Using conventional evaluation index
slide23

Proposed solution

  • Top-Down overview
    • Dataset collection (3D/2D) and analysis
      • Extracting Stereoscopic image pair generation for 3D display
      • Perform experiments to gain fixation maps
      • Observations and Statistics
    • Incorporating depth priors
    • Experiments and results
slide25

Dataset Examples

  • Rejected images from dataset
    • images overlapping content with other images
    • Images with significant artifacts after the smoothing process
slide26

Apply smoothing to depth maps

D==0 ?

D==0 ?

1 1 1

1 0 1

1 1 1

Laplacian

Smooth

Depth map

1/8∙

In the same

super pixel

Depth Map

slide27

Stereoscopic image pair generation for 3D display

Experiments conducted using

active shutter glasses on a

3D LCD display

Pixels translation:

xl = xp+ρ, xr = xp−ρ

ρ = parallax/2

slide28

Quantitative evaluation methods/1

  • Three quantitative methods for performance evaluation
    • Correlation coefficient(for all images in dataset)
      • Correlation between to given maps for different depth range bins
    • Saliency Ratio
      • Saliency mean energy in depth bin
    • AUC of the ROC
      • Use each one saliency map to predict another
slide29

Quantitative evaluation methods/2

  • Correlation coefficient
  • Saliency Ratio
  • AUC of the ROC
slide30

Observations and statistics

  • Observation 1
  • Depth cues modulate visual saliency at farther depth ranges
  • humans fixate preferentially at closer depth ranges
slide31

Observations and statistics

  • Observation 2
  • few objects account for majority of the fixations

The average AUC for the entire 3D fixation dataset is 0.7399

and 0.7046 for 2D fixation dataset

Using fixation maps as predictor for labeled object

slide32

Observations and statistics

  • Observation 3
  • The relationship between depth and saliency is non-linear and characteristic for low and high depth-of-field scenes
slide33

Observations and statistics

  • Observation 4:
  • Fixation distribution discrepancy when multiple salient stimuli present different depth planes

CC – Correlation coefficient

slide34

Incorporating depth priors/1

  • Proposed pdf
    • P(s,d/DOF) is GMM (Gaussian Mixture Model)

Training set

(80% of the

dataset)

Down

Sample

EM

Algo.

GMM model

Parameters

For each

DOF bin

P(s(x)|d(x), DOF)

200x200 pix

640x480 pix

S(x) = ψ(x)(⊕/⊗)P(s/d,DOF)

slide35

Incorporating depth priors/2

GMM

Pdf

From training

Selected

GMM

Pdf

Calc

DOF

Depth map

Improved

Saliency map

+/*

Saliency map

Saliency

Algo.

Input image

slide39

Proposed solution

  • Depth saliency
slide40

Pros & Cons

  • Pros
    • Novel and provides a basis for further research
    • Simple approach
    • Gives an overall improvement for existing alg.
  • Cons
    • Few missing implementation details
    • No source code/other descriptive material
    • Requires a depth map
    • Still no basis for comparison with other methods

Future ideas

Integrate depth priors in various algo. instead of late fusion

ad