1 / 38

Introduction of Saliency Map

Introduction of Saliency Map. Presenter: Chien-Chi Chen Advisor: Jian-Jiun Ding. Outline. Introduction of saliency map Button-up approach L. Itti’s approach Frequency-tuned Multi-scale contrast Depth of field Spectral Residual approach Global contrast based Top-down approach

karah
Download Presentation

Introduction of Saliency Map

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Introduction of Saliency Map Presenter: Chien-Chi Chen Advisor: Jian-Jiun Ding

  2. Outline • Introduction of saliency map • Button-up approach • L. Itti’s approach • Frequency-tuned • Multi-scale contrast • Depth of field • Spectral Residual approach • Global contrast based • Top-down approach • Context-aware • Information maximum • Measuring visual saliency by site entropy rate

  3. Outline • Introduction of saliency map • Button-up approach • L. Itti’s approach • Frequency-tuned • Multi-scale contrast • Depth of field • Spectral Residual approach • Global contrast based • Top-down approach • Context-aware • Information maximum • Measuring visual saliency by site entropy rate

  4. Introduction of saliency map • Low-level(contrast) • Color • Orientation • Size • Motion • Depth • High-level • People • Context Important! Low-level With face detection Judd et al, 2009

  5. Outline • Introduction of saliency map • Button-up approach • L. Itti’s approach • Frequency-tuned • Multi-scale contrast • Depth of field • Spectral Residual approach • Global contrast based • Top-down approach • Context-aware • Information maximum • Measuring visual saliency by site entropy rate

  6. L. Itti’s approach • Architecture: Gaussian Pyramids R,G,B,Y Gabor pyramids for q = {0º, 45º, 90º, 135º}

  7. L. Itti’s approach • Center-surround Difference • Achieve center-surround difference through across-scale difference • Operated denoted by Q: Interpolation to finer scale and point-to-point subtraction • One pyramid for each channel: I(s), R(s), G(s), B(s), Y(s)where sÎ [0..8] is the scale

  8. L. Itti’s approach • Center-surround Difference • Intensity Feature Maps • I(c, s) = | I(c)QI(s)| • cÎ {2, 3, 4} • s = c + d where dÎ {3, 4} • So I(2, 5) = | I(2) QI(5)|I(2, 6) = | I(2) QI(6)|I(3, 6) = | I(3) QI(6)| … •  6 Feature Maps

  9. L. Itti’s approach Center-surround Difference Color Feature Maps Center-surround Difference Orientation Feature Maps Red-Green and Yellow-Blue Same c and s as with intensity +B-Y +Y-B +R-G +G-R +B-Y +G-R +Y-B +B-Y +R-G RG(c, s) = | (R(c) - G(c)) Q (G(s) - R(s)) | BY(c, s) = | (B(c) - Y(c)) Q (Y(s) - B(s)) |

  10. L. Itti’s approach • Normalization Operator • Promotes maps with few strong peaks • Surpresses maps with many comparable peaks • Normalization of map to range [0…M] • Compute average m of all local maxima • Find the global maximum M • Multiply the map by (M – m)2

  11. L. Itti’s approach Example of Operation: Inhibition of return

  12. Frequency-tuned Image Average Gaussian blur

  13. Multi-scale contrast • Saliency algorithm Multi-scale contrast Center-surround histogram Conditional Random Field Image Saliency map Color spatial-distribution

  14. Multi-scale contrast Multi-scale contrast Center-surround histogram Distance between histograms of RGB color: • Local summation of laplacian pyramid

  15. Multi-scale contrast • Color spatial-distribution The variance of Coordinate of pixel x and y Image(RGB) GMM Distance from pixel x to image center

  16. Multi-scale contrast • Energy term: • Saliency object: • Pairwise feature:

  17. Multi-scale contrast • CRF: • The derivative of the log-likelihood with respect to

  18. Depth of field • As the spread of single lens reflex camera, more and more low depth of field(DOF) images are captured. • However, current saliency detection methods work poorly for the low DOF images.

  19. Depth of field • Algorithm:

  20. Depth of field • Classification: • Focal Point: In a low DOF image Rectangle with the highest edge density, and center is initial focal point DOG • Composition Analysis: segmentation Region

  21. Spectral Residual Approach • First scaling image to 64x64. • Then we smoothed the saliency map with a gaussian filter g(x) ( = 8).

  22. Global contrast-based • Histogram based contrast(Lab): Quantization of Lab Each channel to have 12 different value 85

  23. Global contrast-based • Region based contrast • Segment the Image • [Efficient graph-based image segmentation]

  24. Outline • Introduction of saliency map • Button-up approach • L. Itti’s approach • Frequency-tuned • Center-surround • Depth of field • Spectral Residual approach • Global contrast based • Top-down approach • Context-aware • Information maximum • Measuring visual saliency by site entropy rate

  25. Context-Aware • Goal: Convey the image content Liu et al, 2007

  26. Context-Aware • Distance between a pair of patches: High salient

  27. Context-Aware • Distance between a pair of patches: K most similar patches at scale r High for K most similar Saliency

  28. Context-Aware • Salient at: • Multiple scales  foreground • Few scales  background Scale 1 Scale 4

  29. Context-Aware • Foci = • Include distance map X

  30. Outline • Introduction of saliency map • Button-up approach • L. Itti’s approach • Frequency-tuned • Center-surround • Depth of field • Spectral Residual approach • Global contrast based • Top-down approach • Context-aware • Information maximum • Measuring visual saliency by site entropy rate

  31. Measuring visual saliency by site entropy rate 1

  32. Measuring visual saliency by site entropy rate 2 A fully-connected graph representation is adopted for each

  33. Sub-band graph representation

  34. Sub-band graph representation

  35. Measuring visual saliency by site entropy rate 3 A random walk is adopted on each sub-band graph. And Site entropy rate(SER) is measured the average information from a node to the other

  36. The site entropy rate

  37. Conclusion • Image processing is funny • Unusual in its neighborhood will correspond to high saliency weight • Contrast is the key of saliency

  38. Reference [1] R. Achanta, F. Estrada, P. Wils, and S. S¨usstrunk. Salient region detection and segmentation. In ICVS, pages 66–75. Springer, 2008. 410, 412, 414 [2] R. Achanta, S. Hemami, F. Estrada, and S. S¨usstrunk. Frequency-tuned salient region detection. In CVPR, pages 1597–1604, 2009. 409, 410, 412, 413, 414, 415 [3] L. Itti, C. Koch, and E. Niebur. A model of saliency based visual attention for rapid scene analysis. IEEE TPAMI, 20(11):1254–1259, 1998. 409, 410, 412, 414 [4] X. Hou and L. Zhang. Saliency detection: A spectral residual approach. In CVPR, pages 1–8, 2007. 410, 412, 413, 414 [5] S. Goferman, L. Zelnik-Manor, and A. Tal. Context-aware saliency detection. In CVPR, 2010. 410, 412, 413, 414, 415 [6] MM Cheng, GX Zhang, N. J. Mitra, X. Huang, S.M. Hu. Global Contrast based Salient Region Detect. In CVPR, 2011 . [7] T. Liu, Z. Yuan, J. Sun, J.Wang, N. Zheng, T. X., and S. H.Y. Learning to detect a salient object. IEEE TPAMI, 33(2):353–367, 2011. 410 [8] W. Wang, Y. Wang, Q. Huang, W. Gao, Measuring Visaul Saliency by Site Entropy Rate, In CVPR, 2010.

More Related