1 / 40

GRAPH CUTS BY USING LOCAL TEXTURE FEATURES OF WAVELET COEFFICIENT FOR IMAGE SEGMENTATION

GRAPH CUTS BY USING LOCAL TEXTURE FEATURES OF WAVELET COEFFICIENT FOR IMAGE SEGMENTATION. Keita Fukuda, Tetsuya Takiguchi, Yasuo Ariki Graduate School of Engineering, Kobe University, Japan. Table of contents. Introduction conventional method, problem with graph cuts Proposed method

kolton
Download Presentation

GRAPH CUTS BY USING LOCAL TEXTURE FEATURES OF WAVELET COEFFICIENT FOR IMAGE SEGMENTATION

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. GRAPH CUTS BY USING LOCAL TEXTURE FEATURES OF WAVELET COEFFICIENT FOR IMAGE SEGMENTATION Keita Fukuda, Tetsuya Takiguchi, Yasuo Ariki Graduate School of Engineering, Kobe University, Japan 2008 International Conference on Multimedia & Expo

  2. Table of contents • Introduction conventional method, problem with graph cuts • Proposed method iterated graph cuts using texture and smoothing and detail of each method • Experiments • Conclusion and Future Work 2008 International Conference on Multimedia & Expo

  3. “obj” “bkg” Introduction Extracting the foreground objects in static images is one of the most fundamental tasks (Image Segmentation Problem). 2008 International Conference on Multimedia & Expo

  4. S O B T Background Recently, the image segmentation problem is formalized as an optimal solution problem. e.g. Snakes, Level Set Method, Graph Cuts Y.Boykov, M.P.Jolly,“Interactive graph cuts for optimal boundary & region segmentation of object in N-D images”, IEEE International Conference on Computer Vision and Pattern Recognition Advantage of Graph Cuts • possible to compute global optimal solution The cost function is general enough to include both region and boundary properties of the segments 2008 International Conference on Multimedia & Expo

  5. S:”obj” O B S min cut T:”bkg” O B T Graph Cuts Object likelihood Neighbor similarity Construct a graph n-link A 3-by-4 image Searching the minimum cost cut t-link Background likelihood Assigned the corresponding label Segmentation result 2008 International Conference on Multimedia & Expo

  6. Problem (1) with GC It has been difficult to segment images that include complex noisy edges. • To solve this problem, using iterated graph cuts based on smoothing. T. Nagahashi, H.Fujiyoshi, and T.Kanade，“ Image Segmentation Using Iterated Graph Cuts Based on Multi-scale Smoothing， ”ACCV2007 Noise exists in strong edge 2008 International Conference on Multimedia & Expo

  7. Proposed method Iterated Graph Cuts based on local texture features as well as low frequency features of wavelet coefficient. Problem (2) with GC It is difficult to segment images with an object whose color is similar to the background. • To solve this problem, employing thetexture likelihood as well as color likelihood. The color of object is similar to that of background 2008 International Conference on Multimedia & Expo

  8. Proposed Method seed LL: Smoothing Multiresolution analysis (level k) LL,LH,HL,HHare obtained Graph Cuts Segmentation Input Output n-link t-link LH,HL,HH: Local texture features Prior probability GMM(color+texture)update k ← k-1 Graph Cuts segmentation is carried out, and these processes are repeated until k = 0. The likelihood are derived from local texture features as well as color. And the prior probabilities are defined using the previous segmentation result. After initializing the level k, the input image is decomposed into subbands using multiresolution wavelet analysis. To solve problem (2) with GC, high pass subbands (LHk,HLk,HHk) are used for t-link. Local texture features are defined from them. To solve problem (1) with GC, low pass subband (LLk) is used for n-link. The obtained neighbor similarity is set to n-link as edge cost. 2008 International Conference on Multimedia & Expo

  9. HL2 LL2 LL1 HL1 HL1 LH2 HH2 Input image LH1 HH1 LH1 HH1 Level 1 Level 2 Multiresolution analysis downsampled downsampled Subbands information for n-link LL: low-pass information LH: vertical HL: horizontal HH: diagonal orientation for t-link Input image Level 1 Level 2 2008 International Conference on Multimedia & Expo

  10. Level 3 Level 2 Level 1 coarse fine p q n-link Global to local segmentation can be performed by iterated Graph Cuts with multiresolution analysis using from coarse to fine level for n-link . B{p,q} is large when p and q are similar, it is close to 0 when p and q are very different. 2008 International Conference on Multimedia & Expo

  11. Local texture features d: wavelet coefficient k: level Local texture features Local texture features are defined by averaging the absolute wavelet coefficient in the window (3×3) surrounding pixel p Local texture features Tpare larger in complex region, and smaller in flat region. LH (level 1) HH (level 1) HL (level 1) 2008 International Conference on Multimedia & Expo

  12. S:”obj” p T:”bkg” t-link Object likelihood Pr(Yp|B) Color (RGB) :Cp Local texture:Tp GMM －ln Pr(B|Yp) Distance trans. Pr(B) Background Pr(Yp|O) Color (RGB) :Cp Local texture:Tp GMM －ln Pr(O|Yp) Distance trans. Background likelihood Object Pr(O) 2008 International Conference on Multimedia & Expo

  13. Graph Cuts Segmentation S Edge cost of the graph mincut O B T The boundary between the object and the background is found by searching the minimum cost cut on the graph Yuri Boykov and Vladimir Kolmogorov, “An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Computer Vision” 2008 International Conference on Multimedia & Expo

  14. S:”obj” S:”obj” O O B B T:”bkg” T:”bkg” Summarize proposed method Multiresolution at level k Multiresolution at level k-1 Multiresolution level is set to k-1 The prior probability t-link (texture with color) n-link Segmentation result 2008 International Conference on Multimedia & Expo

  15. Experimental Condition • We compared using the same labels: method (1) Interactive Graph Cuts [Boykov 04] method (2) Iterated Graph Cuts using smoothing [Nagahashi 07] Proposed Iterated Graph Cuts using smoothing and texture • Berkley Image database (50 images) • The segmentation error rate is defined as follow: = Mask image Error (in red and blue ) Output image 2008 International Conference on Multimedia & Expo

  16. Experimental Result Err (%) in segmentation result at multiresolution level k • The proposed method improved the error rate compared to two conventional methods. • In this database, two conventional methods could achieve the low error rate. ⇒ The examples with high error rate are shown from next page. 2008 International Conference on Multimedia & Expo

  17. Experimental result (1) • Effect of smoothing Method (1) Method (2) Proposed 6.27 % 1.16 % 0.85 % 3.69 % 0.83 % 0.79 % The method (2) and proposed method can achieve the better image segmentation for the images with complex edges than the method (1). 2008 International Conference on Multimedia & Expo

  18. 10.44 % 8.28 % 2.86 % 11.38 % 16.64 % 2.15 % Experimental result (2) • Effect of local texture features Method (1) Method (2) Proposed The proposed method can achieve the better image segmentation for theimages with object colors similar to background than method (1) and(2) 2008 International Conference on Multimedia & Expo

  19. Discussion • To solve problem (1) : method (2), proposed • Prior probability The brief shape information can be given from previous segmentation result • Smoothing removing local strong edges. • To solve problem (2) ：proposed • Local texture features for the image with object colors similar to background. 2008 International Conference on Multimedia & Expo

  20. Summary Graph Cuts by using local texture features of wavelet coefficient for image segmentation. New concept: Graph Cuts segmentation based on local texture features as well as smoothing process. • In future work: • The weight to texture and color for segmentation • Others texture features. 2008 International Conference on Multimedia & Expo

  21. Thank you for your attention ! 2008 International Conference on Multimedia & Expo

  22. The cost function The soft constraints that we impose of label f are described by the cost function E(f) as follow: The regional term assumes that the individual penalties for assigning pixel p to labels. The boundary term assumes that a penalty for a discontinuity between p and q 2008 International Conference on Multimedia & Expo

  23. Ｓ Ｓ 4/10 3/3 10 10 3 3 5 5 5/5 2/2 5/6 2 2 6 6 Ｓ 8 3 0/2 0/6 2 2 2 6 6 5 2/7 7 7 2 2 9 9 2 6 2/2 8/9 Ｔ Ｔ 2 6 7 2 9 Ｔ Minimum cost cut • 1. Graph Construct • Edge label = capacity • Edge = pipe • Goal : max flow from S to T 2. Residual graph Find augmenting paths until finished. 3. Current flow from residual graph e.g. 8/9 = (flow) / (capacity) Example of a graph Residual Graph Current flow 2008 International Conference on Multimedia & Expo

  24. Ｓ 10 3 5 2 6 2 6 7 2 9 Ｔ S T T Minimum cost cut • 1. Graph Construct • Edge label = capacity • Edge = pipe • Goal : max flow from S to T Ｓ mincut 4/10 3/3 10 3 5 5/5 2/2 5/6 2 6 2. Residual graph Find augmenting paths until finished. 0/2 0/6 2 6 2/7 7 2 9 2/2 8/9 3. Current flow from residual graph e.g. 8/9 = (flow) / (capacity) Ｔ 4. Minimum cost cut Saturated edges are found from 2. Each node is assigned for the corresponding labels. Yuri Boykov and Vladimir Kolmogorov, “An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Computer Vision” 2008 International Conference on Multimedia & Expo

  25. Experimental result (3) • for the unclear boundary images. Method (1) Method (2) Proposed The proposed method is effective for images which have the unclear boundary between object and background. 2008 International Conference on Multimedia & Expo

  26. Time cost Time cost The proposed method carried out 3 times graph cuts segmentation. And Gaussian Mixture Model is derived from 6 dimensional features. It is because the proposed method is slow. 2008 International Conference on Multimedia & Expo

  27. Discussion (2) • Remained problem The proposed method is ineffective for a flat image such as artificial images due to a few edges. Future work include optimization of the weight to texture and color for segmentation. 2008 International Conference on Multimedia & Expo

  28. S S T T p p q q Noisy edge problem If the strong edges exist , Else , n-link edge cost is small. (easy to pass the n-link edge) n-link edge cost is large. (hardly pass the n-link edge) 2008 International Conference on Multimedia & Expo

  29. Influence of seeds position • Graph cuts technique is quite stable and normally produces the same results regardless of particular seed positioning within the same image. 2008 International Conference on Multimedia & Expo

  30. Prior probability Prior probability • Distance d from the border between the object and background is normalized from 0.5 to 1.0 . • The prior probability is defined by using dojb and dbkg. 2008 International Conference on Multimedia & Expo

  31. S:”obj” p Color (RGB) :Cp Local texture:Tp Color (RGB) :Cp Local texture:Tp GMM GMM T:”bkg” First time segmentation Pr(Yp|B) －ln Pr(Yp|B) “bkg” seeds Pr(Yp|O) “obj” seeds －ln Pr(Yp|O) 2008 International Conference on Multimedia & Expo

  32. Multiresolution level At level 2 At level 3 The low pass subband LL3 is 1 / 64 * the original image. The low pass subband LL4 is 1 / 256 * the original image. 2008 International Conference on Multimedia & Expo

  33. Parameter Graph Cuts parameter : λ = 0.05 σ = 12.0 The constant K is larger than sum of all n-link costs not to label the opposite label. Gaussian Mixture Model: 5 components 2008 International Conference on Multimedia & Expo

  34. Local Texture features 2008 International Conference on Multimedia & Expo

  35. Results of proposed method 2008 International Conference on Multimedia & Expo

  36. Result of proposed method 2008 International Conference on Multimedia & Expo

  37. Question • Please repeat the question. • Thank you for your question. Are you asking about … ? • What you see here is … • I believe that … • To be honest , we never looked into possibility. • The question was … • There is not the particular deep meaning. • Not yet • Thank you for your advice. 2008 International Conference on Multimedia & Expo

  38. 追加スライド • n-linkの問題について • ユーザの引き方に問題は？ • 数パーセントでも違いがはっきりしている • パラメータの説明 • Level4がない理由 • 問題点 • 一回目のセグメンテーション 2008 International Conference on Multimedia & Expo

  39. Ｓ 10 3 5 2 6 2 6 7 2 9 Ｔ Minimum cost cut 2+2+5+3 = 12 (mincut) 10+2+5+6+9 = 32 10+5+6+9 = 30 2008 International Conference on Multimedia & Expo

  40. Ｓ 8 3 2 5 2 6 2 6 7 2 9 Ｔ input labeled mask output error 2008 International Conference on Multimedia & Expo

More Related