1 / 45

Adaptive Segmentation Based on a Learned Quality Metric

Adaptive Segmentation Based on a Learned Quality Metric. I. Frosio 1 , E. Ratner 2 1 NVIDIA, USA, 2 Lyrical Labs, USA. Motivation: good / bad segmentation. SLIC ( Achanta , 2012). Motivation: good / bad segmentation. GRAPH-CUT ( Felzenszwalb , 2004).

danal
Download Presentation

Adaptive Segmentation Based on a Learned Quality Metric

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Adaptive Segmentation Based on a Learned Quality Metric I. Frosio1, E. Ratner2 1 NVIDIA, USA, 2 Lyrical Labs, USA

  2. Motivation: good / bad segmentation SLIC (Achanta, 2012)

  3. Motivation: good / bad segmentation GRAPH-CUT (Felzenszwalb, 2004)

  4. Motivation: good / bad segmentation ADAPTIVE GRAPH-CUT (our)

  5. Motivation: good / bad segmentation > > ? ? ? SLIC (Achanta, 2012) GRAPH-CUT (Felzenszwalb, 2004) ADAPTIVE GRAPH-CUT (our)

  6. Motivation: good / bad segmentation • Achanta, 2012 (SLIC); Kaufhold, 2004: segmentation algorithms aggregate sets of perceptually similar pixels in an image. • Felzenszwalb, 2004 (graph-cut): a segmentation algorithm should capture perceptually important groupings or regions, which often reflect global aspects of the image.

  7. Motivation: segmentation & video compression Segment motion estimation Frame segmentation True block and sub-block motion vectors Encoding

  8. Aim #1: use the human factor(aka segmentation quality metric)

  9. Aim #2: automatic parameter tuning

  10. Road map 3) … And put them together (autotuning). 2) … Learn a quality metric including the human factor (application needs) … 1) Pick a segmentation algorithm…

  11. Graph-cut Graph: Nodes: Edges: Weights: w(vi, vj)>>0 w(vi, vj)>0 vj vi w(vi, vj)=0

  12. Graph-cut Internal difference: Cm

  13. Graph-cut Difference between components: Cm Cn

  14. Graph-cut Boundary predicate: Ck 12 10 15 Cn

  15. Graph-cut Boundary predicate: C1 11 15 8 C2

  16. Graph-cut Boundary predicate: Observation scale ~ k C1 C2

  17. Graph-cut K = 3 K = 10,000 K = 100

  18. Road map 3) … And put them together (autotuning). 2) … Learn a quality metric including the human factor… 1) Pick a segmentation algorithm…

  19. (Weighted) symmetric uncertainty 4 bits ------------------ = 33% 7 bits + 5 bits Entropy based average

  20. k vs. Uw vs. quality 160 x 120 image block

  21. k vs. Uw vs. quality Training 160 x 120 blocks 320x240 rgb images K = [1, …, 10,000] visual inspection & classification

  22. k vs. Uw vs. quality Training 160 x 120 blocks 640x480 rgb images K = [1, …, 10,000] visual inspection & classification

  23. Learning the metric Uw = m log(k) + b

  24. Road map 3) … And put them together (autotuning). 2) … Learn a quality metric including the human factor… 1) Pick a segmentation algorithm…

  25. Automatic k selection

  26. Automatic k selection

  27. Automatic k selection

  28. Automatic k selection

  29. Automatic k selection

  30. … and adaptivity k = k(x,y)

  31. Road map

  32. Results - Quality Adaptive graph-cut (ours) Graph-cut (Felzensswalb, 2004) * SLIC (Achanta, 2012) * * Same number of segments forced for each algorithm

  33. Results

  34. Results SLIC Graph-cut Adaptive graph-cut

  35. Results

  36. Results Graph-cut Adaptive graph-cut SLIC

  37. Results: inter-class contrast(the higher the better) Sum of the contrasts among segments weighted by their areas (Chabrier, 2004)

  38. Results: intra-class uniformity(the lower the better) Sum of the normalized standard deviation for each region (Chabrier, 2004)

  39. Results: contrast - uniformity ratio(the higher the better)

  40. Discussion • LEARNED segmentation quality metric including the HUMAN FACTOR • Iterative method to AUTOMATICALLY and ADAPTIVELY compute the optimal scale parameter

  41. A more general approach(edge thresholding segmentation in YUV)

  42. A more general approach(edge thresholding segmentation in YUV) Openboradcast encoding (x264) Lyricallabs encoding (adaptive segmentation) Show

  43. A more general approach(edge thresholding segmentation in YUV) Openboradcast encoding (x264) Lyricallabs encoding (adaptive segmentation) Show

  44. Open issues & improvements Resolution dependency (160x120 blocks) Learning: the Berkeley Segmentation Dataset Avoid iterations (see I. Frosio, SPIE EI 2015)

  45. Questions ? ? ?

More Related