1 / 17

Bi-layer segmentation of binocular stereo video

deidra
Download Presentation

Bi-layer segmentation of binocular stereo video

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

    2. Problem Two layers (Bg and Fg) Scenarios: video-conferencing, live bluescreening without bluescreen Two cameras Background may not be static Goal: accurately segment foreground object in real-time Applications: background substitution, ...

    3. Sources of information Stereo Foreground object has larger disparity Colour Background and foreground have distinct colour distributions Contrast There is image gradient at Bg/Fg transition Spatial coherence MRF model live demo...live demo...

    4. Previous work Colour/Contrast (+2D coherence) Graph cuts (Boykov et al. 01, Rother et al. 04)

    5. Fusing colour/contrast and stereo Colour and stereo complement each other Result from fusion:

    6. Ideally: Fusing colour/contrast and stereo

    7. Approximations Simplify the model to get real-time performance Two different approaches: Layered Dynamic Programming (LDP) Layered Graph Cut (LGC) Probabilistic formulation Parameters can mostly be set automatically Very similar error statistics Consistently better than colour/contrast or stereo alone probabilistic formulation aids fusion of different cuesprobabilistic formulation aids fusion of different cues

    8. Layered Dynamic Programming (LDP) Approximation: in the prior neglect coupling between scanlines

    9. Layered Graph Cut (LGC) Approximation: in the prior neglect conditioning of disparity dp on disparities of neighbours conditioned only on segmentation label xp?{Bg,Fg,Occ} Marginalise disparities out Energy minimisation problem with 3 labels Solve it using 2 graph cut computations Approx. 20 frames per second (320 x 240, 3GHz)

    10. Setting parameters Two approaches: Generative: from physics (e.g. from average width of occluded regions) Discriminative: minimize error rates Consistent results! See technical report 2005 (http://research.microsoft.com/vision/cambridge/i2i/)

    11. Experiments: ground truth data 19 calibrated stereo sequences 6 with ground truth segmentation Every 5th or 10th frame Pixels marked as Bg, Fg or Unknown

    12. Accuracy of segmentation

    13. Accuracy of segmentation

    14. Accuracy of segmentation

    15. Conclusion Two algorithms based on different approximations Fuse colour/constrast, stereo, and spatial coherence Probabilistic formulation Capable of real-time performance Similar error statistics Consistently better than state-of-the art techniques Different characteristics LDP: Parallelisable (scanlines processed independently) LGC: Marginalisation could be done on GPU MSN: i2i cambridge

    17. Segmentation errors (LGC)

    18. Segmentation errors (LGC)

More Related