- By
**moana** - Follow User

- 131 Views
- Uploaded on

Download Presentation
## PowerPoint Slideshow about ' Diverse M-Best Solutions in Markov Random Fields' - moana

**An Image/Link below is provided (as is) to download presentation**

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

### Diverse M-Best Solutionsin Markov Random Fields

,

,

,

Dhruv Batra

TTI-Chicago / Virginia Tech

PaymanYadollahpour

TTI-Chicago

Abner Guzman-Rivera

UIUC

Greg Shakhnarovich

TTI-Chicago

Problems with MAP

Model-Class is Wrong!

-- Approximation Error

- Human Body ≠ Tree

(C) Dhruv Batra

Figure Courtesy: [Yang & Ramanan ICCV ‘11]

Problems with MAP

Model-Class is Wrong!

Not Enough Training Data!

-- Approximation Error

-- Estimation Error

(C) Dhruv Batra

Problems with MAP

Model-Class is Wrong!

Not Enough Training Data!

MAP is NP-Hard

-- Approximation Error

-- Estimation Error

-- Optimization Error

(C) Dhruv Batra

Problems with MAP

Model-Class is Wrong!

Not Enough Training Data!

MAP is NP-Hard

Inherent Ambiguity

-- Approximation Error

-- Estimation Error

-- Optimization Error

-- Bayes Error

?

?

Rotating clockwise / anti-clockwise?

Old Lady looking left / Young woman looking away?

One instance / Two instances?

(C) Dhruv Batra

Problems with MAP

Single Prediction = Uncertainty Mismanagement

Model-Class is Wrong!

Not Enough Training Data!

MAP is NP-Hard

Inherent Ambiguity

-- Approximation Error

-- Estimation Error

-- Optimization Error

-- Bayes Error

Make Multiple Predictions!

(C) Dhruv Batra

Multiple Predictions

x

x

x

x

x

x

x

x

x

x

x

x

x

Sampling

Porway & Zhu, 2011

TU & Zhu, 2002

Rich History

(C) Dhruv Batra

M-Best Modes

Multiple PredictionsM-Best MAP

Sampling

✓

Porway & Zhu, 2011

TU & Zhu, 2002

Rich History

Flerova et al., 2011

Fromeret al., 2009

Yanover et al., 2003

(C) Dhruv Batra

M-Best Modes

Multiple PredictionsM-Best MAP

Sampling

✓

This Paper: Diverse M-Best in MRFs

Porway & Zhu, 2011

TU & Zhu, 2002

Rich History

- Don’t hope for diversity. Explicitly encode it.
- Not guaranteed to be modes.

Flerova et al., 2011

Fromeret al., 2009

Yanover et al., 2003

(C) Dhruv Batra

Diverse M-Best

(C) Dhruv Batra

Diverse 2nd-Best

Q1: How do we solve DivMBest?

Q2: What kind of diversity functions are allowed?

Q3: How much diversity?

See Paper for Details

(C) Dhruv Batra

Diverse 2nd-Best

- Lagrangian Relaxation

Diversity-Augmented Energy

Many ways to solve:

upergradient Ascent. Optimal. Slow.

Primal

See Paper for Details

2. Binary Search. Optimal for M=2. Faster.

Dualize

3. Grid-search on lambda. Sub-optimal. Fastest.

Dual

Div2Best energy

Concave (Non-smooth)

Lower-Bound on Div2Best En.

(C) Dhruv Batra

Diverse 2nd-Best

Q1: How do we solve Div2Best?

Q2: What kind of diversity functions are allowed?

Q3: How much diversity?

See Paper for Details

(C) Dhruv Batra

Diversity

- [Special Case] 0-1 Diversity M-Best MAP
- [Yanover NIPS03; Fromer NIPS09; Flerova Soft11]
- [Special Case] Max Diversity [Park & RamananICCV11]
- Hamming Diversity
- Cardinality Diversity
- Any Diversity

See Paper for Details

(C) Dhruv Batra

Hamming Diversity

- Diversity Augmented Inference:

Unchanged. Can still use graph-cuts!

Simply edit node-terms. Reuse MAP machinery!

(C) Dhruv Batra

Experiments

- 3 Applications
- Interactive Segmentation: Hamming, Cardinality (in paper)
- Pose Estimation: Hamming
- Semantic Segmentation: Hamming
- Baselines:
- M-Best MAP (No Diversity)
- Confidence-Based Perturbation (No Optimization)
- Metrics
- Oracle Accuracies
- User-in-the-loop; Upper-Bound
- Re-ranked Accuracies

(C) Dhruv Batra

Experiment #1

- Interactive Segmentation
- Model: Color/Texture + Potts Grid CRF
- Inference: Graph-cuts
- Dataset: 50 train/val/test images

Image + Scribbles

MAP

2nd Best MAP

Diverse 2nd Best

1-2 Nodes Flipped

100-500 Nodes Flipped

(C) Dhruv Batra

Experiment #2

- Pose Tracking
- Model: Mixture of Parts from [Park & Ramanan, ICCV ‘11]
- Inference: Dynamic Programming
- Dataset: 4 videos, 585 frames

(C) Dhruv Batra

Image Credit: [Yang & Ramanan, ICCV ‘11]

Experiment #2

- Pose Tracking w/ Chain CRF

M BestSolutions

(C) Dhruv Batra

Image Credit: [Yang & Ramanan, ICCV ‘11]

Experiment #2

Better

DivMBest (Re-ranked)

13% Gain

Same FeaturesSame Model

[Park & Ramanan, ICCV ‘11] (Re-ranked)

PCP Accuracy

Confidence-based Perturbation (Re-ranked)

#Solutions / Frame

(C) Dhruv Batra

Experiment #3

- Semantic Segmentation
- Model: Hierarchical CRF [Ladicky et al. ECCV ’10, ICCV ‘09]
- Inference: Alpha-expansion
- Dataset: Pascal Segmentation Challenge (VOC 2010)
- 20 categories + background; 964 train/val/test images

(C) Dhruv Batra

- Image Credit: [Ladicky et al. ECCV ’10, ICCV ’09]

Experiment #3

DivMBest (Oracle)

Better

MAP

22%-gain possible

Same FeaturesSame Model

PACAL Accuracy

DivMBest (Re-ranked) [Yadollahpouret al.]

Confidence-based Perturbation (Oracle)

#Solutions / Image

(C) Dhruv Batra

Summary

- All models are wrong
- Some beliefs are useful
- DivMBest
- First principled formulation for Diverse M-Best in MRFs
- Efficient algorithm. Re-uses MAP machinery.
- Big impact possible on many applications!

(C) Dhruv Batra

Thank you!

- Think about YOUR problem.
- Are you or a loved one, tired of a single solution?
- If yes, then DivMBest might be right for you!*

* DivMBest is not suited for everyone. People with perfect models, and love of continuous variables should not use DivMBest. Consult your local optimization expert before startingDivMBest. Please do not drive or operate heavy machinery while on DivMBest.

(C) Dhruv Batra

Download Presentation

Connecting to Server..