- 103 Views
- Uploaded on
- Presentation posted in: General

Diverse M-Best Solutions in Markov Random Fields

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Diverse M-Best Solutionsin Markov Random Fields

,

,

,

Dhruv Batra

TTI-Chicago / Virginia Tech

PaymanYadollahpour

TTI-Chicago

Abner Guzman-Rivera

UIUC

Greg Shakhnarovich

TTI-Chicago

- Graphical Models

Hat

x1

x2

MAP

Inference

…

xn

Cat

Most Likely Assignment

(C) Dhruv Batra

Model-Class is Wrong!

-- Approximation Error

- Human Body ≠ Tree

(C) Dhruv Batra

Figure Courtesy: [Yang & Ramanan ICCV ‘11]

Model-Class is Wrong!

Not Enough Training Data!

-- Approximation Error

-- Estimation Error

(C) Dhruv Batra

Model-Class is Wrong!

Not Enough Training Data!

MAP is NP-Hard

-- Approximation Error

-- Estimation Error

-- Optimization Error

(C) Dhruv Batra

Model-Class is Wrong!

Not Enough Training Data!

MAP is NP-Hard

Inherent Ambiguity

-- Approximation Error

-- Estimation Error

-- Optimization Error

-- Bayes Error

?

?

Rotating clockwise / anti-clockwise?

Old Lady looking left / Young woman looking away?

One instance / Two instances?

(C) Dhruv Batra

Single Prediction = Uncertainty Mismanagement

Model-Class is Wrong!

Not Enough Training Data!

MAP is NP-Hard

Inherent Ambiguity

-- Approximation Error

-- Estimation Error

-- Optimization Error

-- Bayes Error

Make Multiple Predictions!

(C) Dhruv Batra

x

x

x

x

x

x

x

x

x

x

x

x

x

Sampling

Porway & Zhu, 2011

TU & Zhu, 2002

Rich History

(C) Dhruv Batra

Ideally:

M-Best Modes

M-Best MAP

Sampling

✓

Porway & Zhu, 2011

TU & Zhu, 2002

Rich History

Flerova et al., 2011

Fromeret al., 2009

Yanover et al., 2003

(C) Dhruv Batra

Ideally:

M-Best Modes

M-Best MAP

Sampling

✓

This Paper: Diverse M-Best in MRFs

Porway & Zhu, 2011

TU & Zhu, 2002

Rich History

- Don’t hope for diversity. Explicitly encode it.
- Not guaranteed to be modes.

Flerova et al., 2011

Fromeret al., 2009

Yanover et al., 2003

(C) Dhruv Batra

kx1

(C) Dhruv Batra

1

0

0

0

kx1

(C) Dhruv Batra

0

1

0

0

kx1

(C) Dhruv Batra

0

0

1

0

kx1

(C) Dhruv Batra

0

0

0

1

kx1

(C) Dhruv Batra

0

0

0

1

kx1

k2x1

(C) Dhruv Batra

0

0

0

1

kx1

k2x1

(C) Dhruv Batra

Graphcuts, BP, Expansion, etc

(C) Dhruv Batra

Diversity

MAP

(C) Dhruv Batra

(C) Dhruv Batra

Q1: How do we solve DivMBest?

Q2: What kind of diversity functions are allowed?

Q3: How much diversity?

See Paper for Details

(C) Dhruv Batra

- Lagrangian Relaxation

Diversity-Augmented Energy

Many ways to solve:

upergradient Ascent. Optimal. Slow.

Primal

See Paper for Details

2. Binary Search. Optimal for M=2. Faster.

Dualize

3. Grid-search on lambda. Sub-optimal. Fastest.

Dual

Div2Best energy

Concave (Non-smooth)

Lower-Bound on Div2Best En.

(C) Dhruv Batra

Q1: How do we solve Div2Best?

Q2: What kind of diversity functions are allowed?

Q3: How much diversity?

See Paper for Details

(C) Dhruv Batra

- [Special Case] 0-1 Diversity M-Best MAP
- [Yanover NIPS03; Fromer NIPS09; Flerova Soft11]

- [Special Case] Max Diversity [Park & RamananICCV11]
- Hamming Diversity
- Cardinality Diversity
- Any Diversity

See Paper for Details

(C) Dhruv Batra

0

1

0

0

0 1 0 0

0

1

0

0

1 0 0 0

(C) Dhruv Batra

- Diversity Augmented Inference:

(C) Dhruv Batra

- Diversity Augmented Inference:

Unchanged. Can still use graph-cuts!

Simply edit node-terms. Reuse MAP machinery!

(C) Dhruv Batra

- 3 Applications
- Interactive Segmentation: Hamming, Cardinality (in paper)
- Pose Estimation: Hamming
- Semantic Segmentation: Hamming

- Baselines:
- M-Best MAP (No Diversity)
- Confidence-Based Perturbation (No Optimization)

- Metrics
- Oracle Accuracies
- User-in-the-loop; Upper-Bound

- Re-ranked Accuracies

- Oracle Accuracies

(C) Dhruv Batra

- Interactive Segmentation
- Model: Color/Texture + Potts Grid CRF
- Inference: Graph-cuts
- Dataset: 50 train/val/test images

Image + Scribbles

MAP

2nd Best MAP

Diverse 2nd Best

1-2 Nodes Flipped

100-500 Nodes Flipped

(C) Dhruv Batra

+3.62%

+1.61%

+0.05%

(Oracle)

(Oracle)

(Oracle)

M=6

(C) Dhruv Batra

- Pose Tracking
- Model: Mixture of Parts from [Park & Ramanan, ICCV ‘11]
- Inference: Dynamic Programming
- Dataset: 4 videos, 585 frames

(C) Dhruv Batra

Image Credit: [Yang & Ramanan, ICCV ‘11]

- Pose Tracking w/ Chain CRF

M BestSolutions

(C) Dhruv Batra

Image Credit: [Yang & Ramanan, ICCV ‘11]

MAP

DivMBest + Viterbi

(C) Dhruv Batra

Better

DivMBest (Re-ranked)

13% Gain

Same FeaturesSame Model

[Park & Ramanan, ICCV ‘11] (Re-ranked)

PCP Accuracy

Confidence-based Perturbation (Re-ranked)

#Solutions / Frame

(C) Dhruv Batra

- Semantic Segmentation
- Model: Hierarchical CRF [Ladicky et al. ECCV ’10, ICCV ‘09]
- Inference: Alpha-expansion
- Dataset: Pascal Segmentation Challenge (VOC 2010)
- 20 categories + background; 964 train/val/test images

(C) Dhruv Batra

- Image Credit: [Ladicky et al. ECCV ’10, ICCV ’09]

Input

MAP

Best of 10-Div

(C) Dhruv Batra

DivMBest (Oracle)

Better

MAP

22%-gain possible

Same FeaturesSame Model

PACAL Accuracy

DivMBest (Re-ranked) [Yadollahpouret al.]

Confidence-based Perturbation (Oracle)

#Solutions / Image

(C) Dhruv Batra

- All models are wrong
- Some beliefs are useful
- DivMBest
- First principled formulation for Diverse M-Best in MRFs
- Efficient algorithm. Re-uses MAP machinery.
- Big impact possible on many applications!

(C) Dhruv Batra

- Think about YOUR problem.
- Are you or a loved one, tired of a single solution?
- If yes, then DivMBest might be right for you!*
* DivMBest is not suited for everyone. People with perfect models, and love of continuous variables should not use DivMBest. Consult your local optimization expert before startingDivMBest. Please do not drive or operate heavy machinery while on DivMBest.

(C) Dhruv Batra