Diverse m best solutions in markov random fields
This presentation is the property of its rightful owner.
Sponsored Links
1 / 39

Diverse M-Best Solutions in Markov Random Fields PowerPoint PPT Presentation


  • 81 Views
  • Uploaded on
  • Presentation posted in: General

Diverse M-Best Solutions in Markov Random Fields. ,. ,. ,. Dhruv Batra TTI-Chicago / Virginia Tech. Payman Yadollahpour TTI-Chicago. Abner Guzman-Rivera UIUC. Greg Shakhnarovich TTI-Chicago. Local Ambiguity. Graphical Models. Hat. x 1. x 2. MAP Inference. …. x n. C at.

Download Presentation

Diverse M-Best Solutions in Markov Random Fields

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Diverse m best solutions in markov random fields

Diverse M-Best Solutionsin Markov Random Fields

,

,

,

Dhruv Batra

TTI-Chicago / Virginia Tech

PaymanYadollahpour

TTI-Chicago

Abner Guzman-Rivera

UIUC

Greg Shakhnarovich

TTI-Chicago


Local ambiguity

Local Ambiguity

  • Graphical Models

Hat

x1

x2

MAP

Inference

xn

Cat

Most Likely Assignment

(C) Dhruv Batra


Problems with map

Problems with MAP

Model-Class is Wrong!

-- Approximation Error

  • Human Body ≠ Tree

(C) Dhruv Batra

Figure Courtesy: [Yang & Ramanan ICCV ‘11]


Problems with map1

Problems with MAP

Model-Class is Wrong!

Not Enough Training Data!

-- Approximation Error

-- Estimation Error

(C) Dhruv Batra


Problems with map2

Problems with MAP

Model-Class is Wrong!

Not Enough Training Data!

MAP is NP-Hard

-- Approximation Error

-- Estimation Error

-- Optimization Error

(C) Dhruv Batra


Problems with map3

Problems with MAP

Model-Class is Wrong!

Not Enough Training Data!

MAP is NP-Hard

Inherent Ambiguity

-- Approximation Error

-- Estimation Error

-- Optimization Error

-- Bayes Error

?

?

Rotating clockwise / anti-clockwise?

Old Lady looking left / Young woman looking away?

One instance / Two instances?

(C) Dhruv Batra


Problems with map4

Problems with MAP

Single Prediction = Uncertainty Mismanagement

Model-Class is Wrong!

Not Enough Training Data!

MAP is NP-Hard

Inherent Ambiguity

-- Approximation Error

-- Estimation Error

-- Optimization Error

-- Bayes Error

Make Multiple Predictions!

(C) Dhruv Batra


Multiple predictions

Multiple Predictions

x

x

x

x

x

x

x

x

x

x

x

x

x

Sampling

Porway & Zhu, 2011

TU & Zhu, 2002

Rich History

(C) Dhruv Batra


Multiple predictions1

Ideally:

M-Best Modes

Multiple Predictions

M-Best MAP

Sampling

Porway & Zhu, 2011

TU & Zhu, 2002

Rich History

Flerova et al., 2011

Fromeret al., 2009

Yanover et al., 2003

(C) Dhruv Batra


Multiple predictions2

Ideally:

M-Best Modes

Multiple Predictions

M-Best MAP

Sampling

This Paper: Diverse M-Best in MRFs

Porway & Zhu, 2011

TU & Zhu, 2002

Rich History

  • Don’t hope for diversity. Explicitly encode it.

  • Not guaranteed to be modes.

Flerova et al., 2011

Fromeret al., 2009

Yanover et al., 2003

(C) Dhruv Batra


Map integer program

MAP Integer Program

kx1

(C) Dhruv Batra


Map integer program1

MAP Integer Program

1

0

0

0

kx1

(C) Dhruv Batra


Map integer program2

MAP Integer Program

0

1

0

0

kx1

(C) Dhruv Batra


Map integer program3

MAP Integer Program

0

0

1

0

kx1

(C) Dhruv Batra


Map integer program4

MAP Integer Program

0

0

0

1

kx1

(C) Dhruv Batra


Map integer program5

MAP Integer Program

0

0

0

1

kx1

k2x1

(C) Dhruv Batra


Map integer program6

MAP Integer Program

0

0

0

1

kx1

k2x1

(C) Dhruv Batra


Map integer program7

MAP Integer Program

Graphcuts, BP, Expansion, etc

(C) Dhruv Batra


Diverse 2 nd best

Diverse 2nd-Best

Diversity

MAP

(C) Dhruv Batra


Diverse m best

Diverse M-Best

(C) Dhruv Batra


Diverse 2 nd best1

Diverse 2nd-Best

Q1: How do we solve DivMBest?

Q2: What kind of diversity functions are allowed?

Q3: How much diversity?

See Paper for Details

(C) Dhruv Batra


Diverse 2 nd best2

Diverse 2nd-Best

  • Lagrangian Relaxation

Diversity-Augmented Energy

Many ways to solve:

upergradient Ascent. Optimal. Slow.

Primal

See Paper for Details

2. Binary Search. Optimal for M=2. Faster.

Dualize

3. Grid-search on lambda. Sub-optimal. Fastest.

Dual

Div2Best energy

Concave (Non-smooth)

Lower-Bound on Div2Best En.

(C) Dhruv Batra


Diverse 2 nd best3

Diverse 2nd-Best

Q1: How do we solve Div2Best?

Q2: What kind of diversity functions are allowed?

Q3: How much diversity?

See Paper for Details

(C) Dhruv Batra


Diversity

Diversity

  • [Special Case] 0-1 Diversity M-Best MAP

    • [Yanover NIPS03; Fromer NIPS09; Flerova Soft11]

  • [Special Case] Max Diversity [Park & RamananICCV11]

  • Hamming Diversity

  • Cardinality Diversity

  • Any Diversity

See Paper for Details

(C) Dhruv Batra


Hamming diversity

Hamming Diversity

0

1

0

0

0 1 0 0

0

1

0

0

1 0 0 0

(C) Dhruv Batra


Hamming diversity1

Hamming Diversity

  • Diversity Augmented Inference:

(C) Dhruv Batra


Hamming diversity2

Hamming Diversity

  • Diversity Augmented Inference:

Unchanged. Can still use graph-cuts!

Simply edit node-terms. Reuse MAP machinery!

(C) Dhruv Batra


Experiments

Experiments

  • 3 Applications

    • Interactive Segmentation: Hamming, Cardinality (in paper)

    • Pose Estimation: Hamming

    • Semantic Segmentation: Hamming

  • Baselines:

    • M-Best MAP (No Diversity)

    • Confidence-Based Perturbation (No Optimization)

  • Metrics

    • Oracle Accuracies

      • User-in-the-loop; Upper-Bound

    • Re-ranked Accuracies

(C) Dhruv Batra


Experiment 1

Experiment #1

  • Interactive Segmentation

    • Model: Color/Texture + Potts Grid CRF

    • Inference: Graph-cuts

    • Dataset: 50 train/val/test images

Image + Scribbles

MAP

2nd Best MAP

Diverse 2nd Best

1-2 Nodes Flipped

100-500 Nodes Flipped

(C) Dhruv Batra


Experiment 11

Experiment #1

+3.62%

+1.61%

+0.05%

(Oracle)

(Oracle)

(Oracle)

M=6

(C) Dhruv Batra


Experiment 2

Experiment #2

  • Pose Tracking

    • Model: Mixture of Parts from [Park & Ramanan, ICCV ‘11]

    • Inference: Dynamic Programming

    • Dataset: 4 videos, 585 frames

(C) Dhruv Batra

Image Credit: [Yang & Ramanan, ICCV ‘11]


Experiment 21

Experiment #2

  • Pose Tracking w/ Chain CRF

M BestSolutions

(C) Dhruv Batra

Image Credit: [Yang & Ramanan, ICCV ‘11]


Experiment 22

Experiment #2

MAP

DivMBest + Viterbi

(C) Dhruv Batra


Experiment 23

Experiment #2

Better

DivMBest (Re-ranked)

13% Gain

Same FeaturesSame Model

[Park & Ramanan, ICCV ‘11] (Re-ranked)

PCP Accuracy

Confidence-based Perturbation (Re-ranked)

#Solutions / Frame

(C) Dhruv Batra


Experiment 3

Experiment #3

  • Semantic Segmentation

    • Model: Hierarchical CRF [Ladicky et al. ECCV ’10, ICCV ‘09]

    • Inference: Alpha-expansion

    • Dataset: Pascal Segmentation Challenge (VOC 2010)

      • 20 categories + background; 964 train/val/test images

(C) Dhruv Batra

  • Image Credit: [Ladicky et al. ECCV ’10, ICCV ’09]


Experiment 31

Experiment #3

Input

MAP

Best of 10-Div

(C) Dhruv Batra


Experiment 32

Experiment #3

DivMBest (Oracle)

Better

MAP

22%-gain possible

Same FeaturesSame Model

PACAL Accuracy

DivMBest (Re-ranked) [Yadollahpouret al.]

Confidence-based Perturbation (Oracle)

#Solutions / Image

(C) Dhruv Batra


Summary

Summary

  • All models are wrong

  • Some beliefs are useful

  • DivMBest

    • First principled formulation for Diverse M-Best in MRFs

    • Efficient algorithm. Re-uses MAP machinery.

    • Big impact possible on many applications!

(C) Dhruv Batra


Thank you

Thank you!

  • Think about YOUR problem.

  • Are you or a loved one, tired of a single solution?

  • If yes, then DivMBest might be right for you!*

    * DivMBest is not suited for everyone. People with perfect models, and love of continuous variables should not use DivMBest. Consult your local optimization expert before startingDivMBest. Please do not drive or operate heavy machinery while on DivMBest.

(C) Dhruv Batra


  • Login