diverse m best solutions in markov random fields n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Diverse M-Best Solutions in Markov Random Fields PowerPoint Presentation
Download Presentation
Diverse M-Best Solutions in Markov Random Fields

Loading in 2 Seconds...

play fullscreen
1 / 45

Diverse M-Best Solutions in Markov Random Fields - PowerPoint PPT Presentation


  • 125 Views
  • Uploaded on

Diverse M-Best Solutions in Markov Random Fields. Dhruv Batra Virginia Tech Joint work with: Students: Payman Yadollahpour (TTIC ), Abner Guzman-Rivera (UIUC) Colleagues: Chris Dyer (CMU), Greg Shakhnarovich (TTIC), Pushmeet Kohli (MSRC), Kevin Gimpel (TTIC).

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Diverse M-Best Solutions in Markov Random Fields' - alvis


Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
diverse m best solutions in markov random fields

Diverse M-Best Solutions inMarkov Random Fields

Dhruv Batra

Virginia Tech

Joint work with:Students: PaymanYadollahpour (TTIC), Abner Guzman-Rivera (UIUC)

Colleagues: Chris Dyer (CMU), Greg Shakhnarovich (TTIC), PushmeetKohli (MSRC), Kevin Gimpel (TTIC)

ambiguity ambiguity ambiguity
Ambiguity Ambiguity Ambiguity

One instance / Two instances?

?

?

(C) Dhruv Batra

problems with map
Problems with MAP

Single Prediction = Uncertainty Mismanagement

Model-Class is Wrong!

Not Enough Training Data!

MAP is NP-Hard

Inherent Ambiguity

-- Approximation Error

-- Estimation Error

-- Optimization Error

-- Bayes Error

Make Multiple Predictions!

(C) Dhruv Batra

multiple predictions
Multiple Predictions

x

x

x

x

x

x

x

x

x

x

x

x

x

Sampling

Porway & Zhu, 2011

TU & Zhu, 2002

Rich History

(C) Dhruv Batra

multiple predictions1

Ideally:

M-Best Modes

Multiple Predictions

M-Best MAP

Sampling

Porway & Zhu, 2011

TU & Zhu, 2002

Rich History

Flerova et al., 2011

Fromeret al., 2009

Yanover et al., 2003

(C) Dhruv Batra

multiple predictions2

Ideally:

M-Best Modes

Multiple Predictions

M-Best MAP

Sampling

Our work: Diverse M-Best in MRFs [ECCV ‘12]

Porway & Zhu, 2011

TU & Zhu, 2002

Rich History

  • Don’t hope for diversity. Explicitly encode it.
  • Not guaranteed to be modes.

Flerova et al., 2011

Fromeret al., 2009

Yanover et al., 2003

(C) Dhruv Batra

example result
Example Result

(C) Dhruv Batra

example result1
Example Result

Discriminative Re-ranking of Diverse Segmentation

[Yadollahpour et al., CVPR13, Wednesday Poster]

(C) Dhruv Batra

map integer program
MAP Integer Program

kx1

(C) Dhruv Batra

map integer program1
MAP Integer Program

1

0

0

0

kx1

(C) Dhruv Batra

map integer program2
MAP Integer Program

0

1

0

0

kx1

(C) Dhruv Batra

map integer program3
MAP Integer Program

0

0

1

0

kx1

(C) Dhruv Batra

map integer program4
MAP Integer Program

0

0

0

1

kx1

(C) Dhruv Batra

map integer program5
MAP Integer Program

0

0

0

1

kx1

k2x1

(C) Dhruv Batra

map integer program6
MAP Integer Program

0

0

0

1

kx1

k2x1

(C) Dhruv Batra

map integer program7
MAP Integer Program

Graphcuts, BP, Expansion, etc

(C) Dhruv Batra

diverse 2 nd best
Diverse 2nd-Best

Diversity

MAP

(C) Dhruv Batra

diverse m best
Diverse M-Best

(C) Dhruv Batra

diverse 2 nd best1
Diverse 2nd-Best

Q1: How do we solve DivMBest?

Q2: What kind of diversity functions are allowed?

Q3: How much diversity?

(C) Dhruv Batra

diverse 2 nd best2
Diverse 2nd-Best

Diversity-Augmented Score

Primal

Dualize

(C) Dhruv Batra

diverse 2 nd best3
Diverse 2nd-Best
  • Lagrangian Relaxation

Diversity-Augmented Score

Dual

Subgradient Descent

Concave (Non-smooth)

Upper-Bound on Div2Best Score

Div2Best score

(C) Dhruv Batra

diverse 2 nd best4
Diverse 2nd-Best
  • Lagrangian Relaxation

Diversity-Augmented Energy

Many ways to solve:

Subgradient Ascent. Optimal. Slow.

2. Binary Search. Optimal for M=2. Faster.

Dualize

3. Grid-search on lambda. Sub-optimal. Fastest.

(C) Dhruv Batra

theorem statement
Theorem Statement
  • Theorem [Batra et al ’12]: Lagrangian Dual corresponds to solving the Relaxed Primal:
      • Based on result from [Geoffrion ‘74]

Dual

Relaxed Primal

(C) Dhruv Batra

effect of lagrangian relaxation2
Effect of Lagrangian Relaxation
  • [Mezumanet al. UAI13]

Pairwise Potential Strength

Pairwise Potential Strength

(C) Dhruv Batra

diverse 2 nd best5
Diverse 2nd-Best

Q1: How do we solve DivMBest?

Q2: What kind of diversity functions are allowed?

Q3: How much diversity?

(C) Dhruv Batra

diversity
Diversity
  • [Special Case] 0-1 Diversity M-Best MAP
    • [Yanover NIPS03; Fromer NIPS09; Flerova Soft11]
  • [Special Case] Max Diversity [Park & RamananICCV11]
  • Hamming Diversity
  • Cardinality Diversity
  • Any Diversity

(C) Dhruv Batra

hamming diversity
Hamming Diversity

0

1

0

0

0 1 0 0

0

1

0

0

1 0 0 0

(C) Dhruv Batra

hamming diversity1
Hamming Diversity
  • Diversity Augmented Inference:

(C) Dhruv Batra

hamming diversity2
Hamming Diversity
  • Diversity Augmented Inference:

Unchanged. Can still use graph-cuts!

Simply edit node-terms. Reuse MAP machinery!

(C) Dhruv Batra

diverse 2 nd best6
Diverse 2nd-Best

Q1: How do we solve DivMBest?

Q2: What kind of diversity functions are allowed?

Q3: How much diversity?

(C) Dhruv Batra

how much diversity
How Much Diversity?
  • Empirical Solution: Cross-Val for
  • More Efficient: Cross-Val for

(C) Dhruv Batra

experiments
Experiments
  • 3 Applications
    • Interactive Segmentation: Hamming, Cardinality (in paper)
    • Pose Estimation: Hamming
    • Semantic Segmentation: Hamming
  • Baselines:
    • M-Best MAP (No Diversity)
    • Confidence-Based Perturbation (No Optimization)

(C) Dhruv Batra

interactive segmentation
Interactive Segmentation
  • Setup
    • Model: Color/Texture + Potts Grid CRF
    • Inference: Graph-cuts
    • Dataset: 50 train/val/test images

Image + Scribbles

MAP

2nd Best MAP

Diverse 2nd Best

1-2 Nodes Flipped

100-500 Nodes Flipped

(C) Dhruv Batra

pose tracking
Pose Tracking
  • Setup
    • Model: Mixture of Parts from [Park & Ramanan, ICCV ‘11]
    • Inference: Dynamic Programming
    • Dataset: 4 videos, 585 frames

(C) Dhruv Batra

Image Credit: [Yang & Ramanan, ICCV ‘11]

pose tracking1
Pose Tracking
  • Chain CRF with M states at each time

M BestSolutions

(C) Dhruv Batra

Image Credit: [Yang & Ramanan, ICCV ‘11]

pose tracking2
Pose Tracking

MAP

DivMBest + Viterbi

(C) Dhruv Batra

pose tracking3
Pose Tracking

Better

DivMBest (Re-ranked)

13% Gain

Same FeaturesSame Model

[Park & Ramanan, ICCV ‘11] (Re-ranked)

PCP Accuracy

Confidence-based Perturbation (Re-ranked)

#Solutions / Frame

(C) Dhruv Batra

machine translation
Machine Translation

Input:

Die Regierung will die Folter von “Hexen” unterbinden und gab eineBroschüreheraus

MAP Translation:

The government wants the torture of ‘witch’ and gave out a booklet

(C) Dhruv Batra

machine translation1
Machine Translation

Input:

Die Regierung will die Folter von “Hexen” unterbinden und gab eineBroschüreheraus

5-Best Translations:

The government wants the torture of ‘witch’ and gave out a booklet

The government wants the torture of “witch” and gave out a booklet

The government wants the torture of ‘witch’ and gave out a brochure

The government wants the torture of ‘witch’ and gave out a leaflet

The government wants the torture of “witch” and gave out a brochure

(C) Dhruv Batra

machine translation2
Machine Translation

Input:

Die Regierung will die Folter von “Hexen” unterbinden und gab eineBroschüreheraus

Diverse 5-Best Translations:

The government wants the torture of ‘witch’ and gave out a booklet

The government wants to stop torture of “witch” and issued a leaflet issued

The government wants to “stop the torture of” witches and gave out a brochure

The government intends to the torture of “witchcraft” and were issued a leaflet

The government is the torture of “witches” stamp out and gave a brochure

(C) Dhruv Batra

machine translation3
Machine Translation

Input:

Die Regierung will die Folter von “Hexen” unterbinden und gab eineBroschüreheraus

Diverse 5-Best Translations:

The government wants the torture of ‘witch’ and gave out a booklet

The government wants to stop torture of “witch” and issued a leaflet issued

The government wants to “stop the torture of” witches and gave out a brochure

The government intends to the torture of “witchcraft” and were issued a leaflet

The government is the torture of “witches” stamp out and gave a brochure

Correct Translation:

The government wants to limit the torture of “witches,” a brochure was released

(C) Dhruv Batra