 Download Presentation Probabilistic Inference Lecture 3

# Probabilistic Inference Lecture 3 - PowerPoint PPT Presentation

Probabilistic Inference Lecture 3. M. Pawan Kumar pawan.kumar@ecp.fr. Slides available online http:// cvc.centrale-ponts.fr /personnel/ pawan /. Recap of lecture 1. Exponential Family. P( v ) = exp {- Σ α θ α Φ α ( v ) - A( θ )}. Sufficient Statistics. Log-Partition Function. I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation ## Probabilistic Inference Lecture 3

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
1. Probabilistic InferenceLecture 3 M. Pawan Kumar pawan.kumar@ecp.fr Slides available online http://cvc.centrale-ponts.fr/personnel/pawan/

2. Recap of lecture 1

3. Exponential Family P(v) = exp{-ΣαθαΦα(v) - A(θ)} Sufficient Statistics Log-Partition Function Parameters Random Variables V = {V1,V2,…,Vn} Random Variable Va takes a value or label va va L = {l1,l2,…,lh} Labeling V = v

4. Overcomplete Representation P(v) = exp{-ΣαθαΦα(v) - A(θ)} Sufficient Statistics Log-Partition Function Parameters There exists a non-zero c such that Σα cαΦα(v) = Constant

5. Pairwise MRF P(v) = exp{-ΣαθαΦα(v) - A(θ)} Random Variable V = {V1, V2, …,Vn} Label set L = {l1, l2, …, lh} Neighborhood over variables specified by edges E Sufficient Statistics Parameters for all Va V, li  L Ia;i(va) θa;i θab;ik for all (Va,Vb)  E, li, lk L Iab;ik(va,vb)

6. Pairwise MRF P(v) = exp{-ΣaΣiθa;iIa;i(va) -Σa,bΣi,kθab;ikIab;ik(va,vb) - A(θ)} Random Variable V = {V1, V2, …,Vn} Label set L = {l1, l2, …, lh} Neighborhood over variables specified by edges E Πaψa(va) Π(a,b)ψab(va,vb) Probability P(v) = Z A(θ) : log Z ψa(li) : exp(-θa;i) ψa(li,lk) : exp(-θab;ik) Parameters θ are sometimes also referred to as potentials

7. Pairwise MRF P(v) = exp{-ΣaΣiθa;iIa;i(va) -Σa,bΣi,kθab;ikIab;ik(va,vb) - A(θ)} Random Variable V = {V1, V2, …,Vn} Label set L = {l1, l2, …, lh} Neighborhood over variables specified by edges E Labeling as a function f : {1, 2, … , n} {1, 2, …, h} Variable Va takes a label lf(a)

8. Pairwise MRF P(f) = exp{-Σaθa;f(a) -Σa,bθab;f(a)f(b)- A(θ)} Random Variable V = {V1, V2, …,Vn} Label set L = {l1, l2, …, lh} Neighborhood over variables specified by edges E Labeling as a function f : {1, 2, … , n} {1, 2, …, h} Variable Va takes a label lf(a) Energy Q(f) = Σaθa;f(a) + Σa,bθab;f(a)f(b)

9. Pairwise MRF P(f) = exp{-Q(f)- A(θ)} Random Variable V = {V1, V2, …,Vn} Label set L = {l1, l2, …, lh} Neighborhood over variables specified by edges E Labeling as a function f : {1, 2, … , n} {1, 2, …, h} Variable Va takes a label lf(a) Energy Q(f) = Σaθa;f(a) + Σa,bθab;f(a)f(b)

10. Inference maxv ( P(v) = exp{-ΣaΣiθa;iIa;i(va) -Σa,bΣi,kθab;ikIab;ik(va,vb) - A(θ)} ) Maximum a Posteriori (MAP) Estimation minf ( Q(f) = Σaθa;f(a) + Σa,bθab;f(a)f(b) ) Energy Minimization P(va = li) = ΣvP(v)δ(va = li) P(va = li, vb = lk) = ΣvP(v)δ(va = li)δ(vb= lk) Computing Marginals

11. Recap of lecture 2

12. Definitions Energy Minimization f* = arg min Q(f; ) Q(f; ) = ∑a a;f(a) + ∑(a,b) ab;f(a)f(b) Min-marginals s.t. f(a) = i qa;i= min Q(f; ) Reparameterization ’   Q(f; ’) = Q(f; ), for all f

13. + Mab;k ’b;k= b;k + Mba;i ’a;i = a;i ’ab;ik = ab;ik - Mab;k - Mba;i Belief Propagation Pearl, 1988 General form of Reparameterization Reparameterization of (a,b) in Belief Propagation Mab;k = mini { a;i + ab;ik } Mba;i = 0

14. Belief Propagation on Trees Va Vb Vc Vg Vh Vd Ve Forward Pass: Leaf  Root Backward Pass: Root  Leaf All min-marginals are computed

15. Computational Complexity • Each constant takes O(|L|) • Number of constants - O(|E||L|) O(|E||L|2) • Memory required ? O(|E||L|)

16. Belief Propagation on Cycles b;1 a;1 b;0 a;0 Va Vb c;1 d;1 c;0 d;0 Vd Vc Remember my suggestion? Fix the label of Va

17. Belief Propagation on Cycles b;1 b;0 a;0 Va Vb c;1 d;1 c;0 d;0 Vd Vc Equivalent to a tree-structured problem

18. Belief Propagation on Cycles b;1 a;1 b;0 Va Vb c;1 d;1 c;0 d;0 Vd Vc Equivalent to a tree-structured problem

19. Belief Propagation on Cycles b;1 a;1 b;0 a;0 Va Vb c;1 d;1 c;0 d;0 Vd Vc This approach quickly becomes infeasible Choose the minimum energy solution

20. Vincent Algayres Algorithm b;0 a;0 Va Vb c;1 d;1 c;0 d;0 Vd Vc Compute zero cost paths from all labels of Va to all labels of Vd. Requires fixing Va.

21. Speed-Ups for Special Cases ab;ik= 0, if i = k = C, otherwise. Mab;k = mini { a;i + ab;ik} Felzenszwalb and Huttenlocher, 2004

22. Speed-Ups for Special Cases ab;ik= wab|i-k| Mab;k = mini { a;i + ab;ik} Felzenszwalb and Huttenlocher, 2004

23. Speed-Ups for Special Cases ab;ik= min{wab|i-k|, C} Mab;k = mini { a;i + ab;ik} Felzenszwalb and Huttenlocher, 2004

24. Speed-Ups for Special Cases ab;ik= min{wab(i-k)2, C} Mab;k = mini { a;i + ab;ik} Felzenszwalb and Huttenlocher, 2004

25. Lecture 3

26. Ising Model P(v) = exp{-ΣαθαΦα(v) - A(θ)} Random Variable V = {V1, V2, …,Vn} Label set L = {0, 1} Neighborhood over variables specified by edges E Sufficient Statistics Parameters for all Va V, li  L Ia;i(va) θa;i θab;ik for all (Va,Vb)  E, li, lk L Iab;ik(va,vb) Ia;i(va): indicator for va = li Iab;ik(va,vb): indicator for va = li, vb = lk

27. Ising Model P(v) = exp{-ΣaΣiθa;iIa;i(va) -Σa,bΣi,kθab;ikIab;ik(va,vb) - A(θ)} Random Variable V = {V1, V2, …,Vn} Label set L = {0, 1} Neighborhood over variables specified by edges E Sufficient Statistics Parameters for all Va V, li  L Ia;i(va) θa;i θab;ik for all (Va,Vb)  E, li, lk L Iab;ik(va,vb) Ia;i(va): indicator for va = li Iab;ik(va,vb): indicator for va = li, vb = lk

28. Interactive Binary Segmentation Foreground histogram of RGB values FG Background histogram of RGB values BG ‘1’ indicates foreground and ‘0’ indicates background

29. Interactive Binary Segmentation More likely to be foreground than background

30. Interactive Binary Segmentation θa;0 proportional to -log(BG(da)) θa;1 proportional to -log(FG(da)) More likely to be background than foreground

31. Interactive Binary Segmentation More likely to belong to same label

32. Interactive Binary Segmentation θab;ik proportional to exp(-(da-db)2) if i ≠ k θab;ik = 0 if i = k Less likely to belong to same label

33. Outline • Minimum Cut Problem • Two-Label Submodular Energy Functions • Move-Making Algorithms

34. Directed Graph D = (N, A) 10 n1 n2 Two important restrictions 3 2 (1) Rational arc lengths (2) Positive arc lengths n3 n4 5

35. Cut D = (N, A) • Let N1 and N2 such that • N1 “union” N2 = N • N1 “intersection” N2 = Φ 10 n1 n2 3 2 • C is a set of arcs such that • (n1,n2)  A • n1  N1 • n2  N2 n3 n4 5 C is a cut in the digraph D

36. Cut D = (N, A) N1 What is C? 10 n1 n2 {(n1,n2),(n1,n4)} ? 3 2 {(n1,n4),(n3,n2)} ? ✓ n3 n4 {(n1,n4)} ? 5 N2

37. Cut D = (N, A) N1 N2 What is C? 10 n1 n2 {(n1,n2),(n1,n4),(n3,n2)} ? 3 2 ✓ {(n4,n3)} ? n3 n4 {(n1,n4),(n3,n2)} ? 5

38. Cut D = (N, A) N2 N1 What is C? 10 n1 n2 ✓ {(n1,n2),(n1,n4),(n3,n2)} ? 3 2 {(n3,n2)} ? n3 n4 {(n1,n4),(n3,n2)} ? 5

39. Cut D = (N, A) • Let N1 and N2 such that • N1 “union” N2 = N • N1 “intersection” N2 = Φ 10 n1 n2 3 2 • C is a set of arcs such that • (n1,n2)  A • n1  N1 • n2  N2 n3 n4 5 C is a cut in the digraph D

40. Weight of a Cut D = (N, A) 10 n1 n2 3 Sum of length of all arcs in C 2 n3 n4 5

41. Weight of a Cut D = (N, A) 10 n1 n2 3 w(C) = Σ(n1,n2) C l(n1,n2) 2 n3 n4 5

42. Weight of a Cut D = (N, A) N1 What is w(C)? 10 n1 n2 3 3 2 n3 n4 5 N2

43. Weight of a Cut D = (N, A) N1 N2 What is w(C)? 10 n1 n2 5 3 2 n3 n4 5

44. Weight of a Cut D = (N, A) N2 N1 What is w(C)? 10 n1 n2 15 3 2 n3 n4 5

45. st-Cut s D = (N, A) 1 2 A source “s” 10 n1 n2 A sink “t” 3 2 • C is a cut such that • s  N1 • t  N2 n3 n4 5 7 t 3 C is an st-cut

46. Weight of an st-Cut s D = (N, A) 1 2 10 n1 n2 3 w(C) = Σ(n1,n2) C l(n1,n2) 2 n3 n4 5 7 t 3

47. Weight of an st-Cut s D = (N, A) 1 2 What is w(C)? 10 n1 n2 3 3 2 n3 n4 5 7 t 3

48. Weight of an st-Cut s D = (N, A) 1 2 What is w(C)? 10 n1 n2 15 3 2 n3 n4 5 7 t 3

49. Minimum Cut Problem s D = (N, A) 1 2 Find a cut with the minimum weight !! 10 n1 n2 3 2 C* = argminC w(C) n3 n4 5 7 t 3

50. Solvers for the Minimum-Cut Problem Augmenting Path and Push-Relabel n: #nodes m: #arcs U: maximum arc length [Slide credit: Andrew Goldberg]