1 / 52

Graphical Models - Inference -

MINVOLSET. KINKEDTUBE. PULMEMBOLUS. INTUBATION. VENTMACH. DISCONNECT. PAP. SHUNT. VENTLUNG. VENITUBE. PRESS. MINOVL. FIO2. VENTALV. PVSAT. ANAPHYLAXIS. ARTCO2. EXPCO2. SAO2. TPR. INSUFFANESTH. HYPOVOLEMIA. LVFAILURE. CATECHOL. LVEDVOLUME. STROEVOLUME. ERRCAUTER. HR.

usoa
Download Presentation

Graphical Models - Inference -

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. MINVOLSET KINKEDTUBE PULMEMBOLUS INTUBATION VENTMACH DISCONNECT PAP SHUNT VENTLUNG VENITUBE PRESS MINOVL FIO2 VENTALV PVSAT ANAPHYLAXIS ARTCO2 EXPCO2 SAO2 TPR INSUFFANESTH HYPOVOLEMIA LVFAILURE CATECHOL LVEDVOLUME STROEVOLUME ERRCAUTER HR ERRBLOWOUTPUT HISTORY CO CVP PCWP HREKG HRSAT HRBP BP Mainly based on F. V. Jensen, „Bayesian Networks and Decision Graphs“, Springer-Verlag New York, 2001. Advanced IWS 06/07 Graphical Models- Inference - Junction Tree Wolfram Burgard, Luc De Raedt, Kristian Kersting, Bernhard Nebel Albert-Ludwigs University Freiburg, Germany

  2. Outline • Introduction • Reminder: Probability theory • Basics of Bayesian Networks • Modeling Bayesian networks • Inference (VE, Junction tree) • Excourse: Markov Networks • Learning Bayesian networks • Relational Models

  3. Why Junction trees ? • Multiple inference, e.g., • For each i do variable elimination? - Inference (Junction Tree) No, instead reuse information resp. computations

  4. Potentials A potentialfA over a set of variables A is a function that maps each configuration into a non-negative real number. domfA=A (domain of fA ) - Inference (Junction Tree) Examples: Conditional probability distribution and joint probability distributions are special cases of potentials

  5. Potentials - Inference (Junction Tree) Ex: A potential A,B,C over the set of variables {A,B,C}. Ahas four states, andBand C has three states. domfA,B,C ={A,B,C}

  6. VE using Potentials Let F be a set of potentials (e.g. CPDs), and let X be a variable. X is eliminated from F: • Remove all potentials in F with X in the domain. Let FX that set. • Calculate • Add to F • Iterate - Inference (Junction Tree) Potential where X is not a member of the domain

  7. Domain Graph A1 A1 Moralizing A3 A3 A2 A2 A4 A4 A5 A5 A6 A6 Bayesian Network Domain Graph (Moral Graph) - Inference (Junction Tree) moral link Let F={f1,...,fn} be potentials over U={A1,...,Am} with domfi=Di. The domain graph for F is the undirected graph with variables of U as nodes and with a link between pairs of variables being members of the same Di.

  8. Moralizing A1 A1 Moralizing A3 A3 A2 A2 A4 A4 A5 A5 A6 A6 Bayesian Network Domain Graph (Moral Graph) - Inference (Junction Tree)

  9. Moralizing A1 A1 Moralizing A3 A3 A2 A2 A4 A4 A5 A5 A6 A6 Bayesian Network Domain Graph (Moral Graph) - Inference (Junction Tree) CPDs

  10. Moralizing A1 A1 Moralizing A3 A3 A2 A2 A4 A4 A5 A5 A6 A6 Bayesian Network Domain Graph (Moral Graph) - Inference (Junction Tree) CPDs CPD = potential

  11. Moralizing A1 A1 Moralizing A3 A3 A2 A2 A4 A4 A5 A5 A6 A6 Bayesian Network Domain Graph (Moral Graph) - Inference (Junction Tree) Potentials induce undirected edges CPDs CPD = potential

  12. Moralizing A1 A1 Moralizing A3 A3 A2 A2 A4 A4 A5 A5 A6 A6 moral link Bayesian Network Domain Graph (Moral Graph) - Inference (Junction Tree) Potentials induce undirected edges CPDs CPD = potential

  13. Eliminating A3 A1 A2 A4 A5 A6 Elimination Sequence A1 A1 Moralizing A3 A3 A2 A2 A4 A4 A5 A5 A6 A6 Bayesian Network Moral Graph - Inference (Junction Tree)

  14. Elimination Sequence A1 A1 Moralizing A3 A3 A2 A2 A4 A4 A5 A5 A6 A6 Eliminating A3 Bayesian Network Moral Graph - Inference (Junction Tree) A1 A2 A4 A5 A6

  15. Reminder: Markov Networks • Undirected Graphs • Nodes = random variables • Cliques = potentials (~ local jpd) VE on potentials works for Markov networks - Inference (Junction Tree)

  16. Elimination Sequence A1 A1 Moralizing A3 A3 A2 A2 A4 A4 A5 A5 A6 A6 Eliminating A3 Bayesian Network Moral Graph - Inference (Junction Tree) fill-ins: work with a potential over a domain that was not present originally A1 A2 A4 A5 A6 GOAL:Elimination sequence that does not introduce fill-ins

  17. Perfect Elimination Sequence ... A1 A6 A3 A2 A4 A5 A6 - Inference (Junction Tree)

  18. Perfect Elimination Sequence ... A1 A1 A6 A5 A3 A2 A3 A2 A4 A4 A5 A5 A6 - Inference (Junction Tree)

  19. Perfect Elimination Sequence ... A1 A1 A1 A6 A5 A3 A2 A3 A3 A2 A2 A4 A4 A4 A5 A5 A6 - Inference (Junction Tree) A3

  20. Perfect Elimination Sequence ... A1 A1 A1 A6 A5 A3 A2 A3 A3 A2 A2 A4 A4 A4 A5 A5 A6 - Inference (Junction Tree) A1 A3 A1 A2 A4

  21. Perfect Elimination Sequence ... A1 A1 A1 A6 A5 A3 A2 A3 A3 A2 A2 A4 A4 A4 A5 A5 A6 - Inference (Junction Tree) A1 A3 A1 A2 A2 A2 A4 A4

  22. Perfect Elimination Sequence ... A1 A1 A1 A6 A5 A3 A2 A3 A3 A2 A2 A4 A4 A4 A5 A5 A6 - Inference (Junction Tree) A1 A3 A1 A2 A2 A2 A4 A4 A4

  23. Perfect Elimination Sequence ... ... do not introduce fill-ins A1 A1 A1 A6 A5 A3 A2 A3 A3 A2 A2 A4 A4 A4 A5 A5 A6 - Inference (Junction Tree) A1 A3 A1 A2 A2 There are several perfect elimination sequences ending in A4 A2 A4 A4 A4 But they have different complexities

  24. Complexity of Elimination Sequence • Characterized by the set of domains • Set of domains of potentials produced during the elimination (potentials that are subsets of other potentials are removed). • A6,A5,A3,A1,A2,A4: {{A6,A3},{A2,A3,A5},{A1,A2,A3},{A1,A2},{A2,A4}} - Inference (Junction Tree)

  25. Complexity of Elimination Sequence A1 A1 A1 A6 A5 A3 A2 A3 A3 A2 A2 A4 A4 A4 A5 A5 A6 {A1,A2,A3} {A2,A3,A5} {A3,A6} - Inference (Junction Tree) A1 A3 A1 A2 A2 A2 A4 A4 A4 {A2,A4} {A1,A2}

  26. Complexity of Elimination Sequence • All perfect elimination sequences produce the same domain set, namely the set of cliques of the domain • Any perfect elimination sequence ending with A is optimal with respect to computing P(A) - Inference (Junction Tree)

  27. Triangulated Graphs and Join Trees • An undirected graph with complete elimination sequence is called a triangulated graph A1 A2 A1 A2 - Inference (Junction Tree) A3 A3 A4 A5 A4 A5 nontriangulated triangulated

  28. Triangulated Graphs and Join Trees A1 A1 A2 A2 • Complete neighbour set = simplicial node • X is simplicial iff family of X is a clique Neighbours of A5 Family of A5 A3 A3 A4 A4 A5 A5 - Inference (Junction Tree) A1 A2 Family of A3, i.e. A3 is simplicial A3 A4 A5

  29. Triangulated Graphs and Join Trees Let G be a triangulated graph, and let X be a simplicial node. Let G´be the graph resulting from eliminating X from G. Then G´is a triangulated graph A1 A2 X A3 A4 A5 - Inference (Junction Tree) Eliminating X A1 A2 A3 A4 A5

  30. Triangulated Graphs and Join Trees • A triangulated graph with at least two nodes has at least two simplicial nodes • In a triangulated graph, each variable A has a perfect elimination sequence ending with A • Not all domain graphs are triangulated: An undirected graph is triangulated iff all nodes can be eliminated by successively eliminating a simplicial node - Inference (Junction Tree)

  31. Let G be the set of cliques from an undirected graph, and let the cliques of G be organized in a tree. T is a join tree if for any pair of nodes V, W all nodes on the path between V an W contain the intersection VW Join Trees BCDE ABCD DEFI BCDG CHGJ - Inference (Junction Tree) BCDE ABCD DEFI BCDG CHGJ

  32. Let G be the set of cliques from an undirected graph, and let the cliques of G be organized in a tree. T is a join tree if for any pair of nodes V, W all nodes on the path between V an W contain the intersection VW Join Trees BCDE ABCD DEFI join tree BCDG CHGJ - Inference (Junction Tree) BCDE ABCD DEFI BCDG not a join tree CHGJ

  33. Join Trees • If the cliques of an undirected graph G can be organized into a join tree, then G is triangulated • If the undirected graph is triangulated, then the cliques of G can be organizes into a join tree - Inference (Junction Tree)

  34. Triangulated, undirected Graph -> Join trees A B E • Simplicial node X • Family of X is a clique • Eliminate nodes from family of X which have only neighbours in the family of X • Give family of X is number according to the number of nodes eliminated • Denote the set of remaining nodes Si D C F H G I J - Inference (Junction Tree)

  35. Triangulated, undirected Graph -> Join trees X • Simplicial node X • Family of X is a clique • Eliminate nodes from family of X which have only neighbours in the family of X • Give family of X a number i according to the number of nodes eliminated • Denote the set of remaining nodes Si A B E D C F H G I J - Inference (Junction Tree) ABCD V1 BCD S1 separator

  36. Triangulated, undirected Graph -> Join trees A X • Simplicial node X • Family of X is a clique • Eliminate nodes from family of X which have only neighbours in the family of X • Give family of X a number i according to the number of nodes eliminated • Denote the set of remaining nodes Si A B E {A,B,C,D} D C F H G I J {B,C,D} - Inference (Junction Tree) ABCD V1 BCD S1 separator 1 S1

  37. Triangulated, undirected Graph -> Join trees • Simplicial node X • Family of X is a clique • Eliminate nodes from family of X which have only neighbours in the family of X • Give family of X is number according to the number of nodes eliminated • Denote the set of remaining nodes Si B E X D C F H G I X J - Inference (Junction Tree) ABCD V1 BCD S1 DEFI V3 DE S3

  38. Triangulated, undirected Graph -> Join trees E • Simplicial node X • Family of X is a clique • Eliminate nodes from family of X which have only neighbours in the family of X • Give family of X is number according to the number of nodes eliminated • Denote the set of remaining nodes Si B E {D,E,F,I} X D C F H G I X J {D,E} - Inference (Junction Tree) ABCD V1 BCD S1 3 DEFI V3 DE S3 S3

  39. Triangulated, undirected Graph -> Join trees • Simplicial node X • Family of X is a clique • Eliminate nodes from family of X which have only neighbours in the family of X • Give family of X is number according to the number of nodes eliminated • Denote the set of remaining nodes Si CGHJ V5 CG S5 BCDG V6 BCD S6 BCDE V10 - Inference (Junction Tree) ABCD V1 BCD S1 DEFI V3 DE S3

  40. Triangulated, undirected Graph -> Join trees • Connect each separator Si to a clique Vj, j>i, such that Si is a subset of Vj CGHJ V5 CG S5 BCDG V6 BCD S6 BCDE V10 - Inference (Junction Tree) ABCD V1 BCD S1 DEFI V3 DE S3

  41. Join Tree The potential representation of a join tree (aka clique tree) is the product of the clique potentials, dived by the product of the separator potentials. CGHJ V5 CG S5 BCDG V6 BCD S6 BCDE V10 - Inference (Junction Tree) ABCD V1 BCD S1 DEFI V3 DE S3

  42. Yet Another Example A1 A1 A2 A3 A3 A2 A4 A4 A5 A6 A5 A6 A1,A2,A3 V4 - Inference (Junction Tree) A2,A3 S2 A2 S4 A3 S1 A2,A3,A5 V2 A2,A4 V6 A3,A6 V1

  43. A1 Junction Trees A2 A3 A4 A6 A5 • Let F be a set of potentials with a triangulated domain graph. A junction tree for F is join tree for G with • Each potential f in F is associated to a clique containing dom(f) • Each link has separator attached containing two mailboxes, one for each direction - Inference (Junction Tree)

  44. Propagation on a Junction Tree • Node V can send exactly one message to a neighbour W, and it may only be sent when V has received a message from all of its other neighbours • Choose one clique (arbitrarily) as a root of the tree; collect message to this node and then distribute messages away from it. • After collection and distribution phases, we have all we need in each clique to compute potential for variables. - Inference (Junction Tree)

  45. Junction Trees - Collect Evidence • P(A4) ? • Find clique containing A4 • V6 temporary root • Send messages from leaves to root • V4 assembles the incoming messages, potential - Inference (Junction Tree)

  46. Junction Tree - Messages • Propagation/message passing between two adjacent cliques C1, C2 (S0 is their seperator) • Marginalize C1’s potential to get new potential for S0 • Update C2’s potential • Update S0’s potential to its new potential

  47. Junction Trees - Collect Evidence • P(A4) ? • Find clique containing A4 • V6 temporary root • Send messages from leaves to root • V4 assembles the incoming messages, potential - Inference (Junction Tree)

  48. Junction Trees - Collect Evidence • P(A4) ? • All marginals? - Inference (Junction Tree)

  49. Junction Trees - Distribute Evidence • All marginals? - Inference (Junction Tree)

  50. Nontriangulated Domain Graphs • Embed domain graph in a traingulated graph • Use ist junction tree • Simple idea: • Eliminate variables in some order • If you wish to eliminate a node with non-complete neighbour set, make it complete by adding fill-ins - Inference (Junction Tree) A B C A B C D E D E A,C,H,I,J,B,G,D,E,F F G F G H I J H I J

More Related