Markov Networks

1. Markov Networks Model Definition Comparison to Bayes Nets Inference techniques Learning Techniques

2. We can have potentials on any cliques—not just the maximal ones. So, for example we can have a potential on A in addition to the other four pairwise potentials A Factor says a=b=0 B D Qn: What is the most likely configuration of A&B? C But, marginal says a=0;b=1! Okay, you convinced me that given any potentials we will have a consistent Joint. But given any joint, will there be a potentials I can provide? Hammersley-Clifford theorem… Although A,B would Like to agree, B&C Need to agree, C&D need to disagree And D&A need to agree .and the latter three have Higher weights! Mr. & Mrs. Smith example  Moral: Factors are notmarginals!

3. Markov Networks Smoking Cancer • Undirected graphical models Asthma Cough • Potential functions defined over cliques

7. Log-Linear models for Markov Nets A B D Without loss of generality! C Factors are “functions” over their domains Log linear model consists of  Features fi(Di ) (functions over domains) Weights wi for features s.t.

8. Markov Networks Smoking Cancer • Undirected graphical models Asthma Cough • Log-linear model: Weight of Feature i Feature i

9. Markov Nets vs. Bayes Nets

11. Connection to MCMC: MCMC requires sampling a node given its markov blanket Need to use P(x|MB(x)). For Bayes nets MB(x) contains more nodes than are mentioned in the local distribution CPT(x)  For Markov nets,

13. Inference in Markov Networks • Goal: Compute marginals & conditionals of • Exact inference is #P-complete • Most BN inference approaches work for MNs too • Variable Elimination used factor multiplication—and should work without change.. • Conditioning on Markov blanket is easy: • Gibbs sampling exploits this

14. Gibbs sampling for Markov networks • Example: P(D | ¬c) • Resample non-evidence variables in a pre-defined order or a random order • Suppose we begin with A • B and C are Markov blanket of A • Calculate P(A | B,C) • Use current Gibbs sampling value for B & C • Note: never change (evidence).

15. Example: Gibbs sampling • Resample probability distribution of A Normalized result =

16. Example: Gibbs sampling • Resample probability distribution of B Normalized result =

17. MCMC: Gibbs Sampling state← random truth assignment fori← 1 tonum-samples do for each variable x sample x according to P(x|neighbors(x)) state←state with new value of x P(F) ← fraction of states in which F is true

18. Learning Markov Networks • Learning parameters (weights) • Generatively • Discriminatively • Learning structure (features) • Easy Case: Assume complete data(If not: EM versions of algorithms)

19. Entanglement in log likelihood… a b c

20. Learning for log-linear formulation What is the expected Value of the feature given the current parameterization of the network? Requires inference to answer (inference at every iteration— sort of like EM ) Use gradient ascent Unimodal, because Hessian is Co-variance matrix over features

21. Why should we spend so much time computing gradient? • Given that gradient is being used only in doing the gradient ascent iteration, it might look as if we should just be able to approximate it in any which way • Afterall, we are going to take a step with some arbitrary step size anyway.. • ..But the thing to keep in mind is that the gradient is a vector. We are talking not just of magnitude but direction. A mistake in magnitude can change the direction of the vector and push the search into a completely wrong direction…

22. No. of times feature i is true in data Expected no. times feature i is true according to model Generative Weight Learning • Maximize likelihood or posterior probability • Numerical optimization (gradient or 2nd order) • No local maxima • Requires inference at each step (slow!)

23. Alternative Objectives to maximize.. Given a single data instance x log-likelihood is • Since log-likelihood requires network inference to compute the derivative, we might want to focus on other objectives whose gradients are easier to compute (and which also –hopefully—have optima at the same parameter values). • Two options: • Pseudo Likelihood • Contrastive Divergence Log prob of data Log prob of allotherpossible data instances (w.r.t. current q) Maximize the distance (“increase the divergence”) Compute likelihood of each possible data instance just using markov blanket (approximate chain rule) Pick a sample of typical other instances (need to sample from Pq Run MCMC initializing with the data..)

24. Pseudo-Likelihood • Likelihood of each variable given its neighbors in the data • Does not require inference at each step • Consistent estimator • Widely used in vision, spatial statistics, etc. • But PL parameters may not work well forlong inference chains [Which can lead to disasterous results]

25. Discriminative Weight Learning • Maximize conditional likelihood of query (y) given evidence (x) • Approximate expected counts by counts in MAP state of y given x No. of true groundings of clause i in data Expected no. true groundings according to model

29. Markov Logic: Intuition • A logical KB is a set of hard constraintson the set of possible worlds • Let’s make them soft constraints:When a world violates a formula,It becomes less probable, not impossible • Give each formula a weight(Higher weight  Stronger constraint)

30. Markov Logic: Definition • A Markov Logic Network (MLN) is a set of pairs (F, w) where • F is a formula in first-order logic • w is a real number • Together with a set of constants,it defines a Markov network with • One node for each grounding of each predicate in the MLN • One feature for each grounding of each formula F in the MLN, with the corresponding weight w

31. Example: Friends & Smokers

32. Example: Friends & Smokers

33. Example: Friends & Smokers

34. Example: Friends & Smokers Two constants: Anna (A) and Bob (B)

35. Example: Friends & Smokers Two constants: Anna (A) and Bob (B) Smokes(A) Smokes(B) Cancer(A) Cancer(B)

36. Example: Friends & Smokers Two constants: Anna (A) and Bob (B) Friends(A,B) Friends(A,A) Smokes(A) Smokes(B) Friends(B,B) Cancer(A) Cancer(B) Friends(B,A)

37. Example: Friends & Smokers Two constants: Anna (A) and Bob (B) Friends(A,B) Friends(A,A) Smokes(A) Smokes(B) Friends(B,B) Cancer(A) Cancer(B) Friends(B,A)

38. Example: Friends & Smokers Two constants: Anna (A) and Bob (B) Friends(A,B) Friends(A,A) Smokes(A) Smokes(B) Friends(B,B) Cancer(A) Cancer(B) Friends(B,A)

39. Markov Logic Networks • MLN is template for ground Markov nets • Probability of a world x: • Typed variables and constants greatly reduce size of ground Markov net • Functions, existential quantifiers, etc. • Infinite and continuous domains Weight of formula i No. of true groundings of formula i in x

40. Special cases: Markov networks Markov random fields Bayesian networks Log-linear models Exponential models Max. entropy models Gibbs distributions Boltzmann machines Logistic regression Hidden Markov models Conditional random fields Obtained by making all predicates zero-arity Markov logic allows objects to be interdependent (non-i.i.d.) Relation to Statistical Models

41. Relation to First-Order Logic • Infinite weights  First-order logic • Satisfiable KB, positive weights Satisfying assignments = Modes of distribution • Markov logic allows contradictions between formulas

42. MAP/MPE Inference • Problem: Find most likely state of world given evidence Query Evidence

43. MAP/MPE Inference • Problem: Find most likely state of world given evidence

44. MAP/MPE Inference • Problem: Find most likely state of world given evidence

45. MAP/MPE Inference • Problem: Find most likely state of world given evidence • This is just the weighted MaxSAT problem • Use weighted SAT solver(e.g., MaxWalkSAT [Kautz et al., 1997]) • Potentially faster than logical inference (!)

46. The MaxWalkSAT Algorithm fori← 1 to max-triesdo solution = random truth assignment for j ← 1 tomax-flipsdo if ∑ weights(sat. clauses) > thresholdthen return solution c ← random unsatisfied clause with probabilityp flip a random variable in c else flip variable in c that maximizes ∑ weights(sat. clauses) return failure, best solution found

47. But … Memory Explosion • Problem:If there are n constantsand the highest clause arity is c,the ground network requires O(n ) memory • Solution:Exploit sparseness; ground clauses lazily→ LazySAT algorithm [Singla & Domingos, 2006] c

48. Computing Probabilities • P(Formula|MLN,C) = ? • MCMC: Sample worlds, check formula holds • P(Formula1|Formula2,MLN,C) = ? • If Formula2 = Conjunction of ground atoms • First construct min subset of network necessary to answer query (generalization of KBMC) • Then apply MCMC (or other) • Can also do lifted inference [Braz et al, 2005]

49. Ground Network Construction network←Ø queue← query nodes repeat node← front(queue) remove node from queue add node to network if node not in evidence then add neighbors(node) to queue until queue = Ø

50. But … Insufficient for Logic • Problem:Deterministic dependencies break MCMCNear-deterministic ones make it veryslow • Solution:Combine MCMC and WalkSAT→MC-SAT algorithm [Poon & Domingos, 2006]