Compiling Bayesian Networks

1. Compiling Bayesian Networks

2. Sophisticated Platform (desktop) evaluator n evaluator 1 evaluator 2 compiler Engine AC AC AC Primitive Platforms (embedded) Embedding Probabilistic Inference

3. + * * + + * * * * * * θa λa λb θab θa~b θ~ab λ~b θ~a~b λ~a θ~a

15. Local CPT Structure A B E S A B E Pr(S|A,B,E) a b e 0.95 a b e 0.95 a b e 0.20 a b e 0.05 a b e 0.00 a b e 0.00 a b e 0.00 a b e 0.00 Tabular CPT qs|abe -Functional constraints -Context-specific independence

16. JT vs AC size

17. AC Compile Time

18. AC vs JT Inference Time

19. How does it work?

20. Ø A B true true .03 true false .27 false true .56 false false false .14 false Pr(a) = .03 + .27 = .3

21. A B Ø true true .03 true false .27 false true .56 false false false .14 false Pr(~b) = .27 + .14 = .41

22. A B Ø λaλb .03 .03 true true .27 λaλ~b .27 true false λ~aλb .56 .56 false true false λ~aλ~b .14 .14 false false false F(λ~a,λ~b,λb,λa) = .03λaλb + .27λaλ~b + .56λ~aλb + .14λ~a λ~b

23. + * * * * .14 .56 .27 .03 λ~a λ~b λb λa F(λ~a,λ~b,λb,λa) = .03λaλb + .27λaλ~b + .56λ~aλb + .14λ~a λ~b

24. Effect of Evidence on Values of Evidence Indicators

25. .03 + * * * * .14 .56 .27 .03 λ~a λ~b λb λa 0 0 1 1 Evidence: a & b

26. .27 + * * * * .14 .56 .27 .03 λ~a λ~b λ b λa 0 1 0 1 Evidence: a & ~b

27. .30 + * * * * .14 .56 .27 .03 λ~a λ~b λ b λa 0 1 1 1 Evidence: a

28. 1 + * * * * .14 .56 .27 .03 λ~a λ~b λ b λa 1 1 1 1 Evidence: true

29. The Network Polynomial

30. B D A θc|a C θa θd|bc θb|a F = λa λb λc λd θa θb|a θc|a θd|bc + λa λb λc λ~d θa θb|a θc|a θ~d|bc + …. Each term has 2n variables (n indicators, n parameters) Each variable has degree one (multi-linear function)

31. C B A F = λa λb λc θa θb|a θc|a + λa λb λ~c θa θb|a θ~c|a + λa λ~b λc θa θ~b|a θc|a + λa λ~b λ~c θa θ~b|a θ~c|a ….

32. B D A θc|a C θa θd|bc θb|a F = λa λb λc λd θa θb|a θc|a θd|bc + λa λb λc λ~d θa θb|a θc|a θ~d|bc + …. F(b,~c,d) = F(λa=1, λ~a=1, λb=1, λ~b=0, λc=0, λ~c=1, λd=1,λ~d=0) = θa θb|a θ~c|a θd|b~c + θ~a θb|a θ~c|~a θd|b~c = Pr(b,~c,d)

33. Factoring the Network Polynomialinto an Arithmetic Circuit (AC)

34. + * * + + * * * * * * θa λa λb θ~a θab θa~b θ~ab λ~b θ~a~b λ~a Arithmetic Circuits F = λa λb θa θb|a+ λa λ~b θa θ~b|a+ λ~aλb θ~a θb|~a+ λ~a λ~b θ~a θ~b|~a

35. Evaluating and Differentiating Circuits

36. .3 + .3 0 * * + + 1 1 .3 .1 .9 .8 .2 0 * * * * * * .3 1 .1 1 .9 .8 1 .2 0 .7 θa λa λb θab θa~b θ~ab λ~b θ~a~b λ~a θ~a

37. 1 .3 .3 .03 .3 0 .27 0 .7 0 .3 + 1 1 1 * * .3 0 + + .3 0 1 1 1 .3 .3 1 0 0 * * * * * .3 .1 .9 .8 .2 0 * θa λa λb θab θa~b θ~ab λ~b θ~a~b λ~a θ~a .3 1 .1 1 .9 .8 1 .2 0 .7

38. The Partial Derivatives

39. ØB A B A ØA true true .1 true .3 true false .9 false .7 false true .8 false false .2 false B A false

40. 1 .3 .3 .03 .3 0 .27 0 .7 0 Pr(a) .3 + * * + + * * * * * * θa λa λb θab θa~b θ~ab λ~b θ~a~b λ~a θ~a .3 1 .1 1 .9 .8 1 .2 0 .7

41. When X does not appear in e: Pr(x,e) =F(e) / λx Pr(x | e) =F(e) / λx F(e)

42. If variable X is instantiated in e: Pr(e – X, x́) = F(e)/ λx́

43. For variable X that appears instantiated in e: Σx F(e) λx Pr(e - X) =

44. For every family X U in Network N: F(e) /  θx|u θx|u F(e ) Pr(x,u|e) =

45. Second Partial Derivatives F2(e)/ λx λy = Pr(x,y,e)

46. Generating Circuits

47. Bayesian network A Jointree ABC B C D E BCD H F G BH CE EF EG Representational factorization (17 parameters) Computational factorization (32 parameters) Factorizations

48. Bayesian network A Jointree ABC B C D E BCD H F G BH CE EF EG Representational factorization (17 parameters) Computational factorization (32 parameters) Factorizations Network topology Network parameters

49. A B E S A B E Pr(S|A,B,E) a b e 0.95 Pr(S=s|A=a,B=b)=.95 a b e 0.95 a b e 0.20 a b e 0.05 a b e 0.00 a b e 0.00 a b e 0.00 a b e 0.00 Tabular CPT Local Structure

50. Bayesian network A Jointree ABC B C D E BCD H F G BH CE EF EG Representational factorization Computational factorization Factorizations Network topology Network parameters