cs b553 a lgorithms for optimization and learning
Download
Skip this Video
Download Presentation
CS b553 : A lgorithms for Optimization and Learning

Loading in 2 Seconds...

play fullscreen
1 / 37

CS b553 : A lgorithms for Optimization and Learning - PowerPoint PPT Presentation


  • 82 Views
  • Uploaded on

CS b553 : A lgorithms for Optimization and Learning. Variable Elimination. Last Time. Variable elimination on polytrees Top down inference Linear in size of network Variable elimination in general No guarantees… NP hard in worst case… but when?. Variable Elimination in General Networks.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' CS b553 : A lgorithms for Optimization and Learning' - ping


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
last time
Last Time
  • Variable elimination on polytrees
    • Top down inference
    • Linear in size of network
  • Variable elimination in general
    • No guarantees…
    • NP hard in worst case… but when?
variable elimination in general networks
Variable Elimination in General Networks

Coherence

Difficulty

Intelligence

Grade

SAT

Letter

Job

Happy

variable elimination in general networks1
Variable Elimination in General Networks

Coherence

Difficulty

Intelligence

Grade

SAT

Letter

Job

Happy

joint distribution
Joint distribution
  • P(X) = P(C)P(D|C)P(I)P(G|I,D)P(S|I)P(L|G) P(J|L,S)P(H|G,J)
  • Apply elimination ordering C,D,I,H,G,S,L
going through ve
Going through VE
  • P(X) = P(C)P(D|C)P(I)P(G|I,D)P(S|I)P(L|G) P(J|L,S)P(H|G,J)
  • Apply elimination ordering C,D,I,H,G,S,L
  • 1(D)=SCP(C)P(D|C)
going through ve1
Going through VE
  • SCP(X) = 1(D)P(I)P(G|I,D)P(S|I)P(L|G) P(J|L,S)P(H|G,J)
  • Apply elimination ordering C,D,I,H,G,S,L
  • 1(D)=SCP(C)P(D|C)
going through ve2
Going through VE
  • SCP(X) = 1(D)P(I)P(G|I,D)P(S|I)P(L|G) P(J|L,S)P(H|G,J)
  • Apply elimination ordering C,D,I,H,G,S,L
  • 2(G,I)=SD1(D)P(G|I,D)
going through ve3
Going through VE
  • SC,DP(X) = 2(G,I)P(I)P(S|I)P(L|G) P(J|L,S)P(H|G,J)
  • Apply elimination ordering C,D,I,H,G,S,L
  • 2(G,I)=SD1(D)P(G|I,D)
going through ve4
Going through VE
  • SC,DP(X) = 2(G,I)P(I)P(S|I)P(L|G) P(J|L,S)P(H|G,J)
  • Apply elimination ordering C,D,I,H,G,S,L
  • 3(G,S)=SI2(G,I)P(I)P(S|I)
going through ve5
Going through VE
  • SC,D,IP(X) = 3(G,S)P(L|G)P(J|L,S)P(H|G,J)
  • Apply elimination ordering C,D,I,H,G,S,L
  • 3(G,S)=SI2(G,I)P(I)P(S|I)
going through ve6
Going through VE
  • SC,D,IP(X) = 3(G,S)P(L|G)P(J|L,S)P(H|G,J)
  • Apply elimination ordering C,D,I,H,G,S,L
  • 4(G,J)=SHP(H|G,J)

What values does this factor store?

going through ve7
Going through VE
  • SC,D,I,HP(X) = 3(G,S)P(L|G)P(J|L,S)4(G,J)
  • Apply elimination ordering C,D,I,H,G,S,L
  • 4(G,J)=SHP(H|G,J)
going through ve8
Going through VE
  • SC,D,I,HP(X) = 3(G,S)P(L|G)P(J|L,S)4(G,J)
  • Apply elimination ordering C,D,I,H,G,S,L
  • 5(S,L,J)=SG3(G,S)P(L|G)4(G,J)
going through ve9
Going through VE
  • SC,D,I,H,GP(X) = 5(S,L,J)P(J|L,S)
  • Apply elimination ordering C,D,I,H,G,S,L
  • 5(S,L,J)=SG3(G,S)P(L|G)4(G,J)
going through ve10
Going through VE
  • SC,D,I,H,GP(X) = 5(S,L,J)P(J|L,S)
  • Apply elimination ordering C,D,I,H,G,S,L
  • 6(L,J)=SS 5(S,L,J)P(J|L,S)
going through ve11
Going through VE
  • SC,D,I,H,G,SP(X) = 6(L,J)
  • Apply elimination ordering C,D,I,H,G,S,L
  • 6(L,J)=SS 5(S,L,J)
going through ve12
Going through VE
  • SC,D,I,H,G,SP(X) = 6(L,J)
  • Apply elimination ordering C,D,I,H,G,S,L
  • 7(J)=SL 6(S,L)
going through ve13
Going through VE
  • SC,D,I,H,G,S,LP(X) = 7(J)
  • Apply elimination ordering C,D,I,H,G,S,L
  • 7(J)=SL 6(L,J)
comparing orderings
Comparing Orderings
  • Consider G,I,S,L,H,C,D
understanding ve from bns to undirected graphs
Understanding VE: From BNs to Undirected Graphs
  • Consider each factor as a variable i
  • Draw an edge between any variables appearing in the same factor
building the undirected graph
Building the Undirected Graph

P(C)

Coherence

P(I)

P(D|C)

Difficulty

Intelligence

P(S|I)

P(G|I,D)

Grade

SAT

P(L|G)

Letter

P(J|S,L)

Job

Happy

P(H|G,J)

building the undirected graph1
Building the Undirected Graph

P(C)

Coherence

P(I)

P(D|C)

Difficulty

Intelligence

P(S|I)

P(G|I,D)

Grade

SAT

P(L|G)

Letter

P(J|S,L)

Job

Happy

P(H|G,J)

building the undirected graph2
Building the Undirected Graph

Coherence

Difficulty

Intelligence

Grade

SAT

Letter

Job

Happy

variable elimination
Variable Elimination

Coherence

Difficulty

Intelligence

Grade

SAT

Letter

Job

Happy

variable elimination1
Variable Elimination

Difficulty

Intelligence

Grade

SAT

Letter

Job

Happy

variable elimination2
Variable Elimination

Difficulty

Intelligence

Grade

SAT

Letter

Job

Happy

variable elimination3
Variable Elimination

Intelligence

Grade

SAT

Letter

Job

Happy

variable elimination4
Variable Elimination

Intelligence

Grade

SAT

Letter

Job

Happy

variable elimination5
Variable Elimination

New fill edge

Grade

SAT

Letter

Job

Happy

variable elimination6
Variable Elimination

Grade

SAT

Letter

Job

Happy

variable elimination7
Variable Elimination

Grade

SAT

Letter

Job

variable elimination8
Variable Elimination

Grade

SAT

Letter

Job

induced graph from a ve ordering
Induced Graph from a VE ordering

Coherence

Difficulty

Intelligence

Grade

SAT

Letter

Job

Happy

induced graph from a ve ordering1
Induced Graph from a VE ordering

Coherence

Difficulty

Intelligence

Grade

SAT

  • Theorem:
  • The scope of every intermediate factor in VE is a clique in the induced graph
  • Every maximal clique in the induced graph is the scope of an intermediate factor

Letter

Job

Happy

determining optimal orderings
Determining Optimal orderings
  • Again, NP hard!
  • Good heuristics in practice:
    • Min-neighbors, min-fill, etc
    • Search among elimination orderings while counting size of introduced factors
    • Greedy search often works well
ad