1 / 37

# CS b553 : A lgorithms for Optimization and Learning - PowerPoint PPT Presentation

CS b553 : A lgorithms for Optimization and Learning. Variable Elimination. Last Time. Variable elimination on polytrees Top down inference Linear in size of network Variable elimination in general No guarantees… NP hard in worst case… but when?. Variable Elimination in General Networks.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about ' CS b553 : A lgorithms for Optimization and Learning' - ping

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

### CS b553: Algorithms for Optimization and Learning

Variable Elimination

• Variable elimination on polytrees

• Top down inference

• Linear in size of network

• Variable elimination in general

• No guarantees…

• NP hard in worst case… but when?

Coherence

Difficulty

Intelligence

SAT

Letter

Job

Happy

Coherence

Difficulty

Intelligence

SAT

Letter

Job

Happy

• P(X) = P(C)P(D|C)P(I)P(G|I,D)P(S|I)P(L|G) P(J|L,S)P(H|G,J)

• Apply elimination ordering C,D,I,H,G,S,L

• P(X) = P(C)P(D|C)P(I)P(G|I,D)P(S|I)P(L|G) P(J|L,S)P(H|G,J)

• Apply elimination ordering C,D,I,H,G,S,L

• 1(D)=SCP(C)P(D|C)

• SCP(X) = 1(D)P(I)P(G|I,D)P(S|I)P(L|G) P(J|L,S)P(H|G,J)

• Apply elimination ordering C,D,I,H,G,S,L

• 1(D)=SCP(C)P(D|C)

• SCP(X) = 1(D)P(I)P(G|I,D)P(S|I)P(L|G) P(J|L,S)P(H|G,J)

• Apply elimination ordering C,D,I,H,G,S,L

• 2(G,I)=SD1(D)P(G|I,D)

• SC,DP(X) = 2(G,I)P(I)P(S|I)P(L|G) P(J|L,S)P(H|G,J)

• Apply elimination ordering C,D,I,H,G,S,L

• 2(G,I)=SD1(D)P(G|I,D)

• SC,DP(X) = 2(G,I)P(I)P(S|I)P(L|G) P(J|L,S)P(H|G,J)

• Apply elimination ordering C,D,I,H,G,S,L

• 3(G,S)=SI2(G,I)P(I)P(S|I)

• SC,D,IP(X) = 3(G,S)P(L|G)P(J|L,S)P(H|G,J)

• Apply elimination ordering C,D,I,H,G,S,L

• 3(G,S)=SI2(G,I)P(I)P(S|I)

• SC,D,IP(X) = 3(G,S)P(L|G)P(J|L,S)P(H|G,J)

• Apply elimination ordering C,D,I,H,G,S,L

• 4(G,J)=SHP(H|G,J)

What values does this factor store?

• SC,D,I,HP(X) = 3(G,S)P(L|G)P(J|L,S)4(G,J)

• Apply elimination ordering C,D,I,H,G,S,L

• 4(G,J)=SHP(H|G,J)

• SC,D,I,HP(X) = 3(G,S)P(L|G)P(J|L,S)4(G,J)

• Apply elimination ordering C,D,I,H,G,S,L

• 5(S,L,J)=SG3(G,S)P(L|G)4(G,J)

• SC,D,I,H,GP(X) = 5(S,L,J)P(J|L,S)

• Apply elimination ordering C,D,I,H,G,S,L

• 5(S,L,J)=SG3(G,S)P(L|G)4(G,J)

• SC,D,I,H,GP(X) = 5(S,L,J)P(J|L,S)

• Apply elimination ordering C,D,I,H,G,S,L

• 6(L,J)=SS 5(S,L,J)P(J|L,S)

• SC,D,I,H,G,SP(X) = 6(L,J)

• Apply elimination ordering C,D,I,H,G,S,L

• 6(L,J)=SS 5(S,L,J)

• SC,D,I,H,G,SP(X) = 6(L,J)

• Apply elimination ordering C,D,I,H,G,S,L

• 7(J)=SL 6(S,L)

• SC,D,I,H,G,S,LP(X) = 7(J)

• Apply elimination ordering C,D,I,H,G,S,L

• 7(J)=SL 6(L,J)

• Consider G,I,S,L,H,C,D

• Consider each factor as a variable i

• Draw an edge between any variables appearing in the same factor

P(C)

Coherence

P(I)

P(D|C)

Difficulty

Intelligence

P(S|I)

P(G|I,D)

SAT

P(L|G)

Letter

P(J|S,L)

Job

Happy

P(H|G,J)

P(C)

Coherence

P(I)

P(D|C)

Difficulty

Intelligence

P(S|I)

P(G|I,D)

SAT

P(L|G)

Letter

P(J|S,L)

Job

Happy

P(H|G,J)

Coherence

Difficulty

Intelligence

SAT

Letter

Job

Happy

Coherence

Difficulty

Intelligence

SAT

Letter

Job

Happy

Difficulty

Intelligence

SAT

Letter

Job

Happy

Difficulty

Intelligence

SAT

Letter

Job

Happy

Intelligence

SAT

Letter

Job

Happy

Intelligence

SAT

Letter

Job

Happy

New fill edge

SAT

Letter

Job

Happy

SAT

Letter

Job

Happy

SAT

Letter

Job

SAT

Letter

Job

SAT

Letter

Job

Coherence

Difficulty

Intelligence

SAT

Letter

Job

Happy

Coherence

Difficulty

Intelligence

SAT

• Theorem:

• The scope of every intermediate factor in VE is a clique in the induced graph

• Every maximal clique in the induced graph is the scope of an intermediate factor

Letter

Job

Happy

• Again, NP hard!

• Good heuristics in practice:

• Min-neighbors, min-fill, etc

• Search among elimination orderings while counting size of introduced factors

• Greedy search often works well