- By
**ping** - Follow User

- 82 Views
- Uploaded on

Download Presentation
## PowerPoint Slideshow about ' CS b553 : A lgorithms for Optimization and Learning' - ping

**An Image/Link below is provided (as is) to download presentation**

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

### CS b553: Algorithms for Optimization and Learning

Variable Elimination

Last Time

- Variable elimination on polytrees
- Top down inference
- Linear in size of network
- Variable elimination in general
- No guarantees…
- NP hard in worst case… but when?

Joint distribution

- P(X) = P(C)P(D|C)P(I)P(G|I,D)P(S|I)P(L|G) P(J|L,S)P(H|G,J)
- Apply elimination ordering C,D,I,H,G,S,L

Going through VE

- P(X) = P(C)P(D|C)P(I)P(G|I,D)P(S|I)P(L|G) P(J|L,S)P(H|G,J)
- Apply elimination ordering C,D,I,H,G,S,L
- 1(D)=SCP(C)P(D|C)

Going through VE

- SCP(X) = 1(D)P(I)P(G|I,D)P(S|I)P(L|G) P(J|L,S)P(H|G,J)
- Apply elimination ordering C,D,I,H,G,S,L
- 1(D)=SCP(C)P(D|C)

Going through VE

- SCP(X) = 1(D)P(I)P(G|I,D)P(S|I)P(L|G) P(J|L,S)P(H|G,J)
- Apply elimination ordering C,D,I,H,G,S,L
- 2(G,I)=SD1(D)P(G|I,D)

Going through VE

- SC,DP(X) = 2(G,I)P(I)P(S|I)P(L|G) P(J|L,S)P(H|G,J)
- Apply elimination ordering C,D,I,H,G,S,L
- 2(G,I)=SD1(D)P(G|I,D)

Going through VE

- SC,DP(X) = 2(G,I)P(I)P(S|I)P(L|G) P(J|L,S)P(H|G,J)
- Apply elimination ordering C,D,I,H,G,S,L
- 3(G,S)=SI2(G,I)P(I)P(S|I)

Going through VE

- SC,D,IP(X) = 3(G,S)P(L|G)P(J|L,S)P(H|G,J)
- Apply elimination ordering C,D,I,H,G,S,L
- 3(G,S)=SI2(G,I)P(I)P(S|I)

Going through VE

- SC,D,IP(X) = 3(G,S)P(L|G)P(J|L,S)P(H|G,J)
- Apply elimination ordering C,D,I,H,G,S,L
- 4(G,J)=SHP(H|G,J)

What values does this factor store?

Going through VE

- SC,D,I,HP(X) = 3(G,S)P(L|G)P(J|L,S)4(G,J)
- Apply elimination ordering C,D,I,H,G,S,L
- 4(G,J)=SHP(H|G,J)

Going through VE

- SC,D,I,HP(X) = 3(G,S)P(L|G)P(J|L,S)4(G,J)
- Apply elimination ordering C,D,I,H,G,S,L
- 5(S,L,J)=SG3(G,S)P(L|G)4(G,J)

Going through VE

- SC,D,I,H,GP(X) = 5(S,L,J)P(J|L,S)
- Apply elimination ordering C,D,I,H,G,S,L
- 5(S,L,J)=SG3(G,S)P(L|G)4(G,J)

Going through VE

- SC,D,I,H,GP(X) = 5(S,L,J)P(J|L,S)
- Apply elimination ordering C,D,I,H,G,S,L
- 6(L,J)=SS 5(S,L,J)P(J|L,S)

Going through VE

- SC,D,I,H,G,SP(X) = 6(L,J)
- Apply elimination ordering C,D,I,H,G,S,L
- 6(L,J)=SS 5(S,L,J)

Going through VE

- SC,D,I,H,G,SP(X) = 6(L,J)
- Apply elimination ordering C,D,I,H,G,S,L
- 7(J)=SL 6(S,L)

Going through VE

- SC,D,I,H,G,S,LP(X) = 7(J)
- Apply elimination ordering C,D,I,H,G,S,L
- 7(J)=SL 6(L,J)

Comparing Orderings

- Consider G,I,S,L,H,C,D

Understanding VE: From BNs to Undirected Graphs

- Consider each factor as a variable i
- Draw an edge between any variables appearing in the same factor

Building the Undirected Graph

P(C)

Coherence

P(I)

P(D|C)

Difficulty

Intelligence

P(S|I)

P(G|I,D)

Grade

SAT

P(L|G)

Letter

P(J|S,L)

Job

Happy

P(H|G,J)

Building the Undirected Graph

P(C)

Coherence

P(I)

P(D|C)

Difficulty

Intelligence

P(S|I)

P(G|I,D)

Grade

SAT

P(L|G)

Letter

P(J|S,L)

Job

Happy

P(H|G,J)

Induced Graph from a VE ordering

Coherence

Difficulty

Intelligence

Grade

SAT

- Theorem:
- The scope of every intermediate factor in VE is a clique in the induced graph
- Every maximal clique in the induced graph is the scope of an intermediate factor

Letter

Job

Happy

Determining Optimal orderings

- Again, NP hard!
- Good heuristics in practice:
- Min-neighbors, min-fill, etc
- Search among elimination orderings while counting size of introduced factors
- Greedy search often works well

Download Presentation

Connecting to Server..