Learning the structure of Deep sparse Graphical Model

1 / 14

# Learning the structure of Deep sparse Graphical Model - PowerPoint PPT Presentation

Learning the structure of Deep sparse Graphical Model. Ryan Prescott Adams Hanna M Wallach Zoubin Ghahramani. Presented by Zhengming Xing. Some pictures are directly copied from the paper and Hanna Wallachâ€™s slides. outline. Introduction Finite belief network Infinite belief network

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## Learning the structure of Deep sparse Graphical Model

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
1. Learning the structure of Deep sparse Graphical Model Ryan Prescott Adams Hanna M Wallach Zoubin Ghahramani Presented by Zhengming Xing Some pictures are directly copied from the paper and Hanna Wallach’s slides

2. outline • Introduction • Finite belief network • Infinite belief network • Inference • Experiment

3. Introduction Main contribution: combine deep belief network and nonparametric bayesian together. Main idea: use IBP to learn the structure of the network Structure of the network include: Depth Width Connectivity

4. Single layer network Use Binary matrix to represent the network. Black refer to 1(two unit were connected) White refer to 0 (two unit were not connected) Z IBP can be used as the prior for infinite columns binary matrix

5. Review IBP 1.First customer tries dishes. 2. Nth customer tries Tasked dishes K with probability new dishes

6. Multi-layer network

7. Cascading IBP Also parameterize by Each dishes in the restaurant is also a customer in another Indian buffet process Each matrix is exchangeable both rows and columns This chain can reach the state with probability one ( number of unit in layer m) Properties: For unit in layer m+1 Expected number of parents: Expected number of children:

8. Sample from the CIBP prior

9. model m refer to the layers and increase upto M. weights bias Place layer wise Gaussian prior on weights and bias, Gamma prior on noise precision

10. Inference Weights, bias, noise variance can be sampled with Gibbs sampler.

11. Inference( sample Z) Two step: 1. 2. Sample existing dishes MH-sample Add a new unit and, and insert connection to this unit with MH ratio For a exist unit remove the connection to this unit with MH ratio

12. Experiment result Olivetti faces Remove bottom halves of the test image.

13. Experiment result MNIST Digits

14. Experiment result Frey Faces