Learning the structure of Deep sparse Graphical Model. Ryan Prescott Adams Hanna M Wallach Zoubin Ghahramani. Presented by Zhengming Xing. Some pictures are directly copied from the paper and Hanna Wallachâ€™s slides. outline. Introduction Finite belief network Infinite belief network
Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.
Learning the structure of Deep sparse Graphical Model Ryan Prescott Adams Hanna M Wallach Zoubin Ghahramani Presented by Zhengming Xing Some pictures are directly copied from the paper and Hanna Wallach’s slides
outline • Introduction • Finite belief network • Infinite belief network • Inference • Experiment
Introduction Main contribution: combine deep belief network and nonparametric bayesian together. Main idea: use IBP to learn the structure of the network Structure of the network include: Depth Width Connectivity
Single layer network Use Binary matrix to represent the network. Black refer to 1(two unit were connected) White refer to 0 (two unit were not connected) Z IBP can be used as the prior for infinite columns binary matrix
Review IBP 1.First customer tries dishes. 2. Nth customer tries Tasked dishes K with probability new dishes
Cascading IBP Also parameterize by Each dishes in the restaurant is also a customer in another Indian buffet process Each matrix is exchangeable both rows and columns This chain can reach the state with probability one ( number of unit in layer m) Properties: For unit in layer m+1 Expected number of parents: Expected number of children:
model m refer to the layers and increase upto M. weights bias Place layer wise Gaussian prior on weights and bias, Gamma prior on noise precision
Inference Weights, bias, noise variance can be sampled with Gibbs sampler.
Inference( sample Z) Two step: 1. 2. Sample existing dishes MH-sample Add a new unit and, and insert connection to this unit with MH ratio For a exist unit remove the connection to this unit with MH ratio
Experiment result Olivetti faces Remove bottom halves of the test image.
Experiment result MNIST Digits
Experiment result Frey Faces