Lecture 15. Artificial Neural Networks Presentation (3 of 4): Pattern Recognition using Unsupervised ANNs. Monday, February 21, 2000 Prasanna Jayaraman Department of Computing and Information Sciences, KSU http://www.cis.ksu.edu/~prasanna Readings:
Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.
Lecture 15
Artificial Neural Networks Presentation (3 of 4):
Pattern Recognition using Unsupervised ANNs
Monday, February 21, 2000
Prasanna Jayaraman
Department of Computing and Information Sciences, KSU
http://www.cis.ksu.edu/~prasanna
Readings:
“The WakeSleep Algorithm For Unsupervised Neural Networks”
 Hinton, Dayan, Frey and Neal
Presentation Outline
The Core Idea
Economical representation and accurate reconstruction of input.
To minimize the “description length”.
Driving the neurons of ANN by the appropriate connection in the corresponding phase achieves the desired goal.
Sleep & Wake Phases
To the Receiver
j22
j12
Hidden Layer
α
j11
j21
j31
Input Vector
d2
Explanatory Figures
Fundamentals of Wake  Sleep Algorithm
d1
d1
Output Unit
Only One Hidden Layer
Basics of Other Training Algorithms
Input Unit
Sample Figures
Wake  Sleep Algorithm
1
=
=
Pr
ob
(
s
1
)
å
u
+

1
exp(b
s
w
u
u
u
u
u
Wake  Sleep Algorithm
Helmholtz Machine
Q(.  d ) over α.
Factorial Distribution
Kullback  Leibler Divergence
Sample Figures
Summary Points
Summary Points