Chapter 9 Perceptrons and their generalizations

1. Chapter 9Perceptrons and their generalizations

2. Rosenblatt’s perceptron • Proofs of the theorem • Method of stochastic approximation and sigmoid approximation of indicator functions • Method of potential functions and Radial basis functions • Three theorem of optimization theory • Neural Networks

3. Perceptrons (Rosenblatt, 1950s)

4. Recurrent Procedure

21. Proofs of the theorems

22. Method of stochastic approximation and sigmoid approximation of indicator functions

24. Method of Stochastic Approximation

28. Sigmoid Approximation of Indicator Functions

29. Basic Frame for learning process • Use the sigmoid approximation at the stage of estimating the coefficients • Use the indicator functions at the stage of recognition.

30. Method of potential functions and Radial Basis Functions

31. Potential function • On-line • Only one element of the training data • RBFs (mid-1980s) • Off-line

32. Method of potential functions in asymptotic learning theory • Separable condition • Deterministic setting of the PR • Non-separable condition • Stochastic setting of the PR problem

33. Deterministic Setting

34. Stochastic Setting

35. RBF Method

37. Three Theorems of optimization theory • Fermat’s theorem (1629) • Entire space, without constraints • Lagrange multipliers rule (1788) • Conditional optimization problem • Kuhn-Tucker theorem (1951) • Convex optimizaiton

42. To find the stationary points of functions • It is necessary to solve a system of n equations with n unknown values.

43. Lagrange Multiplier Rules (1788)