1 / 88

Hypercubes and Neural Networks

Hypercubes and Neural Networks. bill wolfe 10/23/2005. Modeling. Simple Neural Model. a i Activation e i External input w ij Connection Strength Assume: w ij = w ji (“symmetric” network)  W = (w ij ) is a symmetric matrix . Net Input. Vector Format:. Dynamics. Basic idea:.

Hypercubes and Neural Networks

E N D

Presentation Transcript

1. Hypercubes and Neural Networks bill wolfe 10/23/2005

2. Modeling

3. Simple Neural Model • aiActivation • ei External input • wij Connection Strength Assume: wij = wji (“symmetric” network) W = (wij) is a symmetric matrix

4. Net Input Vector Format:

5. Dynamics • Basic idea:

6. Energy

7. Lower Energy • da/dt = net = -grad(E)  seeks lower energy

8. Problem: Divergence

9. A Fix: Saturation

10. Keeps the activation vector inside the hypercube boundaries Encourages convergence to corners

11. Summary: The Neural Model aiActivation eiExternal Input wijConnection Strength W (wij = wji) Symmetric

12. Example: Inhibitory Networks • Completely inhibitory • wij = -1 for all i,j • k-winner • Inhibitory Grid • neighborhood inhibition

13. Traveling Salesman Problem • Classic combinatorial optimization problem • Find the shortest “tour” through n cities • n!/2n distinct tours

14. TSP solution for 15,000 cities in Germany

15. TSP50 City Example

16. Random

17. Nearest-City

18. 2-OPT

19. An Effective Heuristic for the Traveling Salesman Problem S. Lin and B. W. Kernighan Operations Research, 1973 http://www.jstor.org/view/0030364x/ap010105/01a00060/0

20. Centroid

21. Monotonic

22. Neural Network Approach neuron

23. Tours – Permutation Matrices tour: CDBA permutation matrices correspond to the “feasible” states.

24. Not Allowed

25. Only one city per time stopOnly one time stop per cityInhibitory rows and columns inhibitory

26. Distance Connections: Inhibit the neighboring cities in proportion to their distances.

27. putting it all together:

28. Research Questions • Which architecture is best? • Does the network produce: • feasible solutions? • high quality solutions? • optimal solutions? • How do the initial activations affect network performance? • Is the network similar to “nearest city” or any other traditional heuristic? • How does the particular city configuration affect network performance? • Is there a better way to understand the nonlinear dynamics?

29. typical state of the network before convergence