Applications of Cellular Neural/Nonlinear Networks in Physics

1. Babes-Bolyai University Péter Pázmány Catholic University Applications of Cellular Neural/Nonlinear Networks in Physics Mária-Magdolna Ercsey-Ravasz Scientific advisors: Dr. Prof. Zoltán Néda Dr. Prof. Tamás Roska

2. Outline • CNN computing • A realisticrandomnumbergenerator • Stochasticsimulationson CNN computers • The site-percolation problem • The two-dimensional Ising model • Optimizationofspin-glasses on a space-variant CNN • Pulse-coupled oscillators communicating with light pulses

3. The standard CNN model • Each cell has a circuit with: • Input voltage:u • Statevoltage:x • Outputvoltage:y Template: A= , B= , z L. O. Chua , L. Yang, IEEE Transactions on Circuits and Systems 35. No. 10, 1988 , .

4. The CNN Universal Machine ACE16K CNN chip: 128*128 cells Bi-i V2 • programmable • parallel processing • continuous in time • continuous (analog) in values • discrete in space • Universal (in Turing sense) on integers and on analog array signals T. Roska, L. O. Chua, IEEE Transactions on Circuits and Systems – II, 40, 1993

5. CNN computing • image processing • real-time algorithms • fast and smart camera computer • robot eyes, bionic eye-glass • cellular automata models • partial differential equations • Research goals: • applications in physics • how should CNN computers be further developed? – from physicist perspectives

6. Generating realistic random numbers • A good pseudo-random generator Yalcin, et alle., Int.J. Circ. Theor. Appl., 32, 591-607, 2004 • Chaotic cellular automaton perturbed with the natural noise of the chip P’(t)=P(t) xor N(t) N(t): - very few black pixels - strong correlations but real stochastic fluctuations P(t) P’1(t) XOR P’2(t)

7. a good random binary image in t=116 µs • 1 single random value: • ACE16K  7ns • Pentium 4 at 2.8 GHz (Linux) 33ns Increasing the size of the chip in the future will assure even much bigger advantage for CNN chips Trend for the simulation time as a function of the chip size M. Ercsey-Ravasz, T. Roska, Z. Neda, Int. J. of Modern Physics C, Vol. 17, No. 6, p. 909 (2006)

8. generating random images with different p density of the black pixels --- using more images with ½ density • --- if p is an n bit number we need n images p=0.25 p=0.375 p=0.03125 • Correlations: • in space (first neighbors): 0.05% - 0.4% • in time (consecutive steps): 0.7% - 0.8 % M. Ercsey-Ravasz, T. Roska, Z. Neda, Int. J. of Modern Physics C, Vol. 17, No. 6, p. 909 (2006)

9. Stochastic simulations on CNN computers The site percolation problem • Used for modeling: • conductivity or mechanical properties of composite materials • magnetization of dilute magnets at low temperatures • fluid passing through porous materials • propagation of diseases • Probability of percolation - density of black pixels • 2nd order geometrical phase-transition • With CNN: 1 single template detecting percolation • input: the random image • initial state: the first row • output: the parts connected to the first row ACE16k chip

10. Percolation probability - for each p: 10000 different initial conditions - results agree with the accepted critical value: • pc=0.407 Trend for the simulation time in function of the chip size • Time needed: • CNN:t ~ L • digital computers:t ~ L • For L=128 CNN is slower • if L grows still promises advantage 2 M. Ercsey-Ravasz, T. Roska, Z. Neda, Int. J. of Modern Physics C, Vol. 17, No. 6, p. 909 (2006)

11. The two-dimensional Ising model Energy of the system: Metropolis algorithm - randomly choose a spin and flip it with p probability On CNN: Aparallel Metropolis algorithm is used • Because parallel computing we have to avoid flipping 2 neighbors simultaneouslychessboard mask • Odd (even)step : spins marked withblack (white)are updated • Equivalent with a Metropolis algorithm in which spins are chosen in a well defined order

12. Algorithm scheme for 1 MC step: • Build 3 masks marking: -generate 2 random images • Spins with 4 similar neighbors (E=8J):M1----AND------P1with exp(-8J/kT) • Spins with 3 similar neighbors (E=4J):M2----AND------P2with exp(-4J/kT) • Other spins (E0):M3 • Build the composed mask M=(M1 AND P1) OR (M2 AND P2) ORM3 • Use the (inverse) chessboard mask:M’= M AND C in (even) odd steps • Flip the spins marked on M’ T=2 T=2.3 T=2.6 ( J/k=1) Movies obtained with the ACE16K chip M. Ercsey-Ravasz, T. Roska, Z. Neda, Eur. Phys. J. B, Vol. 51, No. 3, p. 407, (2006)

13. Specific heat Results for the Ising model Magnetization Susceptibility • Initial state: homogeneous • Boundary conditions: fixed • 5000 transition MC steps • Averaging over 10000 MC steps M. Ercsey-Ravasz, T. Roska, Z. Neda, Eur. Phys. J. B, Vol. 51, No. 3, p. 407, (2006)

14. Time needed for 1 MC step (128*128 lattice): •  4.3 mson the ACE16K chip •  2.2 mson a Pentium 4 at 2.4 GHz M. Ercsey-Ravasz, T. Roska, Z. Neda, Eur. Phys. J. B, Vol. 51, No. 3, p. 407, (2006)

15. Optimization of spin-glasses on a space-variant CNN only simulated • final state after an operation: yij = 1 • Lyapunov function (energy) of the CNN: CNN Spin-glass • monotone decreasing • final state: local minimum (dE/dt=0) y[-1,1] y=±1 same local minima 1 operation ↔ 1 local minimum

16. The stochastic optimization algorithm • Same principles as in simulated annealing: • noise: random input U • b ↔ strength of noise • b slowly decreases • we choose: • b0=5 • Δb=0.05 1 cooling process

17. NP-hard problem hard for p<0.6 • Speed estimation: • the A templates must be introduced only once for each problem • we can use the characteristic parameters of the ACE16k chip 1000- 5000 steps / second Independent of size ! Many applications:error-correcting codes, econophysics, computer science etc. M. Ercsey-Ravasz, T. Roska, Z. Neda, Physica D: Nonlinear Phenomena, Special issue: “Novel computing paradigms: Quo vadis?, accepted, (2008), http://dx.doi.org/10.1016/j.physd.2008.03.028

18. Pulse-coupled oscillators communicating with light pulses Motivations:- studying a CNN with pulse-coupled oscillators - communicating with light global coupling - perspectives: separately programmable oscillators - first part of the study: collective behavior of identical units • The oscillators: • “electronic fireflies” • simple integrate-and-fire type neurons • Photoresistor (R,U) + LED • light R U • G: threshold • if U>G LED fires • not before Tmin • not after Tmax

19. Collective behavior Tmin 800 ms Tmax  2700 ms Firing  200 ms Reaction time of the photoresistor  40 ms Deviations: 2-10 %

20. Order parameter: - normalized phase-histogram: smoothing:

21. Perspectives • Separately programmable oscillators: • Tmin, Tmax, Tflash, light intensity A, threshold G CNN model using pulse-coupled oscillators: Benefits: - global coupling - dynamical inputs - time-delays No independent A(i,j;k,l) • pattern recognition, detecting spatio-temporal events • studying the role of reaction time Δ • dynamically changing the parameters

22. Conclusion • Realistic random numbers • Stochastic simulations on lattice models • The site-percolation problem • The two-dimensional ising model - many related problems could be also simulated • Locally variant CNN – very fast stochastic optimization algorithm for spin-glass models • Further motivating the development of CNN-UM hardwares • CNN built up by pulse-coupled oscillators • Ineteresting collective behavior – further studies • Communication with light could be useful idea also in hardware projects

23. Journal publications • M. Ercsey-Ravasz, T. Roska, Z. Néda, “Perspectives for Monte Carlo simulations on the CNN universal machine”, Int. J. of Modern Physics C, Vol. 17, No. 6, pp. 909-923, 2006 • M. Ercsey-Ravasz, T. Roska, Z. Néda, “Stochastic simulations on the cellular wave computers”, Eur. Phys. J. B, Vol. 51, No. 3, pp. 407-412, 2006 • M. Ercsey-Ravasz, T. Roska, Z. Néda, “Statistical physics on cellular neural network computers”, Physica D: Nonlinear Phenomena, vol. Special issue: “Novel computing paradigms: Quo Vadis?”, 2008, accepted, http://dx.doi.org/10.1016/j.physd.2008.03.28

24. International conferences • M. Ercsey-Ravasz, T. Roska, and Z. Neda, “Random number generator andmonte carlo type simulations on the cnn-um,” in Proceedings of the 10th IEEEInternational Workshop on Cellular Neural Networks and their applications,(Istanbul, Turkey), pp. 47–52, Aug. 2006. • M. Ercsey-Ravasz, Z. Sarkozi, Z. Neda, A. Tunyagi, and I. Burda, “Collectivebehavior of ”electronic fireflies”,SynCoNet 2007: International SymposiumonSynchronization in Complex Networks, July 2007. • M. Ercsey-Ravasz, T. Roska, and Z. Neda, “Statistical physics on cellularneural network computers.” International conference ”Unconventionalcomputing:Quo vadis?”, Mar. 2007. • M. Ercsey-Ravasz, T. Roska, and Z. Neda, “Spin-glasses on a locally variantcellular neural network.” International Conference on Complex Systems andNetworks, July 2007. • M. Ercsey-Ravasz, T. Roska, and Z. Neda, “Applications of cellular neuralnetworks in physics.” RHIC Winterschool, Nov. 2005. • M. Ercsey-Ravasz, T. Roska, and Z. N´eda, “The cellular neural networkuniversal machine in physics.” International Conference on ComputationalMethods in Physics, Nov. 2006. • M. Ercsey-Ravasz, T. Roska, and Z. Neda, “NP-hard optimization using locally variant CNN,” accepted in the Proceedings of the CNNA2008.

25. Thank You!