Dr. Strangestuff Or: How I learned to stop programming and love carbon foam

1. Dr. StrangestuffOr: How I learned to stop programming and love carbon foam Jonathan W. Mills • Research Fellow and 2007Leverhulme Trust Professor • University of the West of England • Bristol BS16 1QY • United Kingdom • Associate ProfessorComputer Science • Indiana University • Bloomington, IN 47405 • United States of America • Visiting ResearcherThe Courant Institute • New York University • New York, NY 20002 • United States of America

2. Organization of This Talk • The People • Digital vs. Analog Computers • The Extended Analog Computer (EAC) • Implementations of the EAC • A Few Applications • Protein Structure Prediction • Embedding Deutsch’s Problem in the EAC • New Kinds of Computers!

3. The People

4. Lee A. Rubel Rubel first described the extended analog computer (EAC) in 1993 to overcome limitations of Shannon’s general purpose analog computer (GPAC). Aside from some correspondence with Mills, it was his only publication about the EAC. Lee A. Rubel (1928-1995)

5. Primary EAC Research Team Bryce Himebaugh (left) designs the EACs that Mills (right)uses to conduct real and gedanken computing experiments.

6. “Second String” EAC Research Team This team performs “acoustic experiments” at Arcosanti, AZ.

7. Digital vs. Analog Computers

8. What Is a Computer? A computer is any physical object that can be reconfigured to solve multiple problems, that is, to answer many different questions. (If it cannot be reconfigured, and can answer only one question, it is nota computer according to this definition, but it may be an experiment.)

9. Algorithm (digital) compared toAnalogy (analog) From Mills, The Nature of the EAC, Physica D (2008)

10. Another View of the EAC Paradigm AnalogyMills, The Nature of the EAC, Physica D (2008)

11. Feynman questioned an apparent limit of digital computers (Algorithm):“It always bothers me that, according to the laws [of physics] as we understand them today, it takes a [digital] computing machine an infinite number of logical operations to figure out what goes on in no matter how tiny a volume of space, and no matter how tiny a region of time.”He also said, “There’s a lot of room at the bottom.”

12. Feynman was using the paradigm algorithm But by following the paradigm analogy to its logical conclusion, the laws of physics as we understand them comprise the “instruction set” of an EAC.Bulk and minimally structured matter form its computational components. The EAC takes advantage of Feynman’s “… room at the bottom.”

13. Computational Paths of algorithm and analogy

14. The Extended Analog Computer (EAC)

15. Basic EAC Model An extended analog computer implements an analogy, explicitly using physical laws and implicitly using mathematical principles to compute. It is implemented as a continuous-valued, inherently parallel, reconfigurable processor with two “instruction” classes: solve-partial-differential-equation, and compute-piecewise-linear-function.

16. Operating Principles Physics Mathematics Conservation Laws Abstract Objects mass, energy, charge, time, information variables, constants, metric spaces Symmetries in Physical Law Operators translation in space, translation in time, arithmetic, inversion, ordinary and rotation in space, velocity (relativity), partial differentiation, substitution replacement of atoms and charge carriers Pauli Exclusion PrinciplePrinciples “pruning” principle prevents exponential limits, analytic continuation, extremely growth in number of states to be computed well-posed determinism (EWP), quantum non-determinism (Q-box)

17. Dualities • Turing machine • Extended Analog Computer • Algorithms • Analogies • 1D von Neumann bottleneck • 2D/3D Non-von bottleneck • Inherently sequential • Inherently parallel • Fixed internal precision • Fixedexternal precision that increases temporally that increases spatially • Explicit pseudo-randomness • Implicit randomness • Silicon, GaAs • Anything conductive • Transistors • Diodes, surfaces, solids • Increasingly error prone • Implicitly error tolerant as devices get smaller via structure and matter

18. Liberation from von Neumann, Moore and Backus • No von Neumann bottleneck (no CPU-cache-memory path) • No “memory wall” (no computational memory) • Single EACs are inherently parallel (no internal CPU pipelines) • Moore’s Law irrelevant (computational “devices” in sheet are atoms) • EAC “CPU” is robust (no logic gates, transistors or memory cells) • Multiple EACs are composed functionally (no side-effects) • Scalable parallelism (may be multi-unit pipelines, MIMD, or mixed) • “Software” is reusable (physical laws and given materials are invariant)

19. Digital 1D versus EAC 2D Bottlenecks Digital multi-core processor Extended Analog Computer

20. The Benefit of Noise Electromagnetic interference in the EAC can create a noisy gradient when small currents are input. It generates sequences of random numbers, which digital computers cannot produce (they compute pseudo-random numbers). Noise is useful in some applications (Monte Carlo methods; simulated annealing, genetic algorithms).

21. Pauli Pruning “Envelope” of possible paths Probable paths An actual path The Pauli Exclusion Principle means that natural computers do not have to compute all possible states to obtain a result. The drawback is that the output, although it may be completely accurate, is probabilistic and not certain. This phenomenon has been observed in the output of Lukasiewicz logic arrays.

22. Inherent Fault Tolerance The sheets are fault tolerant because current flows adaptively in conductive materials, as this experiment demonstrates. The LLAs are error-reducing, resisting errors due to their structure as binary trees.(An Information-theoretic Analysis of Lukasiewicz Logic Arrays, Montante, 1994)

23. Implementations of the EAC

24. Early EACs “Sponge Bob,” September 2004 to present(can be manipulated over the web athttp://cgi.cs.indiana.edu/~bhimebau/eac.cgi) 1995

25. “Sponge Bob Marley,” a 3D Jell-O® EAC Prototype for injection molded & laser-polymerized 3D processors

26. USB Carbon Foam EAC with Digital Host October 2005 to present

27. EAC Architecture Block Diagram

28. Host Data Is Converted to Continuous EAC Inputs Once “inside” the interface, computation is continuous

29. EAC Results Discretized and Sent Back to the Host Recurrent computations “inside” the interface remain continuous

30. A Few Applications

31. Three Ways to Configure the EAC • Design an EAC configuration by inspection, using similar properties in each system to create an analogy • Use an evolutionary algorithm, such as Particle Swarm Optimization, to evolve the configuration in a high-order dimensional space • Use sensor feedback and simulated annealing to freeze a minimal result out of an “energy space;” this may be combined with either of the first two techniques

32. Silicon Retina: Alternative to Mead & Koch Originally built in 1995, this design established structure for recent EACs It also established a fundamental application technique: sense (or generate) and recognize, which is useful for many problems other than vision

33. Nortel, Canada : Fast DDoS Detector This simplified model learned an “alphabet” of router traffic patterns to spot DDoS attacks

34. Siemens, People’s Republic of China:Ordinary Differential Equations The intended application integrates an ODE to model a system that the EAC retina “watches” to yield intelligent and adaptive feedback control.

35. Protein Structure Prediction

36. Too Complicated! Ricin, a simple toxic lectin

37. Back to Organic Chem 101 Valine (hydrophobic) Asparagine (hydrophilic)

38. Start by Modeling Coarse Spatial Structure Valine (hydrophobic)

39. Rotation Models are “Slices” Each pair is a spatial slice of the “shape” along the backbone of a protein. How do their sidechains interact to model the protein’s folded structure?

40. Add Another EAC Level to Model van der Waals Forces Two levels interact through Lukasiewicz logic functions to reach an energy minimum between parts of protein. Scalability of EAC allows it to “zoom” in or out to model atoms, molecular groups or side chains.

41. Behavior Can Be Observed As 2D Map of Slice Attract Attract Repel

42. Structure Prediction Needs More Layers This EAC architecture was named “The Torte” after the rich European layer cake, due its alternating layers of conductive sheets and Lukasiewicz logic functions.

43. Particle Swarm Using EAC Outputs Evolves Low-Energy Structure

44. Particle Swarm Optimization Process 1. Initialize population in hyperspace. 2. Evaluate fitness of individual particles. 3. Modify velocities based on previous best and global (or neighborhood) best. 4. Terminate on some condition. 5. Go to step 2.

45. Features of Particle Swarm Optimization • Population initialized by assigning random positions and velocities; potential solutions are then flown through hyperspace (think of a swarm of bees whose center of mass seeks the global optimum, although all bees contribute to it, no one “bee” may be at the optimal point). • Each particle (“bee”) keeps track of its best (highest fitness) position in hyperspace. • This is called “pbest” for an individual particle • It is called “gbest” for the best in the population • It is called “lbest” for the best in a defined neighborhood • At each time step, each particle (“bee”) stochastically accelerates toward its pbest and gbest (or lbest—good for protein folding). • When the center of mass does not vary within some epsilon over a pre-determined number of iterations, a set of particle properties (“the bee swarm”) has been found that defines the potential global optimum.

46. PSO Velocity Update Equations • Global version: Where d is the dimension, c1and c2are positive constants, rand and Rand are random functions, and w is the inertia weight. For neighborhood version, change pgdto pld. PSO has been found to work very well with the EAC

47. Manual Demonstration of PSO