Physical Limits of Computing A Brief Introduction

1 / 42

# Physical Limits of Computing A Brief Introduction - PowerPoint PPT Presentation

Physical Limits of Computing A Brief Introduction. Dr. Michael P. Frank mpf@cise.ufl.edu Dept. of Computer & Information Science & Engineering (Affil. Dept. of Electrical & Computer Engineering) University of Florida, Gainesville, Florida. Presented at:

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about 'Physical Limits of Computing A Brief Introduction' - Melvin

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

### Physical Limits of ComputingA Brief Introduction

Dr. Michael P. Frankmpf@cise.ufl.eduDept. of Computer & Information Science & Engineering(Affil. Dept. of Electrical & Computer Engineering)University of Florida, Gainesville, Florida

Presented at:

2004 Computing Beyond Silicon Summer School (Week 4)California Institute of Technology

Abstract
• Physical and computational systems share a number of common characteristics.
• They are both special cases of the more general concept of dynamical systems.
• In fact, we can even show that there is a fundamental underlying unity between the physical and computational domains.
• E.g., in this talk, we will survey some ways to understand a variety of key physical concepts in computational terms.
• Due to this underlying unity, physical systems have firm limits on their computational capabilities.
• Since a computation embedded within a physical system clearly cannot exceed the raw computational capabilities of the physical system itself.
• We review some of the known limits.
• On information capacity, processing rate, and communication bandwidth.
Physics as Computation
• Preview: Most/all physical quantities can be validly reinterpreted in terms of information and computation.
• Physical entropyis
• Incompressible information.
• Physical actionis
• Total amount of (quantum-physical) computation.
• Physical energyis
• The rate of physical computation.
• The various different forms of energy correspond to physical computation that is occupied doing different kinds of things.
• Physical temperature is (proportional to)
• Physical rate of computing per bit of information capacity.
• The “clock speed” for physical computation.
• Physical momentumis
• Amount of “motional” computation per unit distance translated…
• there are others for angular momentum, velocity, etc. …
• These identities can be made rigorous!
• We will sketch the arguments later if there is time…
Fundamental Physics Implies Various Firm Limits on Computing

ImpliedUniversal Facts

Affected Quantities in Information Processing

Thoroughly ConfirmedPhysical Theories

Speed-of-LightLimit

Communications Latency

Theory ofRelativity

Information Capacity

UncertaintyPrinciple

Information Bandwidth

Definitionof Energy

Memory Access Times

QuantumTheory

Reversibility

2nd Law ofThermodynamics

Processing Rate

Energy Loss per Operation

Gravity

Some limits…
• Communications latency…
• Over distance d is at least t = d/c.
• Despite “spooky” non-local-seeming quantum statistics.
• Information capacity…
• For systems of given size & energy is finite.
• Obtained by counting numbers of distinct quantum states.
• Information bandwidth…
• Limited for flows of given power and cross-sectional area.
• Obtained from capacity and propagation velocity limits
• Memory access times…
• Limited by information density & velocity…
• Processing rate…
• Limited by accessible energy, indirectly by size.
• Also limited by power constraints & energy efficiency.
• Energy efficiency…
• Limited by Landauer bound for irreversible computing,
• No technology-independent limits for reversible computing yet known.
Entropy and Information
• The following definitions of the entropy content of a given physical system can all be shown to be essentially equivalent…
• Expected logarithm of the state improbability 1/p.
• Given a probability distribution over system states.
• Expected size of the smallest compressed description of the system’s state.
• Using the best available description language & compressor.
• Expected amount of information in the state that cannot be reversibly decomputed.
• Using the best available mechanism.
• Expected amount of a system’s information capacity that is in use
• It cannot be used to store newly-computed information for later retrieval.
Action and Amount of Computation
• In quantum mechanics,
• States are represented as complex-valued vectors v,
• & temporal transformations are represented by unitary operators (generalized rotations) U on the vector space.
• The U’s may be parameterized as eiHθ(H hermitian, θ real)
• We can characterize the magnitude of a given vector rotation Uv = eiHθv by
• The area swept out in the complex plane by the normalized vector components as θ is swept from 0 to a given value.
• Important conjecture: This quantity is basis-independent!
• We can characterize the action performed by a given unitary transform operating on a set of possible v’s as the maximum rotation magnitude over the v’s.
• Or, if we have a probability distribution over initial vectors, we can define an expected action accordingly.
• The connection with computation is provided by showing that it takes a minimum area (action) of π/4 to flip a bit.
• I.e., minimum angle of π/2 to rotate to an orthogonal vector.
• It takes a minimum action of π/2 (annihilate/create pair) to move a state forward by 1 position along an unbounded chain.
• The total action of a transform gives the total number of such operations.
Energy and Rate of Computation
• The energy of an eigenvector of H is the corresponding eigenvalue.
• The average energy of a general quantum state follows directly from the eigenstate probabilities.
• The average energy is exactly the rate at which complex-plane area is swept out (action accumulated).
• In the energy basis, and also in other bases.
• Thus, if action is amount of computation, then energy is rate of computation.
Generalized Temperature
• The concept of temperature can be generalized to apply even to non-equilibrium systems.
• Where entropy is less than the maximum.
• Example: Consider an ideal Fermi gas.
• Heat capacity/fermion is C = π2k2T/2μ.
• μ = Fermi energy; k = log e; T = temperature
• Equilibrium temperature turns out to be:
• T = (2/πk)(Exμ)1/2, thus C = πk(Ex/μ)1/2 where:

Ex = E − E0, avg. energy excess/fermion rel. to T=0

• Equilibirum (max) entropy/fermion is:
• Smax = ∫dS = ∫d′Q/T = ∫dEx/T = πk(Ex/μ)1/2 = C
• Consider this to be the total information content Smax = Itot = S + X (entropy plus extropy).
• We thus have: T = 2(Ex/Itot)
• The temperature is simply 2× the excess energy per unit of total information content.
• Note that the expression Ex/Itot is well-defined even for non-equilibrium states, where the entropy is S < Smax = Itot.
• Thus, we can validly ascribe a (generalized) temperature to such states.
Generalized Temperature as “Clock Speed”
• Consider systems such as the Fermi gas, where T= cE/I.
• Where c is a constant of integration.
• E is excess energy above the ground state.
• I is total physical info. content
• For such systems, we can say that the generalized temperature gives a measure of the energy content, per bit of physical information content.

Eb = c-1Tb =c-1kBT ln 2

• Since energy (we saw) gives the rate of computing, the temperature therefore gives the rate of computing per bit.
• In other words, the clock frequency!
• For our case c=2, room temperature corresponds to a max. frequency of:

fmax = 2c-1Tb/h = kB(300 K)(ln 2)/h = ~4.3 THz

• Comparable to freq. of room-T IR photons
• A computational subsystem that is at a generalized temperature equal to room temperature can never update its digital state at a higher frequency than this!

### Information Limits

Some Quantities of Interest
• We would like to know if there are limits on:
• Infropy density
• = Bits per unit volume
• Affects physical size and thus propagation delayacross memories and processors. Also affects cost.
• Infropy flux
• = Bits per unit area per unit time
• Affects cross-sectional bandwidth, data I/O rates, rates of standard-information input & effective entropy removal
• Rate of computation
• = Number of distinguishable-state changes per unit time
• Affects rate of information processing achievable in individual devices
Bit Density: No classical limit
• In classical (continuum) physics, even a single particle has a real-valued position+momentum
• All such states are considered physically distinct
• Each position & momentum coordinate in general requires an infinite string of digits to specify:
• x = 4.592181950149194019240194209490124… meters
• p = 2.393492340940140914291029091230103… kg m/s
• Even the smallest system contains an infinite amount of information!  No limit to bit density.
• This picture is the basis for various analog computing models studied by some theoreticians.
• Wee problem: Classical physics is dead wrong!
The Quantum “Continuum”
• In QM, still  uncountably many describable states (mathematically possible wavefunctions)
• Can theoretically take infinite info. to describe
• But, not all this info has physical relevance!
• States are only physically distinguishable when their state vectors are orthogonal.
• States that are only indistinguishably different can only lead to indistinguishably different consequences (resulting states)
• due to linearity of quantum physics
• There is no physical consequence from presuming an infinite # of bits in one’s wavefunction
Quantum Particle-in-a-Box
• Uncountably manycontinuouswavefunctions?
• No, can expresswave as a vectorover countablymany orthogonalnormal modes.
• Fourier transform
• High-frequencymodes have higherenergy (E=hf); energy limits implythey are unlikely.
Ways of Counting States

• For a system w. a constant # of particles:
• # of states = numerical volume of position-momentum configuration space (phase space)
• in units where h=1.
• Approached in macroscopic limit.
• Unfortunately, # of particles not usually constant!
• Quantum field theory bounds:
• Smith-Lloyd bound. Still ignores gravity.
• General relativistic bounds:
• Bekenstein bound, holographic bound.
Smith-Lloyd Bound

Smith ‘95Lloyd ‘00

• Based on counting field modes.
• S = entropy, M = mass, V = volume
• q = number of distinct particle types
• Lloyd’s bound is tighter by a factor of
• Note:
• Entropy density scales with 3/4 power of mass-energy density
• E.g., Increasing entropy density by a factor of 1,000 requires increasing energy density by 10,000×.
Examples w. Smith-Lloyd Bound
• For systems at the density of water (1 g/cm3), composed only of photons:
• Smith’s example: 1 m3 box holds 6×1034 bits
• = 60 kb/Å3
• Lloyd’s example: 1 liter “ultimate laptop”, 2×1031 b
• = 21 kb/Å3
• Cool, but what’s wrong with this picture?
• Example requires very high temperature+pressure!
• Temperature around 1/2 billion Kelvins!!
• Photonic pressure on the order of 1016 psi!!
• “Like a miniature piece of the big bang.” -Lloyd
• Probably not feasible to implement any time soon!
More Normal Temperatures
• Let’s pick a more reasonable temperature: 1356 K (melting point of copper):
• Entropy density of light only 0.74 bits/m3!
• Less than the bit density in a DRAM today!
• Bit size comparable to wavelength of optical-frequency light emitted by melting copper
• Lesson: Photons are not a viable information storage medium at ordinary temperatures.
• Not dense enough.
• CPUs that do logic with optical photons can’t have logic devices packed very densely.
Entropy Density of Solids
• Can easily calculate from standard empirical thermochemical data.
• Obtain entropy by integrating heat capacity ÷ temperature, as temperature increases…
• Example result, for copper:
• Has one of the highest entropy densities among pure elements at atmospheric pressure
• @ room temperature: 6 bits/atom, 0.5 b/Å3
• At boiling point: 1.5 b/Å3
• Cesium has one of the highest #bits/atom at room temperature, about 15. -But only 0.13 b/Å3
• Lithium has a high #bits/mass, 0.7 bits/amu.

1012×denser thanits light!

Related toconductivity?

Some Quantities of Interest
• We would like to know if there are limits on:
• Infropy density
• = Bits per unit volume
• Affects physical size and thus propagation delayacross memories and processors. Also affects cost.
• Infropy flux
• = Bits per unit area per unit time
• Affects cross-sectional bandwidth, data I/O rates, rates of standard-information input & effective entropy removal
• Rate of computation
• = Number of distinguishable-state changes per unit time
• Affects rate of information processing achievable in individual devices
Smith-Lloyd Bound

Smith ‘95Lloyd ‘00

• Based on counting orthogonal field modes.
• S = entropy, M = mass, V = volume
• q = number of distinct particle types
• Lloyd’s bound is tighter by a factor of
• Note:
• Entropy density scales with 3/4 power of mass-energy density
• E.g., Increasing entropy density by a factor of 1,000 requires increasing energy density by 10,000×.
Whence this scaling relation?
• Note that in the field theory limit, S E3/4.
• Where does this come from?
• Consider a typical freq. in field spectrum
• Note that the minimum size of agiven wavelet is ~its wavelength .
• # of distinguishable wave-packet location states in a given volume  1/3
• Each such state carries a little entropy
• occupation number of that state (# of photons in it)
• 1/3 particles each energy 1/, 1/4 energy
• S1/3  E1/4  SE3/4
Whence the distribution?
• Could the use of more particles (with less energy per particle) yield greater entropy?
• What frequency spectrum (power level or particle number density as a function of frequency) gives the largest # states?
• Note  a minimum particle energy due to box size
• No. The Smith-Lloyd bound is based on the blackbody radiation spectrum.
• We know this spectrum has the maximum infropy among abstract states, b/c it’s the equilibrium state.
• Empirically verified in hot ovens, etc.
General-Relativistic Bounds
• The Smith-Lloyd bound does not take into account the effect of gravity.
• Earlier bound from Bekenstein: Derives a limit on entropy from black-hole physics:

S < 2ER / c

E = total energy

• Limit only attained by black holes!
• Black holes have 1/4 nat entropy per square Planck length of surface (event horizon) area.
• Minimum size of a nat: 2 Planck lengths, square

4×1039 b/Å3average ent. dens.of a 1-m radiusblack hole!(MassSaturn)

The Holographic Bound
• Based on Bekenstein black-hole bound.
• The maximum entropy within any surface of area A (independent of energy!) isA/(2LP)2
• LP is Planck length (see lecture on units)
• Implies any 3D object (of any size) could be completely defined via a flat (2D) “hologram” on its surface having Planck-scale resolution.
• Bound is only really achieved by a black hole with event horizon=that surface.
Do Black Holes Destroy Information?
• Currently, it seems that no one completely understands how information is preserved during black hole accretion for later re-emission as Hawking radiation.
• Via infinite time dialation at surface?
• Some researchers (e.g. Hawking) claimed that black holes must be doing something irreversible in their interior (destroying information).
• The arguments for this seem not very rigorous...
• The issue is not completely resolved, but I have many papers on it if you’re interested.
• Incidentally, Hawking recently conceded a bet on this.
Implications of Density Limits
• Minimum device size
• thus minimum communication latency (as per earlier).
• Minimum device cost, given a minimum cost of matter/energy.
• Implications for communications bandwidth limits (coming up)
Communication Limits
• Latency (propagation-time delay) limit from earlier, due to speed of light.
• Teaches us scalable interconnection technologies
• Bandwidth (infropy rate) limits:
• Classical information-theory limit (Shannon)
• Limit, per-channel, given signal bandwidth & SNR.
• Limits based on field theory (Smith/Lloyd)
• Limit given only area and power.
• Applies to I/O, cross-sectional bandwidths in parallel machines, and entropy removal rates.
Hartley-Shannon Law
• The maximum information rate (capacity) of a single wave-based communication channel is:C = B log (1+S/N)
• B = bandwidth of channel in frequency units
• S = signal power level
• N = noise power level
• Law not sufficiently powerful for our purposes!
• Does not tell us how many effective channels are possible, given available power and/or area.
• Does not give us any limit if we are allowed to increase bandwidth or decrease noise arbitrarily.
Density & Flux
• Note that any time you have:
• a limit  on density (per volume) of something
• & a limit v on its propagation velocity
• this automatically implies:
• a limit F = v on the flux
• by which I mean rate per time per area
• Note also we always have a limit c on velocity!
• At speeds near c must account for relativistic effects
• Slower velocities also relevant:
• electron saturation velocity in various materials
• velocity of air or liquid coolant in a cooling system
• Thus density limit  implies flux limit F=c

Cross-section

v

Relativistic Effects
• For normal matter (bound massive-particle states) moving at a velocity v near c:
• Entropy density increases by factor  = (1(v/c)2)1
• Due to relativistic length contraction
• But, energy density increases by factor 2
• Both length contraction & mass amplification
•  entropy density scales up only w. square root (1/2 power) of energy density from high velocity
• Note that light travels at c already,
• & its entropy density scales with energy density to the 3/4 power.  Light wins as vc.
• If you want to maximize entropy/energy flux
Entropy Flux Using Light

Smith ‘95

• FS = entropy flux
• FE = energy flux
• SB = Stefan-Boltzmann constant, 2kB4/60c23
• Derived from same field-theory arguments as the density bound.
• Again, blackbody spectrum optimizes entropy flux given energy flux
• It is the equilibrium spectrum
Entropy Flux Examples
• Consider a 10cm-wide, flat, square wireless tablet with a 10 W power supply.
• What’s it’s maximum rate of bit transmission?
• Independent of spectrum used, noise floor, etc.
• Energy flux 10 W/(2·(10 cm)2) (use both sides)
• Smith’s formula gives 2.2×1021 bps
• What’s the rate per square nanometer surface?
• Only 109 kbps! (ISDN speed, in a 100 GHz CPU?)
• 100 Gbps/nm2 nearly 1 GW power!

Light is not infropically dense enough for high-BW comms. between densely packed nanometer-scale devices at reasonable power levels!!!

Entropy Flux w. Atomic Matter
• Consider liquid copper (~1.5 b/Å3) moving along at a leisurely 10 cm/s…
• BW=1.5x1027 bps through the 10-cm wide square!
• A million times higher BW than with 10W light!
• 150 Gbps/nm2 entropy flux!
• Plenty for nano-scale devices to talk to their neighbors
• Most of this entropy is in the conduction electrons...
• Less conductive materials have much less entropy
• Lesson:
• For maximum bandwidth density at realistic power levels, encode information using states of matter (electrons) rather than states of radiation (light).

Exercise: Kinetic energy flux?

Some Quantities of Interest
• We would like to know if there are limits on:
• Infropy density
• = Bits per unit volume
• Affects physical size and thus propagation delayacross memories and processors. Also affects cost.
• Infropy flux
• = Bits per unit area per unit time
• Affects cross-sectional bandwidth, data I/O rates, rates of standard-information input & effective entropy removal
• Rate of computation
• = Number of distinguishable-state changes per unit time
• Affects rate of information processing achievable in individual devices

### Speed Limits

The Margolus-Levitin Bound
• The maximum rate  at which a system can transition between distinguishable (orthogonal) states is:  4(E  E0)/h
• where:
• E = average energy (expectation value of energy over all states, weighted by their probability)
• E0 = energy of lowest-energy or ground state of system
• h = Planck’s constant (converts energy to frequency)
• Implication for computing:
• A circuit node can’t switch between 2 logic states faster than this frequency determined by its energy.
Example of Frequency Bound
• Consider Lloyd’s 1 liter, 1 kg “ultimate laptop”
• Total gravitating mass-energy E of 91016 J
• Gives a limit of 51050 bit-operations per second!
• If laptop contains 21031 bits (photonic maximum),
• each bit can change state at a frequency of 2.51019 Hz (25 EHz)
• 12 billion times higher-frequency than today’s 2 GHz Intel processors
• 250 million times higher-frequency than today’s 100 GHz superconducting logic
• But, the Margolus-Levitin limit may be far from achievable!
More Realistic Estimates
• Most of the energy in complex stable structures is not accessible for computational purposes...
• Tied up in the rest masses of atomic nuclei,
• form anchor points for electron orbitals
• mass & energy of “core” atomic electrons,
• fill up low-energy states not involved in bonding,
• & of electrons involved in atomic bonds
• needed to hold the structure together
• Conjecture: Can obtain tighter valid quantum bounds on infropy densities & state-transition rates by considering only the accessible energy.
• Energy whose state-infropy is manipulable.
More Realistic Examples
• Suppose the following system is accessible:1 electron confined to a (10 nm)3 volume, at an average potential of 10 V above ground state.
• Accessible energy: 10 eV
• Accessible-energy density: 10 eV/(10 nm)3
• Maximum entropy in Smith bound: 1.4 bits?
• Not clear whether bound is applicable to this case.
• Maximum rate of change: 9.7 PHz
• 5 million × typical frequencies in today’s CPUs
• 100,000 × frequencies in today’s superconducting logics