Two Tantalizing Concepts

1 / 9

# Two Tantalizing Concepts - PowerPoint PPT Presentation

Two Tantalizing Concepts. Randomness “Any one who considers arithmetical methods of producing random digits is, of course, in a state of sin” – Von Neumann There is no such thing as a random number; there are only methods (e.g., arithmetic vs. quantum) to produce random numbers Redundancy

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about 'Two Tantalizing Concepts' - leroy

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Two Tantalizing Concepts
• Randomness
• “Any one who considers arithmetical methods of producing random digits is, of course, in a state of sin” – Von Neumann
• There is no such thing as a random number; there are only methods (e.g., arithmetic vs. quantum) to produce random numbers
• Redundancy
• Narrow sense, in information theory
• Broad sense, in science and engineering
Pseudorandom Number Generator
• Complexity-based pseudorandomness is a deep concept in theoretical CS (with wide applications in scientific computing such as Monte Carlo methods)
• Engineers generate pseudorandom numbers by arithmetic algorithms such as linear congruential generators and linear feedback shift registers (LFSR)

A maximal LFSR produces an

m-sequence (i.e. cycles through

all possible 2n − 1 states within

the shift register except the state

where all bits are zero), unless it

contains all zeros, in which case

it will never change.

AWGN
• How is AWGN generated under MATLAB?
• Yes, by “randn” function but how does it work?
• RANDN('seed') in MATLAB4 vs. RANDN('state',J) in MATLAB5
• Subtle implication into denoising experiments
Randomness in Nature

Star Constellation

Waitomo Glow-worm Caves on Lake Roturura

Random pin-dropping

Homogeneous Poisson

Redundancy
• What is redundancy in Shannon’s mind?
• In source coding, redundancy refers to the gap between the source entropy and the actual bit rate (so we want to eliminate redundancy).
• In channel coding, redundancy refers to the “extra bits” used for error correction (so we add redundancy in a controlled fashion).
• Divide-and-conquer: an engineering solution
Redundancy in Nature
• Redundancy in language
• Why are human languages redundant? How is it possible for young kids to learn speaking a language?
• Redundancy in natural world
• Why are natural images compressible? How does human vision systems work?
• Redundancy in genetics
• Why are Chromosomes in pairs?
Redundancy in SP
• Sampling – an artificial tool which we have not yet understood well
• No signal from the natural world is band-limited
• Shannon/Nyquist’s sampling theorem never holds in practice
• “Truth is much too complicated to allow anything but approximations.”
• Uniform sampling- a “sin” operator?
Two Contrasting Views