1 / 9

Two Tantalizing Concepts

Two Tantalizing Concepts. Randomness “Any one who considers arithmetical methods of producing random digits is, of course, in a state of sin” – Von Neumann There is no such thing as a random number; there are only methods (e.g., arithmetic vs. quantum) to produce random numbers Redundancy

leroy
Download Presentation

Two Tantalizing Concepts

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Two Tantalizing Concepts • Randomness • “Any one who considers arithmetical methods of producing random digits is, of course, in a state of sin” – Von Neumann • There is no such thing as a random number; there are only methods (e.g., arithmetic vs. quantum) to produce random numbers • Redundancy • Narrow sense, in information theory • Broad sense, in science and engineering

  2. Pseudorandom Number Generator • Complexity-based pseudorandomness is a deep concept in theoretical CS (with wide applications in scientific computing such as Monte Carlo methods) • Engineers generate pseudorandom numbers by arithmetic algorithms such as linear congruential generators and linear feedback shift registers (LFSR) A maximal LFSR produces an m-sequence (i.e. cycles through all possible 2n − 1 states within the shift register except the state where all bits are zero), unless it contains all zeros, in which case it will never change.

  3. AWGN • How is AWGN generated under MATLAB? • Yes, by “randn” function but how does it work? • RANDN('seed') in MATLAB4 vs. RANDN('state',J) in MATLAB5 • Subtle implication into denoising experiments

  4. Randomness in Nature Star Constellation Waitomo Glow-worm Caves on Lake Roturura

  5. Random pin-dropping Homogeneous Poisson

  6. Redundancy • What is redundancy in Shannon’s mind? • In source coding, redundancy refers to the gap between the source entropy and the actual bit rate (so we want to eliminate redundancy). • In channel coding, redundancy refers to the “extra bits” used for error correction (so we add redundancy in a controlled fashion). • Divide-and-conquer: an engineering solution

  7. Redundancy in Nature • Redundancy in language • Why are human languages redundant? How is it possible for young kids to learn speaking a language? • Redundancy in natural world • Why are natural images compressible? How does human vision systems work? • Redundancy in genetics • Why are Chromosomes in pairs?

  8. Redundancy in SP • Sampling – an artificial tool which we have not yet understood well • No signal from the natural world is band-limited • Shannon/Nyquist’s sampling theorem never holds in practice • “Truth is much too complicated to allow anything but approximations.” • Uniform sampling- a “sin” operator?

  9. Two Contrasting Views • Redundancy is bad • Since most natural signals are still so compressible, we should acquire much fewer samples (joint design of sampling and coding) • The emerging “compressive sensing” paradigm • Redundancy is good • Redundancy is essential to human intelligence especially the redundancy exploitation hypothesis advocated by H. Barlow • To solve high-level computer vision problems, get low-level (image sampling, feature extraction) done right first.

More Related