compressive sampling of analog signals l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Compressive Sampling (of Analog Signals) PowerPoint Presentation
Download Presentation
Compressive Sampling (of Analog Signals)

Loading in 2 Seconds...

play fullscreen
1 / 51

Compressive Sampling (of Analog Signals) - PowerPoint PPT Presentation


  • 95 Views
  • Uploaded on

Compressive Sampling (of Analog Signals). Moshe Mishali Yonina C. Eldar Technion – Israel Institute of Technology http://www.technion.ac.il/~moshiko moshiko@tx.technion.ac.il http://www.ee.technion.ac.il/people/YoninaEldar yonina@ee.technion.ac.il.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Compressive Sampling (of Analog Signals)' - silvio


Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
compressive sampling of analog signals

Compressive Sampling(of Analog Signals)

Moshe Mishali Yonina C. Eldar

Technion – Israel Institute of Technology

http://www.technion.ac.il/~moshikomoshiko@tx.technion.ac.il

http://www.ee.technion.ac.il/people/YoninaEldar yonina@ee.technion.ac.il

Advanced topics in sampling (Course 049029)

Seminar talk – November 2008

context sampling
Context - Sampling

Continuous signal

Analog world

Digital world

Sampling

A2D

Reconstruction

D2A

compression
Compression

“Can we not just directly measure the part that will not end up being thrown away ?”

Donoho

Compressed 392 KB15%

Compressed 148 KB6%

Compressed 950 KB38%

Original 2500 KB100%

outline
Outline
  • Mathematical background
  • From discrete to analog
  • Uncertainty principles for analog signals
  • Discussion
r eferences
References
  • M. Mishali and Y. C. Eldar, "Reduce and Boost: Recovering Arbitrary Sets of Jointly Sparse Vectors," IEEE Trans. on Signal Processing, vol. 56, no. 10, pp. 4692-4702, Oct. 2008.
  • M. Mishali and Y. C. Eldar, "Blind Multi-Band Signal Reconstruction: Compressed Sensing for Analog Signals," CCIT Report #639, Sep. 2007, EE Dept., Technion.
  • Y. C. Eldar, "Compressed Sensing of Analog Signals", submitted to IEEE Trans. on Signal Processing, June 2008.
  • Y. C. Eldar and M. Mishali, "Robust Recovery of Signals From a Union of Subspaces", arXiv.org 0807.4581, submitted to IEEE Trans. Inform. Theory, July 2008. #
  • Y. C. Eldar, "Uncertainty Relations for Analog Signals", submitted to IEEE Trans. Inform. Theory, Sept. 2008.
slide6
Mathematical background
  • Basic ideas of compressed sensing
  • Single measurement model (SMV)
  • Multiple- and Infinite- measurement models (MMV, IMV)
  • The “Continuous to finite” block (CTF)
compressed sensing
Compressed Sensing

“Can we not just directly measure the part that will not end up being thrown away ?”

Donoho

“sensing … as a way of extracting information about an object from a small number of randomly selected observations”

Candès et. al.

AnalogAudioSignal

Nyquist rateSampling

CompressedSensing

Compression(e.g. MP3)

High-rate

Low-rate

concept
Concept

Goal: Identify the bucket with fake coins.

Weigh a coinfrom each bucket

Compression

Nyquist:

Bucket #

numbers

1 number

Weigh a linear combinationof coins from all buckets

Compressed Sensing:

Bucket #

1 number

mathematical tools
Mathematical Tools

non-zero entries at least measurements

Recovery: brute-force, convex optimization, greedy algorithms, and more…

cs theory on 2 slides
CS theory – on 2 slides

Compressed sensing (2003/4 and on) – Main results

is uniquely determined by

Donoho and Elad, 2003

Maximal cardinality of linearly independent column subsets

Hard to compute !

cs theory on 2 slides11
CS theory – on 2 slides

Compressed sensing (2003/4 and on) – Main results

is uniquely determined by

Donoho and Elad, 2003

with high probability

is random

Donoho, 2006 and Candès et. al., 2006

Convex and tractable

Donoho, 2006 and Candès et. al., 2006

Greedy algorithms: OMP, FOCUSS, etc.

NP-hard

Tropp, Cotter et. al. Chen et. al. and many other

sparsity models
Sparsity models

unknowns

measurements

MMV

Joint sparsity

SMV

IMV = Infinite Measurement Vectors (countable or uncountable) with joint sparsity prior

How can be found ?

Infinite many variables

Infinite many constraints

Exploit prior  Reduce problem dimensions

reduction framework
Reduction Framework

Find a frame

for

Solve MMV

Theorem

Mishali and Eldar (2008)

Deterministicreduction

IMV

MMV

Infinite structure allows

CS for analog signals

slide14
From discrete to analog
  • Naïve extension
  • The basic ingredients of sampling theorem
  • Sparse multiband model
  • Rate requirements
  • Multicoset sampling and unique representation
  • Practical recovery with the CTF block
  • Sparse union of shift-invariant model
  • Design of sampling operator
  • Reconstruction algorithm
na ve extension to analog domain
Naïve Extension to Analog Domain

Standard CS

Discrete Framework

Analog Domain

Sparsity prior

what is a sparse analog signal ?

Generalized sampling

Continuoussignal

Operator

Infinite

sequence

Finite dimensional elements

Stability

Randomness  Infinitely many

Random is stable w.h.p

Need structure for efficient implementation

Reconstruction

Finite program, well-studied

Undefined program over a continuous signal

na ve extension to analog domain16
Naïve Extension to Analog Domain

Standard CS

Discrete Framework

Analog Domain

  • Questions:
  • What is the definition of analog sparsity ?
  • How to select a sampling operator ?
  • Can we introduce stucture in sampling and still preserve stability ?
  • How to solve infinite dimensional recovery problems ?

Sparsity prior

what is a sparse analog signal ?

Generalized sampling

Continuoussignal

Operator

Infinite

sequence

Finite dimensional elements

Stability

Randomness  Infinitely many

Random is stable w.h.p

Need structure for efficient implementation

Reconstruction

Finite program, well-studied

Undefined program over a continuous signal

a step backward

“Success has many fathers …”

Whittaker

1915

Nyquist

1928

Kotelnikov

1933

Shannon

1949

A step backward

Every bandlimited signal ( Hertz)

can be perfectly reconstructed from uniform sampling

if the sampling rate is greater than

a step backward18
A step backward

Every bandlimited signal ( Hertz)

can be perfectly reconstructed from uniform sampling

if the sampling rate is greater than

Fundamental ingredients of a sampling theorm

  • A signal model
  • A minimal rate requirement
  • Explicit sampling and reconstruction stages
slide19
Discrete Compressed Sensing

Analog Compressive Sampling

slide20

no more than N bands, max width B, bandlimited to

  • More generally

only sequences are non-zero

Analog Compressed Sensing

What is the definition of analog sparsity ?

  • A signal with a multiband structure in some basis
  • Each band has an uncountable number of non-zero elements
  • Band locations lie on an infinite grid
  • Band locations are unknown in advance

(Mishali and Eldar 2007)

(Eldar 2008)

multi band sensing goals
Multi-Band Sensing: Goals

bands

Sampling

Reconstruction

Analog

Infinite

Analog

Goal: Perfect reconstruction

Constraints:

  • Minimal sampling rate
  • Fully blind system

What is the minimal rate ?

What is the sensing mechanism ?

How to reconstruct from infinite sequences ?

rate requirement
Rate Requirement
  • The minimal rate is doubled.
  • For , the rate requirement is samples/sec (on average).

Theorem (blind recovery)

Mishali and Eldar (2007)

Theorem (non-blind recovery)

Landau (1967)

Average sampling rate

  • Subspace scenarios:
  • Minimal-rate sampling and reconstruction (NB) with known band locations (Lin and Vaidyanathan 98)
  • Half blind system (Herley and Wong 99, Venkataramani and Bresler 00)
sampling
Sampling

Multi-Coset: Periodic Non-uniform on the Nyquist grid

In each block of samples, only are kept, as described by

2

Analog signal

0

Point-wise samples

0

3

3

2

0

3

2

Bresler et. al. (96,98,00,01)

the sampler
The Sampler

in vector form

unknowns

Length .

known

matrix

known

Observation:

is sparse

DTFT

of sampling sequences

Constant

Problems:

  • Undetermined system – non unique solution
  • Continuous set of linear systems

is jointly sparse and unique under appropriate parameter selection ( )

paradigm
Paradigm

Solve finiteproblem

Reconstruct

0

S = non-zero rows

1

2

3

4

5

6

continuous to finite
Continuous to Finite

Solve finiteproblem

Reconstruct

CTF block

MMV

  • span a finite space
  • Any basis preserves the sparsity

Continuous

Finite

algorithm
Algorithm

Perfect reconstruction at minimal rate

Blind system: band locations are unkown

Can be applied to CS of general analog signals

Works with other sampling techniques

Continuous-to-finite block: Compressed sensing for analog signals

CTF

blind reconstruction flow
Blind reconstruction flow

Multi-coset with

Universal

SBR4

Yes

CTF

No

SBR2

No

Bi-section

CTF

Yes

Uniform at

Ideal low-pass filter

Spectrum-blind

Sampling

Spectrum-blind

Reconstruction

final reconstruction non blind
Final reconstruction (non-blind)

Bresler et. al. (96,00)

framework analog compressed sensing
Framework: Analog Compressed Sensing

Sampling signals from a union of shift-invariant spaces (SI)

Subspace

generators

framework analog compressed sensing31
Framework: Analog Compressed Sensing

There is no prior knowledge on the exact indices in the sum

What happen if only K<<N sequences are not zero ?

Not a subspace !

Only k sequences are non-zero

framework analog compressed sensing32
Framework: Analog Compressed Sensing

Step 1: Compress the sampling sequences

Step 2: “Push” all operators to analog domain

CTF

System A

High sampling rate = m/TPost-compression

Only k sequences are non-zero

framework analog compressed sensing33
Framework: Analog Compressed Sensing

Low sampling rate = p/TPre-compression

System B

CTF

Theorem

Eldar (2008)

simulations

Minimal rate

Minimal rate

Simulations

Sampling rate

Sampling rate

Brute-Force

M-OMP

simulations 2
Simulations (2)

0% Recovery

100% Recovery

0% Recovery

100% Recovery

Noise-free

Sampling rate

Sampling rate

SBR4

SBR2

Empirical recovery rate

simulations 3
Simulations (3)

Signal

Reconstruction filter

Amplitude

Amplitude

Output

Time (nSecs)

Time (nSecs)

slide38
Break

(10 min. please)

slide39
Uncertainty principles
  • Coherence and the discrete uncertainty principle
  • Analog coherence and principles
  • Achieving the lower coherence bound
  • Uncertainty principles and sparse representations
discrete coherence
Discrete coherence

Which bases achieve the lowest coherence ?

discrete coherence42
Discrete coherence

Which signal achieves the uncertainty bound ?

Spikes

Fourier

discrete to analog
Discrete to analog
  • Shift invariant spaces
  • Sparse representations
  • Questions:
  • What is the analog uncertainty principle ?
  • Which bases has the lowest coherence ?
  • Which signal achieves the lower uncertainty bound ?
analog uncertainty principle

Theorem

Eldar (2008)

Analog uncertainty principle

Theorem

Eldar (2008)

bases with minimal coherence
Bases with minimal coherence

In the DFT domain

Fourier

Spikes

What are the analog counterparts ?

  • Constant magnitude
  • Modulation
  • “Single” component
  • Shifts
bases with minimal coherence46
Bases with minimal coherence

In the frequency domain

sparse representations
Sparse representations
  • In discrete setting
sparse representations49
Sparse representations
  • Analog counterparts

Undefined program !

But, can be transformed into an IMV model

discussion
Discussion
  • IMV model as a fundamental tool for treating sparse analog signals
  • Should quantify the DSP complexity of the CTF block
  • Compare approach with the “analog” model
  • Building blocks of analog CS framework.