590 likes | 778 Views
Compressive Sampling (of Analog Signals). Moshe Mishali Yonina C. Eldar Technion – Israel Institute of Technology http://www.technion.ac.il/~moshiko moshiko@tx.technion.ac.il http://www.ee.technion.ac.il/people/YoninaEldar yonina@ee.technion.ac.il.
E N D
Compressive Sampling(of Analog Signals) Moshe Mishali Yonina C. Eldar Technion – Israel Institute of Technology http://www.technion.ac.il/~moshikomoshiko@tx.technion.ac.il http://www.ee.technion.ac.il/people/YoninaEldar yonina@ee.technion.ac.il Advanced topics in sampling (Course 049029) Seminar talk – November 2008
Context - Sampling Continuous signal Analog world Digital world Sampling A2D Reconstruction D2A
Compression “Can we not just directly measure the part that will not end up being thrown away ?” Donoho Compressed 392 KB15% Compressed 148 KB6% Compressed 950 KB38% Original 2500 KB100%
Outline • Mathematical background • From discrete to analog • Uncertainty principles for analog signals • Discussion
References • M. Mishali and Y. C. Eldar, "Reduce and Boost: Recovering Arbitrary Sets of Jointly Sparse Vectors," IEEE Trans. on Signal Processing, vol. 56, no. 10, pp. 4692-4702, Oct. 2008. • M. Mishali and Y. C. Eldar, "Blind Multi-Band Signal Reconstruction: Compressed Sensing for Analog Signals," CCIT Report #639, Sep. 2007, EE Dept., Technion. • Y. C. Eldar, "Compressed Sensing of Analog Signals", submitted to IEEE Trans. on Signal Processing, June 2008. • Y. C. Eldar and M. Mishali, "Robust Recovery of Signals From a Union of Subspaces", arXiv.org 0807.4581, submitted to IEEE Trans. Inform. Theory, July 2008. # • Y. C. Eldar, "Uncertainty Relations for Analog Signals", submitted to IEEE Trans. Inform. Theory, Sept. 2008.
Mathematical background • Basic ideas of compressed sensing • Single measurement model (SMV) • Multiple- and Infinite- measurement models (MMV, IMV) • The “Continuous to finite” block (CTF)
Compressed Sensing “Can we not just directly measure the part that will not end up being thrown away ?” Donoho “sensing … as a way of extracting information about an object from a small number of randomly selected observations” Candès et. al. AnalogAudioSignal Nyquist rateSampling CompressedSensing Compression(e.g. MP3) High-rate Low-rate
Concept Goal: Identify the bucket with fake coins. Weigh a coinfrom each bucket Compression Nyquist: Bucket # numbers 1 number Weigh a linear combinationof coins from all buckets Compressed Sensing: Bucket # 1 number
Mathematical Tools non-zero entries at least measurements Recovery: brute-force, convex optimization, greedy algorithms, and more…
CS theory – on 2 slides Compressed sensing (2003/4 and on) – Main results is uniquely determined by Donoho and Elad, 2003 Maximal cardinality of linearly independent column subsets Hard to compute !
CS theory – on 2 slides Compressed sensing (2003/4 and on) – Main results is uniquely determined by Donoho and Elad, 2003 with high probability is random Donoho, 2006 and Candès et. al., 2006 Convex and tractable Donoho, 2006 and Candès et. al., 2006 Greedy algorithms: OMP, FOCUSS, etc. NP-hard Tropp, Cotter et. al. Chen et. al. and many other
Sparsity models unknowns measurements MMV Joint sparsity SMV IMV = Infinite Measurement Vectors (countable or uncountable) with joint sparsity prior How can be found ? Infinite many variables Infinite many constraints Exploit prior Reduce problem dimensions
Reduction Framework Find a frame for Solve MMV Theorem Mishali and Eldar (2008) Deterministicreduction IMV MMV Infinite structure allows CS for analog signals
From discrete to analog • Naïve extension • The basic ingredients of sampling theorem • Sparse multiband model • Rate requirements • Multicoset sampling and unique representation • Practical recovery with the CTF block • Sparse union of shift-invariant model • Design of sampling operator • Reconstruction algorithm
Naïve Extension to Analog Domain Standard CS Discrete Framework Analog Domain Sparsity prior what is a sparse analog signal ? Generalized sampling Continuoussignal Operator Infinite sequence Finite dimensional elements Stability Randomness Infinitely many Random is stable w.h.p Need structure for efficient implementation Reconstruction Finite program, well-studied Undefined program over a continuous signal
Naïve Extension to Analog Domain Standard CS Discrete Framework Analog Domain • Questions: • What is the definition of analog sparsity ? • How to select a sampling operator ? • Can we introduce stucture in sampling and still preserve stability ? • How to solve infinite dimensional recovery problems ? Sparsity prior what is a sparse analog signal ? Generalized sampling Continuoussignal Operator Infinite sequence Finite dimensional elements Stability Randomness Infinitely many Random is stable w.h.p Need structure for efficient implementation Reconstruction Finite program, well-studied Undefined program over a continuous signal
“Success has many fathers …” Whittaker 1915 Nyquist 1928 Kotelnikov 1933 Shannon 1949 A step backward Every bandlimited signal ( Hertz) can be perfectly reconstructed from uniform sampling if the sampling rate is greater than
A step backward Every bandlimited signal ( Hertz) can be perfectly reconstructed from uniform sampling if the sampling rate is greater than Fundamental ingredients of a sampling theorm • A signal model • A minimal rate requirement • Explicit sampling and reconstruction stages
Discrete Compressed Sensing Analog Compressive Sampling
no more than N bands, max width B, bandlimited to • More generally only sequences are non-zero Analog Compressed Sensing What is the definition of analog sparsity ? • A signal with a multiband structure in some basis • Each band has an uncountable number of non-zero elements • Band locations lie on an infinite grid • Band locations are unknown in advance (Mishali and Eldar 2007) (Eldar 2008)
Multi-Band Sensing: Goals bands Sampling Reconstruction Analog Infinite Analog Goal: Perfect reconstruction Constraints: • Minimal sampling rate • Fully blind system What is the minimal rate ? What is the sensing mechanism ? How to reconstruct from infinite sequences ?
Rate Requirement • The minimal rate is doubled. • For , the rate requirement is samples/sec (on average). Theorem (blind recovery) Mishali and Eldar (2007) Theorem (non-blind recovery) Landau (1967) Average sampling rate • Subspace scenarios: • Minimal-rate sampling and reconstruction (NB) with known band locations (Lin and Vaidyanathan 98) • Half blind system (Herley and Wong 99, Venkataramani and Bresler 00)
Sampling Multi-Coset: Periodic Non-uniform on the Nyquist grid In each block of samples, only are kept, as described by 2 Analog signal 0 Point-wise samples 0 3 3 2 0 3 2 Bresler et. al. (96,98,00,01)
The Sampler in vector form unknowns Length . known matrix known Observation: is sparse DTFT of sampling sequences Constant Problems: • Undetermined system – non unique solution • Continuous set of linear systems is jointly sparse and unique under appropriate parameter selection ( )
Paradigm Solve finiteproblem Reconstruct 0 S = non-zero rows 1 2 3 4 5 6
Continuous to Finite Solve finiteproblem Reconstruct CTF block MMV • span a finite space • Any basis preserves the sparsity Continuous Finite
Algorithm Perfect reconstruction at minimal rate Blind system: band locations are unkown Can be applied to CS of general analog signals Works with other sampling techniques Continuous-to-finite block: Compressed sensing for analog signals CTF
Blind reconstruction flow Multi-coset with Universal SBR4 Yes CTF No SBR2 No Bi-section CTF Yes Uniform at Ideal low-pass filter Spectrum-blind Sampling Spectrum-blind Reconstruction
Final reconstruction (non-blind) Bresler et. al. (96,00)
Framework: Analog Compressed Sensing Sampling signals from a union of shift-invariant spaces (SI) Subspace generators
Framework: Analog Compressed Sensing There is no prior knowledge on the exact indices in the sum What happen if only K<<N sequences are not zero ? Not a subspace ! Only k sequences are non-zero
Framework: Analog Compressed Sensing Step 1: Compress the sampling sequences Step 2: “Push” all operators to analog domain CTF System A High sampling rate = m/TPost-compression Only k sequences are non-zero
Framework: Analog Compressed Sensing Low sampling rate = p/TPre-compression System B CTF Theorem Eldar (2008)
Minimal rate Minimal rate Simulations Sampling rate Sampling rate Brute-Force M-OMP
Simulations (2) 0% Recovery 100% Recovery 0% Recovery 100% Recovery Noise-free Sampling rate Sampling rate SBR4 SBR2 Empirical recovery rate
Simulations (3) Signal Reconstruction filter Amplitude Amplitude Output Time (nSecs) Time (nSecs)
Break (10 min. please)
Uncertainty principles • Coherence and the discrete uncertainty principle • Analog coherence and principles • Achieving the lower coherence bound • Uncertainty principles and sparse representations
The discrete uncertainty principle Uncertainty principle
Discrete coherence Which bases achieve the lowest coherence ?
Discrete coherence Which signal achieves the uncertainty bound ? Spikes Fourier
Discrete to analog • Shift invariant spaces • Sparse representations • Questions: • What is the analog uncertainty principle ? • Which bases has the lowest coherence ? • Which signal achieves the lower uncertainty bound ?
Theorem Eldar (2008) Analog uncertainty principle Theorem Eldar (2008)
Bases with minimal coherence In the DFT domain Fourier Spikes What are the analog counterparts ? • Constant magnitude • Modulation • “Single” component • Shifts
Bases with minimal coherence In the frequency domain
Sparse representations • In discrete setting
Sparse representations • Analog counterparts Undefined program ! But, can be transformed into an IMV model
Discussion • IMV model as a fundamental tool for treating sparse analog signals • Should quantify the DSP complexity of the CTF block • Compare approach with the “analog” model • Building blocks of analog CS framework.