Compressed Sensing

1. Compressed Sensing

2. MobashirMohammad • Aditya Kulkarni • Tobias Bertelsen • Malay Singh • HirakSarkar • NirandikaWanigasekara • Yamilet Serrano Llerena • ParvathySudhir Compressed Sensing

3. Introduction MobashirMohammad

4. The Data Deluge • Sensors: Better… Stronger… Faster… • Challenge: • Exponentially increasing amounts of data • Audio, Image, Video, Weather, … • Global scale acquisition

6. Sensing by Sampling N Sample

7. Sensing by Sampling (2) N >> L L N Sample Compress JPEG … N >> L N L Decompress

8. Compression: Toy Example

9. Discrete Cosine Transformation Transformation

10. Motivation • Why go to so much effort to acquire all the data when most of the what we get will be thrown away? • Cant we just directly measure the part that wont end up being thrown away? Donoho 2004

11. Compressed Sensing • Constructing Φ • Sparse Signal Recovery • Convex Optimization Algorithm • Applications • Summary • Future Work Outline

12. Compressed Sensing Aditya Kulkarni

13. What is compressed sensing? • A paradigm shift that allows for the saving of time and space during the process of signal acquisition, while still allowing near perfect signal recovery when the signal is needed AnalogAudioSignal CompressedSensing Nyquist rateSampling Compression(e.g. MP3) High-rate Low-rate

14. Sparsity • The concept that most signals in our natural world are sparse Original image c. Image reconstructed by discarding the zero coefficients

15. How It Works

16. Dimensionality Reduction Problem • Measure • Construct sensing matrix • Reconstruct

17. Sampling sparse signal measurements • nonzero • entries

18. sparse signal measurements • nonzero • entries

19. nonzero • entries • nonzero • entries

20. Sparsity • The concept that most signals in our natural world are sparse Original image c. Image reconstructed by discarding the zero coefficients

22. Constructing Φ Tobias Bertelsen

23. RIP - RestrictedIsometry Property • The distance between two points are approximately the same in the signal-space and measure-space A matrix satisfies the RIP of order K if there exists a such that: holds for all -sparse vectors and Or equally holds for all 2K-sparse vectors

24. RIP - RestrictedIsometry Property • RIP ensures that measurement error does not blow up Image: http://www.brainshark.com/brainshark/brainshark.net/portal/title.aspx?pid=zCgzXgcEKz0z0

25. Randomized algorithm • Pick a sufficiently high • Fill randomly according to some random distribution • Which distribution? • How to pick ? • What is the probability of satisfying RIP?

26. Sub-Gaussian distribution • Defined by • Tails decay at least as fast as the Gaussian • E.g.: The Gaussian distribution, any bounded distribution • Satisfies the concentration of measure property (not RIP): For any vector and a matrix with sub-Gaussian entries, there exists a such that holds with exponentially high probability where is a constant only dependent on

27. Johnson-Lidenstrauss Lemma • Generalization to a discrete set of vectors • For any vector the magnitude are preserved with: • For all P vectors the magnitudes are preserved with: • To account for this must grow with

28. Generalizing to RIP • RIP: • We want to approximate all -sparse vectors with unit vectors • The space of all -sparse vectors is made up of -dimensional subspaces – one for each position of non-zero entries in • We sample points on the unit-sphere of each subspace

29. Randomized algorithm • Use sub-Gaussian distribution • Pick • Exponentially high probability of RIP • Formal proofs and specific formulas for constants exists

30. Sparse in another base • We assumed the signal itself was sparse • What if the signal is sparse in another base, i.e. is sparse. • must have the RIP • As long as is an orthogonal basis, the random construction works.

31. Characteristics of Random • Stable • Robust to noise, since it satisfies RIP • Universal • Works with any orthogonal basis • Democratic • Any element in has equal importance • Robust to data loss • Other Methods • Random Fourier submatrix • Fast JL transform

32. Sparse Signal Recovery Malay Singh

33. The Hyperplane of

34. Norms for N dimensional vector x Unit Sphere of norm Unit Sphere of quasinorm Unit Sphere of norm Unit Sphere of norm

35. Balls in higher dimensions

36. How about minimization But the problem is non-convex and very hard to solve

37. We do the minimization We are minimizing the Euclidean distance. But the arbitrary angle of hyperplane matters

38. What if we convexify the to

39. Issues with minimization • is non-convex and minimization is potentially very difficult to solve. • We convexify the problem by replacing by . This leads us to Minimization. • Minimizing results in small values in some dimensions but not necessarily zero. • provides a better result because in its solution most of the dimensions are zero.

40. Convex Optimization HirakSarkar

41. What it is all about … • Find a sparse representation • Here and Moreover • Two ways to solve (P1) where is a measure of sparseness(P2)

42. How to chose and • Take the simplest convex function • A simple • Final unconstrained version

43. Versions of the same problem

44. Formalize • Nature of • Convex • Differentiable • Basic Intuition • Take an arbitrary • Calculate • Use the shrinkage operator • Make corrections and iterate

45. Shrinkage operator • We define the shrinkage operator as follows

46. Algorithm Input: Matrix ignal measurement parameter sequence Output: Signal estimate Initialization:

47. Performance • For closed and convex function any the algorithm converges within finite steps • For and a moderate number of iterations needed is less than 5

48. Single Pixel Camera NirandikaWanigasekara

49. Single Pixel Camera • What is a single pixel camera • An optical computer • sequentially measures the • Directly acquires random linear measurements without first collecting the pixel values

50. Single Pixel Camera- Architecture