Sparse Signal Reconstruction Using FOCUSS

1. Sparse Signal Reconstruction Using FOCUSS Yan Chen

2. References • Irina F. Gorodnitsky and Bhaskar D. Rao, “Sparse Signal Reconstruction from Limited Data Using FOCUSS: A Re-weighted Minimum Norm Algorithm”, IEEE Trans. On Signal Processing, 1997. • Hong Jung and Jong Chul Ye, “Improved k-t BLAST and k-t SENSE using FOCUSS”, Phys. Med. Biol., 2007. • Michael Lustig, David L. Donoho, Juan M. Santos, and John M. Pauly, “Compressed Sensing MRI”, IEEE Signal Processing Magazine, 2008.

3. Outline • Problem Formulation • Uniqueness Conditions for Sparse Solutions • FOCUSS • FOCUSS vs. CS: similarities and differences • Conclusions

4. Problem Formulation • The sparse signal reconstruction problem can be formulated as finding the sparse signal such that where is a mxn (m<n) matrix, is the observed data. • There are infinitely many solutions. And the infinite set of solutions can be expressed as: where is the minimum norm solution, and is any vector in the null space of

5. Unique Representation Property (URP) • A system is said to have the URP if any m columns of the transform matrix are linearly independent.

6. Uniqueness Conditions for Sparse Solutions • Theorem 1: Given a linear system satisfying the URP, which has a k<m/2-dimensional solution, there can be no other solution with dimension less than r=m-k+1.

7. Uniqueness Conditions for Sparse Solutions • Corollary 1: A linear system satisfying the URP can have at most one solution of dimension less than m/2. This solution is the maximally sparse solution.

8. Uniqueness Conditions for Sparse Solutions • Corollary 2: For the system satisfying the URP, the real signal can always be found as the unique maximally sparse solution when the number of the data samples m exceeds the signal dimension by a factor of 2. In this case, if a solution with dimension less than m/2 is found, it is guaranteed to represent the real signal.

9. Minimum norm solution • The minimum norm solution is unique and found by assuming the minimum L2-norm on the solution. It can be computed by: where is the pseudo-inverse of . But it does not provide sparse solutions.

10. L1-norm Minimization • To produce sparse reconstruction, L1-norm minimization is used: • If is in the gradient domain, the object function is the Total Variation (TV) norm. Even if is not in the gradient domain, it is often useful to include a TV penalty as well, which seek sparsity both in the transform domain and the gradient domain. • However, cartoon-like artifacts would be introduced!!

11. FOCUSS: FOcal Underdetermined System Solver (1) • Instead of directly minimizing L2-norm of , let us consider the following optimization problem: where is the solution of the following constrained minimization problem: • The unique solution is given by: The novelty of FOCUSS is that the weighting matrix is updated using the previous solution.

12. FOCUSS: FOcal Underdetermined System Solver (2) • More specifically, if the (n-1)-th iteration of the estimate is given by: • Then, the n-th iteration of FOCUSS can be calculated:

13. FOCUSS: FOcal Underdetermined System Solver (3) • FOCUSS starts by finding a non-sparse low resolution estimate to initialize the weighting matrix. • Then, the solution is pruned to a sparse signal by scaling the entries of the current solution by those of the solution of previous iterations. • Once some entries of the previous solution become zero, these entries are fixed to zero. Therefore, a sparser solution can be found with more iterations.

14. FOCUSS vs. CS • Consider the n-th FOCUSS estimate: • Set p=0.5, then: The FOCUSS solution is asymptotically equivalent to the L1-minimization solution when p=0.5.

15. FOCUSS vs. CS • L1-norm minimization • Sparseness constraint is explicitly involved in the cost function. • No initial estimate. • Annoying visual artifacts may be introduced due to the hard sparseness constraint, especially when TV regularization is used. • L2-norm minimization • Sparseness constraint is implicitly involved in the cost function. • Initial estimate is needed. • Visual quality is good since non-zero values are gradually suppressed. More: complexity, training data (prior information), the number of samples.

16. Conclusions • We introduced a sparse signal reconstruction method-FOCUSS. • We compared FOCUSS with CS, and showed the similarities and differences between them. • FOCUSS is a nice fit to the applications such as dynamic MRI, where a good initial estimate is available.

17. Thanks! • Questions?