Fast Bayesian Matching Pursuit

1. Reading Group Fast Bayesian Matching Pursuit (Authors: Philip Schniter, Lee C. Potter, and Justin Ziniel) Presenter:Changchun Zhang ECE / CMR Tennessee Technological University November 12, 2010

2. Outline Introduction Signal Model Estimation of Basis and Parameters Numerical Results Conclusions

3. Introduction • Linear regression model • Unknown parameter • Unit columns in the regressor matrix • Additive noise • Rough survey of existing approaches • Greedy approaches • Penalized least-squares solution • In the literature, primary focus is placed on the detection of the few significant entries of the sparse • In contrast, this paper adopts a MMSE (minimum mean-squared error) estimation formulation and focus on accurately inferring x from the noisy observations, y.

4. Signal Model • Observing , - a noisy linear combination of the parameters in : Assumptions: • The noise is assumed to be white Gaussian with variance , i.e., • The columns of , are taken to be unit-norm. • The parameters are generated from Gaussian mixture density: • Covariance is determined by a discrete random vector of mixture parameters. • Also, take to be diagonal with , implying that are independent with . • are Bernoulli( ) and to model spare x, choose and . • N>>M, and

5. Signal Model (continued) From the previous assumptions, it can be seen that

6. Estimation of Basis and Parameters • Basis Selection Metric • The non-zero locations in specify which of the basis elements(i.e., columns of A) are “active”. Thus, basis selection reduces to estimation of . • Estimation of , not only computes the which of the basis configurations are most likely, but also how likely these bases are. The latter is accomplished by the estimation of dominant posteriors while it can be written as below, via Bayes rule. where . The estimation is reduced to computing the

7. Estimation of Basis and Parameters • Basis Selection Metric (continued) • The size of makes it impractical to compute or , the set occupying the dominant posteriors is used for simplicity. • Working on the log domain, it is found that: is refer as basis selection metric.

8. Estimation of Basis and Parameters • MMSE Parameter Estimation The MMSE estimate of x from y is while . Due to 2^N terms for , the MMSE estimate is closely approximately using only the dominant posteriors: Likewise, the covariance of the corresponding estimation error can be closely approximated as The primary challenge becomes that of obtaining and for each belongs to .

9. Estimation of Basis and Parameters • Bayesian Matching Pursuit The efficient means of determining , the set of mixture parameters s yielding the dominant values of , or, equivalently, the dominant values of . Steps: • Start from and turns on one mixture parameters each time, yielding a set of N binary vectors s, i.e, • Compute the metrics for these vectors and collect the D largest metrics into • Based on the , all locations of a second active mixture parms are considered, yielding ND-D*(D+1)/2 unique vectors to store in • In , also collect the D largest elements into • Repeat the process until is obtained satisfying that is very small. • constitutes the final estimate of

10. Estimation of Basis and Parameters • Fast Metric Update A fast metric update which computes the change in u(.) that results from the activation of a single mixture parameter. That is to compute the in which the is the vector identical to except for the coefficient, which is active in but inactive in . Property of :

11. Estimation of Basis and Parameters • Fast Metric Update (continued) This equation quantifies the change in the basis selection metric due to the activation of the tap of . So just computing the based on its nearest , will simplify the computation of basic selection metric.

12. Estimation of Basis and Parameters Fast Bayesian Matching Pursuit Here the complexity of computing can be simplified from to be linear in M by exploiting the structure of . Say that contains the indices of active elements in s. Here where and are the value of and when activating index . Then only need to be computed for surviving indices . This trick makes that the number of multiplications required by the algorithm to become

13. Numerical Results • FBMP behavior • Parameters to be considered • N • M • SNR • p1 • P = where P0 = 0.00005 is the target value of • D • Results: • Figure 1 – Normalized MSE ~ M, D • Figure2 – Average number of missed coeffcients ~ M, D • Figure3 – Normalized MSE ~ Number of active coefficents, D • Figure4 – Normalized MSE ~ SNR, D • Figure5- Normalized MSE ~ SNR, for MMSE and MAP cases • Figure6- Average FBMP runtime ~ D

20. Numerical Results (continued) • Comparison To other Algorithms • Algorithms used comparison • SparseBayes • OMP • StOMP • GPSR-Basic • BCS • Measurements for comparison • Normalized MSE ~ M • Normalized MSE ~ SNR • Average Runtime ~ M

24. Conclusion • Brief Review of the Process • FBMP models each unknown coefficient as either inactive or active (with prior probability p1) • The Gaussian distribution (zero mean and variance )is assigned to the values • Observation y is modeled as an AWGN-corrupted version of the unknown coefficients mixed by a known matrix A • FBMP searches the active/inactive configuration S to find the subset S* with dominant posterior probability. • Parameter D is used to control the tradeoff between complexity and accuracy. • Numerical results show that the FBMP estimation outperform (NMSE) those other popular algorithms by several dB in certain situations.

25. Thank you!