Distributed Compressive Video Sensing

1. Distributed Compressive Video Sensing

2. No errors Vanishing error probabilityfor long sequences Distributed Source Coding [Slepian and Wolf, 1973]

3. “Motion JPEG” Encoder “Motion JPEG” Decoder Side Information Distributed Video Coding Wyner-ZivIntraframe Encoder Wyner-ZivInterframe Decoder X X’ [Girod, 2006]

4. Distributed Video Coding • The statistical dependency between X and Y • Laplacian distribution

5. Compressive Sensing • When data is sparse/compressible, one can directly acquire a condensed representation with no/little information loss • Random projection will work [Baraniuk, 2008]

6. Compressive Sensing • Directly acquire “compressed” data • Replace samples by more general “measurements” [Baraniuk, 2008]

7. Compressive Sensing x = Ψθ • y = Фx = ФΨθ = Aθ y θ Ψ Ф M×1 M×N N×N N×1 A = ФΨ [Baraniuk, 2008]

8. Measurement Matrix • Scrambled block Hadamard ensemble (SBHE) • partial block hadamard transform and random column permutation Ф = QMWPN L. Gan, T. T. Do, and T. D. Tran, “Fast compressive imaging using scrambled hadamard ensemble,” in Proc. of European Signal Processing Conf., Lausanne, Switzerland, August 2008 (EUSIPCO2008).

9. Signal Reconstruction • The convex unconstrained optimization problem • Can be seen as a maximum a posteriori criterion for estimating θ from y = A θ + n, where n is white Gaussian noise

10. Signal Reconstruction • Signal recovery from random measurements • Gradient projection for sparse reconstruction (GPSR) • Two-step iterative shrinkage/thresholding algorithm (TwIST) • Orthogonal matching pursuit (OMP) M. A. T. Figueiredo, R. D. Nowak, and S. J. Wright, “Gradient projection for sparse reconstruction: application to compressed sensing and other inverse problems,” IEEE J. of Selected Topics in Signal Processing, vol. 1,no. 4, pp. 586-597, Dec. 2007. J. M. Bioucas-Dias and M. A. T. Figueiredo, “A new TwIST: two-step iterative shrinkage/thresholding algorithms for image restoration,” IEEE Trans. on Image Processing, vol. 16, no. 12, pp. 2992-3004, Dec. 2007. T. Blumensath and M. E. Davies, “Gradient pursuits,” IEEE Trans. on Signal Processing, vol. 56, June 2008.

11. Distributed Compressive Video Sensing • Measurement matrix Ф: scrambled block Hadamard ensemble (SBHE) • Sparse basis matrix Ψ: DWT • Video signal sensing (encoder): general random projection • Video signal recovery (decoder) • Key frame: GPSR with default settings • CS frame • side information generation (motion compensated interpolation) • GPSR with the proposed initialization and the proposed termination criteria

12. Distributed Compressive Video Sensing Compressive video sensing Video signal recovery

13. Distributed Compressive Video Sensing • At the decoder, for a CS frame xt = Ψθt • its side information St = ΨθSt can be generated from its previous reconstructed key frames • Proposed initialization • initial solution at the 0-th iteration: • α(xt, St): the Laplacian parameter of (xt- St)

14. Side information generation yt-1 = Φxt-1 with higher MR GPSR reconstruction Reconstructed frame (t-1) Key frame (t - 1) Proposed Modified GPSR reconstruction yt = Φxt with lower MR Non-key frame t Side information (t) Reconstructed frame (t) yt+1 = Φxt+1 with higher MR GPSR reconstruction Key frame (t + 1) Reconstructed frame (t+1)

15. Distributed Compressive Video Sensing α( , St) α(xt, ) xt St α(xt, St)

16. Proposed Termination Criterion • First: • Second: • Third:

17. Proposed Termination Criterion • MR is low (MR ≤ 20%): if the First criterion with Tα = 0.9 is satisfied, the algorithm will stop • MR is middle (20% < MR ≤ 70%): if the First criterion with Tα = 0.05 or the Second criterion is satisfied, the algorithm will stop • MR is high (MR > 70%): if the Third criterion with TF = 0.001 is satisfied, the algorithm will stop

18. Simulation Results • Foreman and Coastguard CIF video sequences with 300 Y frames (352×288 = 101376 samples for each Y frame) and GOP size = 3 (Key, Non-key, Non-key, Key, …) • The three approaches for comparison (all with default settings) • GPSR, TwIST, OMP • For OMP, block size = 32×32 suggested by V. Stankovic, L. Stankovic, and S. Cheng, “Compressive video sampling,” in Proc. of European Signal Processing Conf., Lausanne, Switzerland, August 2008 (EUSIPCO2008).

19. Simulation Results

20. Simulation Results

21. Simulation Results The reconstruction complexities for the Foreman sequence

22. Simulation Results The PSNR performance at different reconstruction complexities for the Foreman sequence

23. Simulation Results (a) Side information (b) Reconstructed frame

24. Simulation Results The reconstructed Foreman sequences (352×288 for each frame) at measurement rate (MR) = 0.3 using (a) GPSR (gradient projection for sparse reconstruction) (average PSNR = 27.68 dB) (average reconstruction time = 15.14 seconds per frame); and (b) our DCVS (average PSNR = 29.48 dB) (average reconstruction time = 3.68 seconds per frame) (This example shows the 54-th frame).

25. Conclusions • The proposed DCVS approach exploits the two characteristics • distributed video coding (DVC) • compressive sensing (CS) • The proposed DCVS can outperform or be comparable with the three existing approaches for comparison, especially at lower measurement rates • The proposed DCVS can significant outperform the three existing approaches at the same reconstruction complexity