Full-rank Gaussian modeling of convolutive audio mixtures applied to source separation

1. Full-rank Gaussian modeling of convolutive audio mixtures applied to source separation Ngoc Q. K. Duong, Supervisor: R. Gribonval and E. Vincent METISS project team, INRIA, Center de Rennes - Bretagne Atlantique, France Nov. 2010.

2. Table of content • Problem introduction and motivation • Considered framework and contributions • Estimation of model parameters • Conclusion and perspective

3. Under-determined source separation • Use recorded mixture signals to separate sources , where • Convolutive mixing model: • Denotes the source images, i.e. the contribution of a source to all microphones, and the vector of mixture signals • where the vector of mixing filters from source to microphone array

4. Baseline approaches STFT with narrowband approximation Sparsity assumption: only FEW sources are active at each time-frequency point Binary masking (DUET): only ONE source is active at each time-frequency point L1-norm minimization: These techniques remain limited in the realistic reverberant environments since the narrowband approximation does not hold

5. Considered framework Models the STFT coefficients of the source images as zero-mean multivariate Gaussian random variables, i.e. Spatial covariance models Rank-1 model (given by the narrowband assumption): Full-rank unconstrained model: The coefficients of are unrelated a priori I x I spatial covariance matrices encoding spatial position and spatial spread of sources Scalar source variancesencoding spectro-temporal power of sources Most general possible model which allows more flexible modeling the mixing process

6. Considered framework • Source separation can be achieved in two steps: • 1. Model parameters are estimated in the ML sense • - Expectation Maximization (EM) algorithm is well-known as an appropriate choice for this ML estimation of the Gaussian mixing model • 2. Source separation by multichannel Wiener filtering • Raised issues: • - Parameter initialization for EM • - Permutation alignment (well-known in frequency-domain BSS)

7. Proposed algorithm STFT Wiener filtering ISTFT Permutation alignment Initialization by Hierarchical Clustering Model parameter estimation by EM Flow of the BSS algorithms In each step, we adapt the existing methods for the rank-1 model to our proposed full-rank unconstrained model

8. Parameter initialization[S. Winter et al. EURASIP vol.2007] • Principle: perform the hierarchical clustering of the mixture STFT coefficients in each frequency bin after a proper phase and amplitude normalization • Adaptations to our algorithms: • and are computed from the phase normalized STFT coefficients instead of from both phase and amplitude normalized coefficients • We defines the distance between clusters as the average distance between samples instead of the minimum distance between them. • Source variance initialization:

9. EM algorithm • EM for rank-1 model [C. Fevotte and J-F Cardoso, WASPAA2005] • - Mixing model: must consider noise component • Adaptations to the full-rank model • - Apply EM directly to the noiseless mixing model, i.e. • Derive alternating parameter update rule (M-step)by maximizing the likelihood of the complete data

10. Permutation alignment [H. Sawada et al. ICASSP2006 ] Phase of before and after permutation alignment with Principle: permute the source orders base on the estimated source DoAs and the clustered phase-normalized mixing vectors. Adaptation to the full-rank model: Computing the first principal component of by PCA and then applying the algorithm to the “equivalent” mixing vector The order of is permuted identically to that of

11. Experiment setup Source and microphone height: 1.4 m Room dimensions: 4.45 x 3.35 x 2.5 m Microphone distance: d = 0.05 m Reverberation time: 50, 130, 250, 500ms s2 s1 r=0.5m m1 m2 1.8m 1.5m s3 Geometry setting Parameter and program settings

12. Experimental result mixture Full-rank model outperforms both the rank-1 model and baseline approaches in a realistic reverberant environments

13. Conclusion & future work • Contributions • - Proposed to model the convolutive mixing process by full-rankunconstrainedspatial covariance matrices • Designed the model parameter estimation algorithms for the full-rank model by adapting the estimation for rank-1 model • We showed that the proposed algorithm using the full-rank unconstrained spatial covariance model outperforms state-of-the-art approaches. • Current result (in collaboration with S. Arberet and A. Ozerov) • Combined the proposed full-rank unconstrained covariance model with NMF model for source spectra (to appear in ISSPA, May 2010). • Future work • Consider the full-rank unconstrained model in the context of source localization.

14. Thanks for your attention!& Your comments…?