HUMAN AND SYSTEMS ENGINEERING:

1. HUMAN AND SYSTEMS ENGINEERING: Introduction to Particle Filtering Sanjay Patil1 and Ryan Irwin2 Intelligent Electronics Systems, Human and Systems Engineering Center for Advanced Vehicular Systems URL: www.isip.msstate.edu/publications/seminars/msstate/2005/particle_filtering/

2. Abstract • Particle Filtering: • Most conventional techniques used for speech analysis are based on modeling the speech signal as Gaussian mixture models. • Nonlinear approaches are expected to outperform the conventional techniques because of their abilities to compensate for the mismatched channel conditions and to significantly reduce the complexity of the models. • Particle filtering is one such nonlinear method based on sequential Monte Carlo technique. • Particle filtering works by approximating the target probability distribution. Thus, it greatly reduces the complexities associated with the models.

3. Drawing samples from a probability distribution function • Concept of samples and its weights 200 samples • Take p(x)=Gamma(4,1) • Generate some random samples • Plot basic approximation to pdf • Each sample is called as ‘Particle’ 500 samples 5000 samples

4. Particle filtering - • Condensation Algorithm • Survival of the fittest • Different Names – • Sequential Monte Carlo filters • Bootstrap filters General Problem Statement – Filtering – estimation of the states • Tracking the state (parameters or hidden variables) as it evolves over time • Sequentially arriving (noisy and non-Gaussian) observations • Idea is to have best possible estimate of hidden variables

5. Particle filtering algorithm continue… General two-stage Framework (Prediction-Update stages) • Assume that pdf p(xk-1 | y1:k-1) is available at time k -1. • Prediction stage: • This is the prior of the state at time k ( without the information on measurement). Thus, it is the probability of the the state given only the previous measurements • Update stage: • This is posterior pdf from predicted prior pdf and newly available measurement.

6. Particle filtering algorithm step-by-step (1)

7. Particle filtering step-by-step (2)

8. Particle filtering step-by-step (3)

9. Particle filtering step-by-step (4)

10. Particle filtering step-by-step (5)

11. Particle filtering step-by-step (6)

12. Particle filtering - visualization • Drawing samples • Predicting next state • Updating this state • What is THIS STEP??? • Resampling….

13. Sampling Importance Resample algorithm (necessity)

14. Applications: • All the applications are mostly tracking applications in different forms…. • Visual Tracking – e.g. human motion (body parts) • Prediction of (financial) time series – e.g. mapping gold price, stocks • Quality control in semiconductor industry • Military Applications • Target recognition from single or multiple images • Guidance of missiles • What is the application for IES NSF funded project – • Time series estimation for speech signal (Java demo) • Speaker Verification (details on next slide)

15. Pattern Recognition Applet • Java applet that gives a visual of algorithms implemented at IES • Classification of Signals: • PCA - Principle Component Analysis • LDA - Linear Discrimination Analysis • SVM - Support Vector Machines • RVM - Relevance Vector Machines • Tracking of Signals • LP - Linear Prediction • KF - Kalman Filtering • PF – Particle Filtering

16. Pattern Classification • Different data sets need to be differentiated without looking at all the data samples • Classifications distinguishes between sets of data without the samples • Algorithms separate data sets with a line of discrimination • To have zero error the line of discrimination should completely separate the classes • These patterns are easy to classify

17. Pattern Classification • Toroidals are not classified very successfully with a straight line • Error should be around 50% because half of each class is separated • A proper line of discrimination of a toroidal would be a circle enclosing only the inside set

18. Signal Tracking • The input signals are now time based with the x-axis representing time • All the signal tracking algorithms are implemented with interpolated data • The interpolation ensures that the input samples are at regular intervals • Sampling is always done on regular intervals • The linear prediction algorithm is a linear way to predict signals with no noise

19. Signal Tracking • The Kalman filter and particle filter are based on prediction of the states of the signal • States are related to the observations through the state equation • The particle filtering algorithm introduces process and measurement noise • At each iteration possible states are given by the black points • The average of the black points is where the overall state is predicted to be

20. Summary: • Particle filtering promises to be one of the nonlinear techniques. • More points to follow

21. References: • S. Haykin and E. Moulines, "From Kalman to Particle Filters," IEEE International Conference on Acoustics, Speech, and Signal Processing, Philadelphia, Pennsylvania, USA, March 2005. • M.W. Andrews, "Learning And Inference In Nonlinear State-Space Models," Gatsby Unit for Computational Neuroscience, University College, London, U.K., December 2004. • P.M. Djuric, J.H. Kotecha, J. Zhang, Y. Huang, T. Ghirmai, M. Bugallo, and J. Miguez, "Particle Filtering," IEEE Magazine on Signal Processing, vol 20, no 5, pp. 19-38, September 2003. • N. Arulampalam, S. Maskell, N. Gordan, and T. Clapp, "Tutorial On Particle Filters For Online Nonlinear/ Non-Gaussian Bayesian Tracking," IEEE Transactions on Signal Processing, vol. 50, no. 2, pp. 174-188, February 2002. • R. van der Merve, N. de Freitas, A. Doucet, and E. Wan, "The Unscented Particle Filter," Technical Report CUED/F-INFENG/TR 380, Cambridge University Engineering Department, Cambridge University, U.K., August 2000. • S. Gannot, and M. Moonen, "On The Application Of The Unscented Kalman Filter To Speech Processing," International Workshop on Acoustic Echo and Noise, Kyoto, Japan, pp 27-30, September 2003. • J.P. Norton, and G.V. Veres, "Improvement Of The Particle Filter By Better Choice Of The Predicted Sample Set," 15th IFAC Triennial World Congress, Barcelona, Spain, July 2002. • J. Vermaak, C. Andrieu, A. Doucet, and S.J. Godsill, "Particle Methods For Bayesian Modeling And Enhancement Of Speech Signals," IEEE Transaction on Speech and Audio Processing, vol 10, no. 3, pp 173-185, March 2002.