SOM time series clustering and prediction with recurrent neural networks

1. SOM time series clustering and prediction with recurrent neural networks AymenCherif, Hubert Cardot , Romuald Bone 2011, Necurocomputing Presented by Chien-Hao Kung 2011/11/3

2. Outlines • Motivation • Objectives • Methodology • Experiments • Conclusions • Comments

3. Motivation • Local models for regression have been the focus of a great deal of attention in the recent years. • Many models have been proposed to cluster time series and they have been combined with several predictors

4. Objectives This paper presents an extension for recurrent neural networks applied to local models and a discussion about the obtained results.

5. Methodology • From global models to local models • Step1: The time series is embedded into M-dimensional space vectors • Step2:The time series is clustered into sublearning sets. • Step3:Local predictions are performed on each subset.

6. Methodology • Step1: The time series is embedded into • M-dimensional space vectors • Multi-layer perceptron • A multilayer perceptron (MLP) is a feedforward artificial neural network model

7. Methodology • Step2: The time series is clustered into sublearningsets • VQ • VQ was a method used for reducing a large volume of vectors to a smaller number of distribution. • Self-Organizing Maps(SOM) • The SOM has the advantage of being easy to use • However, since the original Self-Organizing Maps algorithm does not take into account the temporal sequence processing.

8. Methodology • Step2: The time series is clustered into sublearningsets • Alternative clustering way • Type1:the use of recurrent processing of time signal with recurrent BMU computation • Temporal Kohonen Map(TKM) • Recursive Self-Organizing Maps(RSOM) • Type2:consists in mapping the temporal dependencies to spatial correlation. • Mege Self-Organizing Maps(MSOM) • The SOM with Temporal Activity Diffusion(SOMTAD)

9. Methodology • Step3:Local predictions • are performed on each subset. • MLP as local predictors • The use of a temporal windows which is precisely the same as the one used in the clustering step. • The feedforward nature of the MLP network • The output calculation and the weights modification are done at the same time step as the learning process.

10. Methodology • Step3:Local predictions • are performed on each subset. • RNN as local predictors • Original RNN • Back-Propagation Through Time(BPTT)

11. Methodology • Step3:Local predictions • are performed on each subset. • RNN as local predictors

12. Methodology • Step3:Local predictions • are performed on each subset. • RNN as local predictors

13. Experiments • Time series • Sunspots time series • Laser time series • The Mackey-Glass(MG)-17

14. Experiments Sunspots time series

15. Experiments • Laser time series

16. Experiments • MG-17 time series

17. Experiments • Experiments on sunspot

18. Experiments • Experiments on Laser time series

19. Experiments • MG-17 time series

20. Conclusions • This paper preferred to use the original SOM algorithm in order to demonstrate the contribution of RNN as a local model. • However, this paper saw that the performance of the model depends on the clustering and also on the nature of the time series.

21. Comments • Drawback • The paper is useful for time series Application • Time sereis