1 / 182

15-826: Multimedia Databases and Data Mining

This lecture discusses the motivation and applications of time series mining and forecasting, as well as similarity search and indexing, linear forecasting, bursty traffic analysis, and non-linear forecasting.

mccray
Download Presentation

15-826: Multimedia Databases and Data Mining

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 15-826: Multimedia Databasesand Data Mining Lecture #25: Time series mining and forecasting Christos Faloutsos

  2. Must-Read Material • Byong-Kee Yi, Nikolaos D. Sidiropoulos, Theodore Johnson, H.V. Jagadish, Christos Faloutsos and Alex Biliris, Online Data Mining for Co-Evolving Time Sequences, ICDE, Feb 2000. • Chungmin Melvin Chen and Nick Roussopoulos, Adaptive Selectivity Estimation Using Query Feedbacks, SIGMOD 1994 (c) C. Faloutsos, 2013

  3. Deepay Chakrabarti (CMU) Spiros Papadimitriou (CMU) Prof. Byoung-Kee Yi (Pohang U.) Thanks (c) C. Faloutsos, 2013

  4. Outline • Motivation • Similarity search – distance functions • Linear Forecasting • Bursty traffic - fractals and multifractals • Non-linear forecasting • Conclusions (c) C. Faloutsos, 2013

  5. Problem definition • Given: one or more sequences x1 , x2 , … , xt , … (y1, y2, … , yt, … … ) • Find • similar sequences; forecasts • patterns; clusters; outliers (c) C. Faloutsos, 2013

  6. Motivation - Applications • Financial, sales, economic series • Medical • ECGs +; blood pressure etc monitoring • reactions to new drugs • elderly care (c) C. Faloutsos, 2013

  7. Motivation - Applications (cont’d) • ‘Smart house’ • sensors monitor temperature, humidity, air quality • video surveillance (c) C. Faloutsos, 2013

  8. Motivation - Applications (cont’d) • civil/automobile infrastructure • bridge vibrations [Oppenheim+02] • road conditions / traffic monitoring (c) C. Faloutsos, 2013

  9. Motivation - Applications (cont’d) • Weather, environment/anti-pollution • volcano monitoring • air/water pollutant monitoring (c) C. Faloutsos, 2013

  10. Motivation - Applications (cont’d) • Computer systems • ‘Active Disks’ (buffering, prefetching) • web servers (ditto) • network traffic monitoring • ... (c) C. Faloutsos, 2013

  11. Stream Data: Disk accesses #bytes time (c) C. Faloutsos, 2013

  12. Problem #1: Goal: given a signal (e.g.., #packets over time) Find: patterns, periodicities, and/or compress count lynx caught per year (packets per day; temperature per day) year (c) C. Faloutsos, 2013

  13. Problem#2: Forecast Given xt, xt-1, …, forecast xt+1 90 80 70 60 Number of packets sent ?? 50 40 30 20 10 0 1 3 5 7 9 11 Time Tick (c) C. Faloutsos, 2013

  14. Problem#2’: Similarity search E.g.., Find a 3-tick pattern, similar to the last one 90 80 70 60 Number of packets sent ?? 50 40 30 20 10 0 1 3 5 7 9 11 Time Tick (c) C. Faloutsos, 2013

  15. Problem #3: • Given: A set of correlatedtime sequences • Forecast ‘Sent(t)’ (c) C. Faloutsos, 2013

  16. Important observations Patterns, rules, forecasting and similarity indexing are closely related: • To do forecasting, we need • to find patterns/rules • to find similar settings in the past • to find outliers, we need to have forecasts • (outlier = too far away from our forecast) (c) C. Faloutsos, 2013

  17. Outline • Motivation • Similarity Search and Indexing • Linear Forecasting • Bursty traffic - fractals and multifractals • Non-linear forecasting • Conclusions (c) C. Faloutsos, 2013

  18. Outline • Motivation • Similarity search and distance functions • Euclidean • Time-warping • ... (c) C. Faloutsos, 2013

  19. Importance of distance functions Subtle, but absolutely necessary: • A ‘must’ for similarity indexing (-> forecasting) • A ‘must’ for clustering Two major families • Euclidean and Lp norms • Time warping and variations (c) C. Faloutsos, 2013

  20. ... Euclidean and Lp • L1: city-block = Manhattan • L2 = Euclidean • L (c) C. Faloutsos, 2013

  21. Day-n Day-2 ... Day-1 Observation #1 • Time sequence -> n-d vector (c) C. Faloutsos, 2013

  22. Observation #2 Euclidean distance is closely related to cosine similarity dot product ‘cross-correlation’ function Day-n Day-2 ... Day-1 (c) C. Faloutsos, 2013

  23. Time Warping allow accelerations - decelerations (with or w/o penalty) THEN compute the (Euclidean) distance (+ penalty) related to the string-editing distance (c) C. Faloutsos, 2013

  24. Time Warping ‘stutters’: (c) C. Faloutsos, 2013

  25. Time warping Q: how to compute it? A: dynamic programming D( i, j ) = cost to match prefix of length i of first sequence x with prefix of length j of second sequence y (c) C. Faloutsos, 2013

  26. Time warping Thus, with no penalty for stutter, for sequences x1, x2, …, xi,; y1, y2, …, yj no stutter x-stutter y-stutter (c) C. Faloutsos, 2013

  27. Time warping VERY SIMILAR to the string-editing distance no stutter x-stutter y-stutter (c) C. Faloutsos, 2013

  28. Time warping • Complexity: O(M*N) - quadratic on the length of the strings • Many variations (penalty for stutters; limit on the number/percentage of stutters; …) • popular in voice processing [Rabiner + Juang] (c) C. Faloutsos, 2013

  29. Other Distance functions piece-wise linear/flat approx.; compare pieces [Keogh+01] [Faloutsos+97] ‘cepstrum’ (for voice [Rabiner+Juang]) do DFT; take log of amplitude; do DFT again! Allow for small gaps [Agrawal+95] See tutorial by [Gunopulos + Das, SIGMOD01] (c) C. Faloutsos, 2013

  30. Other Distance functions In [Keogh+, KDD’04]: parameter-free, MDL based (c) C. Faloutsos, 2013

  31. Conclusions Prevailing distances: Euclidean and time-warping (c) C. Faloutsos, 2013

  32. Outline • Motivation • Similarity search and distance functions • Linear Forecasting • Bursty traffic - fractals and multifractals • Non-linear forecasting • Conclusions (c) C. Faloutsos, 2013

  33. Linear Forecasting (c) C. Faloutsos, 2013

  34. Forecasting "Prediction is very difficult, especially about the future." - Nils Bohr http://www.hfac.uh.edu/MediaFutures/thoughts.html (c) C. Faloutsos, 2013

  35. Outline • Motivation • ... • Linear Forecasting • Auto-regression: Least Squares; RLS • Co-evolving time sequences • Examples • Conclusions (c) C. Faloutsos, 2013

  36. Reference [Yi+00] Byoung-Kee Yi et al.: Online Data Mining for Co-Evolving Time Sequences, ICDE 2000. (Describes MUSCLES and Recursive Least Squares) (c) C. Faloutsos, 2013

  37. Problem#2: Forecast • Example: give xt-1, xt-2, …, forecast xt 90 80 70 60 Number of packets sent ?? 50 40 30 20 10 0 1 3 5 7 9 11 Time Tick (c) C. Faloutsos, 2013

  38. Forecasting: Preprocessing MANUALLY: remove trends spot periodicities 7 days time time (c) C. Faloutsos, 2013

  39. 90 80 70 ?? 60 50 40 30 20 10 0 1 3 5 7 9 11 Time Tick Problem#2: Forecast • Solution: try to express xt as a linear function of the past: xt-2, xt-2, …, (up to a window of w) Formally: (c) C. Faloutsos, 2013

  40. (Problem: Back-cast; interpolate) • Solution - interpolate: try to express xt as a linear function of the past AND the future: xt+1, xt+2, … xt+wfuture;xt-1, … xt-wpast (up to windows of wpast , wfuture) • EXACTLY the same algo’s ?? 90 80 70 60 50 40 30 20 10 0 1 3 5 7 9 11 Time Tick (c) C. Faloutsos, 2013

  41. Linear Regression: idea 85 Body height 80 75 70 65 60 55 50 45 40 15 25 35 45 Body weight • express what we don’t know (= ‘dependent variable’) • as a linear function of what we know (= ‘indep. variable(s)’) (c) C. Faloutsos, 2013

  42. Linear Auto Regression: (c) C. Faloutsos, 2013

  43. Linear Auto Regression: 85 ‘lag-plot’ 80 75 70 65 Number of packets sent (t) 60 55 50 45 40 15 25 35 45 Number of packets sent (t-1) • lag w=1 • Dependent variable = # of packets sent (S[t]) • Independent variable = # of packets sent (S[t-1]) (c) C. Faloutsos, 2013

  44. Outline • Motivation • ... • Linear Forecasting • Auto-regression: Least Squares; RLS • Co-evolving time sequences • Examples • Conclusions (c) C. Faloutsos, 2013

  45. More details: • Q1: Can it work with window w>1? • A1: YES! xt xt-1 xt-2 (c) C. Faloutsos, 2013

  46. More details: • Q1: Can it work with window w>1? • A1: YES! (we’ll fit a hyper-plane, then!) xt xt-1 xt-2 (c) C. Faloutsos, 2013

  47. More details: • Q1: Can it work with window w>1? • A1: YES! (we’ll fit a hyper-plane, then!) xt xt-1 xt-2 (c) C. Faloutsos, 2013

  48. More details: • Q1: Can it work with window w>1? • A1: YES! The problem becomes: X[N w]a[w 1] = y[N 1] • OVER-CONSTRAINED • a is the vector of the regression coefficients • X has the Nvalues of the w indep. variables • y has the N values of the dependent variable (c) C. Faloutsos, 2013

  49. More details: • X[N w]a[w 1] = y[N 1] Ind-var-w Ind-var1 time (c) C. Faloutsos, 2013

  50. More details: • X[N w]a[w 1] = y[N 1] Ind-var-w Ind-var1 time (c) C. Faloutsos, 2013

More Related