1.56k likes | 1.58k Views
Explore time series mining and forecasting applications across multimedia databases, covering similarity search, linear and non-linear forecasting, and practical examples in various fields.
E N D
15-826: Multimedia Databasesand Data Mining Lecture #27: Time series mining and forecasting Christos Faloutsos
Must-Read Material • Byong-Kee Yi, Nikolaos D. Sidiropoulos, Theodore Johnson, H.V. Jagadish, Christos Faloutsos and Alex Biliris, Online Data Mining for Co-Evolving Time Sequences, ICDE, Feb 2000. • Chungmin Melvin Chen and Nick Roussopoulos, Adaptive Selectivity Estimation Using Query Feedbacks, SIGMOD 1994 (c) C. Faloutsos, 2017
Deepay Chakrabarti (UT-Austin) Spiros Papadimitriou (Rutgers) Prof. Byoung-Kee Yi (Samsung) Thanks (c) C. Faloutsos, 2017
Outline • Motivation • Similarity search – distance functions • Linear Forecasting • Bursty traffic - fractals and multifractals • Non-linear forecasting • Conclusions (c) C. Faloutsos, 2017
Problem definition • Given: one or more sequences x1 , x2 , … , xt , … (y1, y2, … , yt, … … ) • Find • similar sequences; forecasts • patterns; clusters; outliers (c) C. Faloutsos, 2017
Motivation - Applications • Financial, sales, economic series • Medical • ECGs +; blood pressure etc monitoring • reactions to new drugs • elderly care (c) C. Faloutsos, 2017
Motivation - Applications (cont’d) • ‘Smart house’ • sensors monitor temperature, humidity, air quality • video surveillance (c) C. Faloutsos, 2017
Motivation - Applications (cont’d) • civil/automobile infrastructure • bridge vibrations [Oppenheim+02] • road conditions / traffic monitoring (c) C. Faloutsos, 2017
Motivation - Applications (cont’d) • Weather, environment/anti-pollution • volcano monitoring • air/water pollutant monitoring (c) C. Faloutsos, 2017
Motivation - Applications (cont’d) • Computer systems • ‘Active Disks’ (buffering, prefetching) • web servers (ditto) • network traffic monitoring • ... (c) C. Faloutsos, 2017
Stream Data: Disk accesses #bytes time (c) C. Faloutsos, 2017
Problem #1: Goal: given a signal (e.g.., #packets over time) Find: patterns, periodicities, and/or compress count lynx caught per year (packets per day; temperature per day) year (c) C. Faloutsos, 2017
Problem#2: Forecast Given xt, xt-1, …, forecast xt+1 90 80 70 60 Number of packets sent ?? 50 40 30 20 10 0 1 3 5 7 9 11 Time Tick (c) C. Faloutsos, 2017
Problem#2’: Similarity search E.g.., Find a 3-tick pattern, similar to the last one 90 80 70 60 Number of packets sent ?? 50 40 30 20 10 0 1 3 5 7 9 11 Time Tick (c) C. Faloutsos, 2017
Problem #3: • Given: A set of correlatedtime sequences • Forecast ‘Sent(t)’ (c) C. Faloutsos, 2017
Important observations Patterns, rules, forecasting and similarity indexing are closely related: • To do forecasting, we need • to find patterns/rules • to find similar settings in the past • to find outliers, we need to have forecasts • (outlier = too far away from our forecast) (c) C. Faloutsos, 2017
Outline • Motivation • Similarity Search and Indexing • Linear Forecasting • Bursty traffic - fractals and multifractals • Non-linear forecasting • Conclusions (c) C. Faloutsos, 2017
Outline • Motivation • Similarity search and distance functions • Euclidean • Time-warping • ... (c) C. Faloutsos, 2017
Importance of distance functions Subtle, but absolutely necessary: • A ‘must’ for similarity indexing (-> forecasting) • A ‘must’ for clustering Two major families • Euclidean and Lp norms • Time warping and variations (c) C. Faloutsos, 2017
... Euclidean and Lp • L1: city-block = Manhattan • L2 = Euclidean • L (c) C. Faloutsos, 2017
Day-n Day-2 ... Day-1 Observation #1 • Time sequence -> n-d vector (c) C. Faloutsos, 2017
Observation #2 Euclidean distance is closely related to cosine similarity dot product ‘cross-correlation’ function Day-n Day-2 ... Day-1 (c) C. Faloutsos, 2017
Time Warping allow accelerations - decelerations (with or w/o penalty) THEN compute the (Euclidean) distance (+ penalty) related to the string-editing distance (c) C. Faloutsos, 2017
Time Warping ‘stutters’: (c) C. Faloutsos, 2017
Time warping Q: how to compute it? A: dynamic programming D( i, j ) = cost to match prefix of length i of first sequence x with prefix of length j of second sequence y (c) C. Faloutsos, 2017
Full-text scanning • Approximate matching - string editing distance: d( ‘survey’, ‘surgery’) = 2 = min # of insertions, deletions, substitutions to transform the first string into the second SURVEY SURGERY (c) C. Faloutsos, 2017
Time warping Thus, with no penalty for stutter, for sequences x1, x2, …, xi,; y1, y2, …, yj no stutter x-stutter y-stutter (c) C. Faloutsos, 2017
Time warping VERY SIMILAR to the string-editing distance no stutter x-stutter y-stutter (c) C. Faloutsos, 2017
Full-text scanning if s[i] = t[j] then cost( i, j ) = cost(i-1, j-1) else cost(i, j ) = min ( 1 + cost(i, j-1) // deletion 1 + cost(i-1, j-1) // substitution 1 + cost(i-1, j) // insertion ) (c) C. Faloutsos, 2017
Time warping VERY SIMILAR to the string-editing distance Time-warping String editing 1 + cost(i-1, j-1) // sub. 1 + cost(i, j-1) // del. 1 + cost(i-1, j) // ins. cost(i,j) = min (c) C. Faloutsos, 2017
Time warping • Complexity: O(M*N) - quadratic on the length of the strings • Many variations (penalty for stutters; limit on the number/percentage of stutters; …) • popular in voice processing [Rabiner + Juang] (c) C. Faloutsos, 2017
Other Distance functions piece-wise linear/flat approx.; compare pieces [Keogh+01] [Faloutsos+97] ‘cepstrum’ (for voice [Rabiner+Juang]) do DFT; take log of amplitude; do DFT again! Allow for small gaps [Agrawal+95] See tutorial by [Gunopulos + Das, SIGMOD01] (c) C. Faloutsos, 2017
Other Distance functions In [Keogh+, KDD’04]: parameter-free, MDL based (c) C. Faloutsos, 2017
Conclusions Prevailing distances: Euclidean and time-warping (c) C. Faloutsos, 2017
Outline • Motivation • Similarity search and distance functions • Linear Forecasting • Bursty traffic - fractals and multifractals • Non-linear forecasting • Conclusions (c) C. Faloutsos, 2017
Linear Forecasting (c) C. Faloutsos, 2017
Forecasting "Prediction is very difficult, especially about the future." - Nils Bohr http://www.hfac.uh.edu/MediaFutures/thoughts.html (c) C. Faloutsos, 2017
Outline • Motivation • ... • Linear Forecasting • Auto-regression: Least Squares; RLS • Co-evolving time sequences • Examples • Conclusions (c) C. Faloutsos, 2017
Reference [Yi+00] Byoung-Kee Yi et al.: Online Data Mining for Co-Evolving Time Sequences, ICDE 2000. (Describes MUSCLES and Recursive Least Squares) (c) C. Faloutsos, 2017
Problem#2: Forecast • Example: give xt-1, xt-2, …, forecast xt 90 80 70 60 Number of packets sent ?? 50 40 30 20 10 0 1 3 5 7 9 11 Time Tick (c) C. Faloutsos, 2017
Forecasting: Preprocessing MANUALLY: remove trends spot periodicities 7 days time time (c) C. Faloutsos, 2017
90 80 70 ?? 60 50 40 30 20 10 0 1 3 5 7 9 11 Time Tick Problem#2: Forecast • Solution: try to express xt as a linear function of the past: xt-2, xt-2, …, (up to a window of w) Formally: (c) C. Faloutsos, 2017
(Problem: Back-cast; interpolate) • Solution - interpolate: try to express xt as a linear function of the past AND the future: xt+1, xt+2, … xt+wfuture;xt-1, … xt-wpast (up to windows of wpast , wfuture) • EXACTLY the same algo’s ?? 90 80 70 60 50 40 30 20 10 0 1 3 5 7 9 11 Time Tick (c) C. Faloutsos, 2017
Linear Regression: idea 85 Body height 80 75 70 65 60 55 50 45 40 15 25 35 45 Body weight • express what we don’t know (= ‘dependent variable’) • as a linear function of what we know (= ‘indep. variable(s)’) (c) C. Faloutsos, 2017
Linear Auto Regression: (c) C. Faloutsos, 2017
Linear Auto Regression: 85 ‘lag-plot’ 80 75 70 65 Number of packets sent (t) 60 55 50 45 40 15 25 35 45 Number of packets sent (t-1) • lag w=1 • Dependent variable = # of packets sent (S[t]) • Independent variable = # of packets sent (S[t-1]) (c) C. Faloutsos, 2017
Outline • Motivation • ... • Linear Forecasting • Auto-regression: Least Squares; RLS • Co-evolving time sequences • Examples • Conclusions (c) C. Faloutsos, 2017
More details: • Q1: Can it work with window w>1? • A1: YES! xt xt-1 xt-2 (c) C. Faloutsos, 2017
More details: • Q1: Can it work with window w>1? • A1: YES! (we’ll fit a hyper-plane, then!) xt xt-1 xt-2 (c) C. Faloutsos, 2017
More details: • Q1: Can it work with window w>1? • A1: YES! (we’ll fit a hyper-plane, then!) xt xt-1 xt-2 (c) C. Faloutsos, 2017