180 likes | 251 Views
Ulrike Fischer. Processing and Optimization of Forecast Queries. Motivation. Time series data appears in many domains. Sales and inventory. Renewable energy ressources. High accuracy possible Sophisticated models Sophisticated estimators. Runtime restrictions
E N D
Ulrike Fischer Processing and Optimization of Forecast Queries
Motivation • Time seriesdataappears in manydomains Salesandinventory Renewableenergyressources • High accuracypossible • Sophisticatedmodels • Sophisticatedestimators • Runtimerestrictions • Large numberof time series • Short amountof time available Two Optimization Dimensions: Accuracy and Runtime Processing and Optimization of Forecast Queries
Outline • Motivation • Integration ofForecastinginside a DBMS • Processing of Forecast Queries • Optimizationof Forecast Queries in Hierarchies • Summary Processing andOptimizationof Forecast Queries
Model-based Time Series Forecasting • 1. Model Creation • Model Identification • Parameter Estimation • 3. Model Maintenance • Model Evaluation • Threshold-based, time-based … • Model Adaption • Parameter Re-estimation • 2. Model Usage • Forecasting Model • Triple Exponential Smoothing ! ! Processing andOptimizationof Forecast Queries
Time Series Forecasting in DBMS Transparency and Effienciency M M export M M SQL export M SQL Reuse of models and results Processing and Optimization of Forecast Queries
Project Overview EU FP7 project Scheduling quantity date 2012 34,000 SELECT date, quantity FROM sales WHERE … FORECAST … Forecasting Aggregation 38,000 2013 … … FlexOffers DWH Supply Demand Processing and Optimization of Forecast Queries
Overview F2DB Forecast Queries Inserts Query Interface Model Usage Model Maintenance Model Index Query Processing & Optimization On-Demand Estimation QP in Hierarchies Hybrid Maintenance PublishSubscribeQueries Model Pool Model Model Model Model M1 Model Creation Time Series Time Series Time Series + := M1 M2 M3 Ensemble Models Base Tables Physical Design AR(2), BFGS, MSE … Processing andOptimizationof Forecast Queries
Outline • Motivation • Integration ofForecastinginside a DBMS • Processing of Forecast Queries • Optimizationof Forecast Queries in Hierarchies • Summary Processing andOptimizationof Forecast Queries
Forecast Query Processing SELECTdate, SUM(quantity) FROMsales WHEREproduct= ‘HTC‘ GROUP BYdate FORECAST 3 • Extension of SQL language • Horizon, measureand time column,model type andparameters, … • Logical query plan • Forecast operatorΨ Physicalquery plan Forecast MHTC Ψk=3 Forecast πdate, quantity BuildModel Aggregate γdate:AGG(sales) Scan σproduct= 'HTC' sales sales Processing andOptimizationof Forecast Queries
Advanced Forecast Query Processing • Data warehousecontains multidimensional data SELECTdate, SUM(quantity) FROMsales WHEREproduct= ‘HTC‘ GROUP BYdate FORECAST 3 days Mobiles Aggregation 3. Disaggregation DisAgg Forecast Forecast 1. Direct MHD2 MSmart Forecast Key Nokia HTC 2. Aggregation MMobiles HD2 Smart Processing andOptimizationof Forecast Queries
Aggregation vs. Disaggregation Top-Down (Disaggregation) Bottom-Up (Aggregation) Complete (Direct) Efficiency Accuracy Model creationeasier Noinformationloss Edwards andOrcuss (1969) Schwarzkopf et. al. (1988) Hubrich (2005) … GrunfeldandGriliches (1960) Grossand Sohl (1990) Zellner and Tobias (2000) …. Depends on data set, quality of forecast model, correlation … Processing and Optimization of Forecast Queries
Outline • Motivation • Integration ofForecastinginside a DBMS • Processing of Forecast Queries • Optimizationof Forecast Queries in Hierarchies • Summary Processing andOptimizationof Forecast Queries
ConfigurationAdvisor Updates Forecast Queries Workload W Preference α • Problem: Exponentialsearchspace • GreedyAlgorithm(monotonicmaintenancecosts) • Start onemodelatthe top, addmodelsstep-by-step Query Interface Model Advisor Analyze Cost BW + Error EW Create Configuration CW DWH Model Pool Configuration + Strategy WeightedAccuracy WeightedEfficiency Processing and Optimization of Forecast Queries
Performance Comparison • Complete (C) All models, onlydirectforecasts • Bottom-Up (B) Onlymodelsatlevelone, othersuseaggregation • Top-Down (T) Onlyonemodelfor top element, othersusedisaggregation • Greedy (G) Processing andOptimizationof Forecast Queries
Extensions • Observation: aggregation(bottom-up) hardlyused in real datasets • Reason: large numberofchild time series • Sample Aggregation • Use sample ofchildmodels • Group Design • Relax fixedaggregationgroups ? ? Virtual Group ? • aggregation + estimation supportofdisjunctivequeries • Estimateusinghistoricalproportion • Weightedsampling Processing and Optimization of Forecast Queries
Outline • Motivation • Integration ofForecastinginside a DBMS • Processing of Forecast Queries • Optimizationof Forecast Queries in Hierarchies • Summary Processing andOptimizationof Forecast Queries
Summary • DBMS Integration • Sophisticatedmodelscomputationally expensive • DBMS integrationforreuse, transparencyandoptimization • Forecast Queries • New query type withforecasthorizon • Face twootimizationdimensions • HierarchicalForecasting • Reducemaintenancecostswithderivationschemes • Possibleincreaseofaccuracy • Large searchspace Processing andOptimizationof Forecast Queries
Ulrike Fischer Processing and Optimization of Forecast Queries