1 / 27

Forecasting Engine

Forecasting Engine. Project Members Kaushal Mittal Abhishek Seth Amar Agrawal Guide: Prof. Bernard Menezes Pankaj Gulhane Kalam Shah. Problem Definition.

spiro
Download Presentation

Forecasting Engine

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Forecasting Engine Project Members Kaushal Mittal Abhishek Seth Amar Agrawal Guide: Prof. Bernard Menezes Pankaj Gulhane Kalam Shah

  2. Problem Definition • Develop a forecast engine that interacts with multiple forecasting back-ends such as SAS, MATLAB and user programs to generate useful forecasts for the given input series. • Provide support for using ensemble of experts in order to judge the performance of these experts at different instances of time series.

  3. Client • This project is useful for Academic/Industrial people, who are working on Forecasting of Time Series. • Present Clients • Pankaj Gulhane (MTech’03 KReSIT) • Kalam Shah (MTech’03 KReSIT)

  4. User Requirements • Must be able to talk to multiple backend forecasting systems (SAS, MATLAB). • Facilitate the addition of new systems/experts with minimum effort. • Use an ensemble of experts and combine their results in a meaningful way. • Support adaptive forecasting by using only the values in the past to forecast current observation point.

  5. Challenges • Various forecasting applications have • Different input / output formats • Different scripting language. • Lack of support for adaptive forecasting. • A simple and powerful interface required • Minimum human interactions. • Generic – addition of new forecasting system.

  6. Solutions • Providing a consistent interface to heterogeneous forecasting back ends by • Generating customized scripts for each backend. • Interpretation of different outputs from these back ends • Providing a consistent report to the user. • Abstracting these interface details from the user. • Addition of new experts • Template file support for defining the execution script for experts. • Populating template files through parameter-files to define experts.

  7. Solutions cont.. • Combining opinion of various experts • Analyzing the output from various experts collectively • Find the best n model for a given series. • Find the best n model for each observation • Adaptive forecasting • Writing specific scripts that use only past data to predict the next value. • Use the script repeatedly to get the forecast for each observation.

  8. Usage Scenario • Input • Time series • Ensemble of experts to use for the series. • Output • A report of best ‘k’ models for any of the following two scenarios. • Scenarios • Obtain the best forecasting models from the ensemble and use them to forecast on the series. • Fit multiple experts on the series and obtain the models that give the best predictions for each observation point in the series.

  9. Scenario 1 Sample Report Forecasts using the best 3 models. Summary Series Error Model Name 5.172482640050754 LINEAR_TREND 9.669957978203671 WINTERS_ADDITIVE 11.325417084252539 LINEAR_EXPO ObsId ActualValue ForecastValue Error Value ModelName 1 106922.0 108620.52 1.5885598847758218 LINEAR_TREND 106922.0 109083.75 2.0218009390022633 WINTERS_ADDITIVE 106922.0 124301.56 16.2544284618694 LINEAR_EXPO 2 130621.0 114201.32 12.570474885355337 LINEAR_TREND 130621.0 108651.4 16.81934757810766 WINTERS_ADDITIVE 130621.0 122540.29 6.1863789130384905 LINEAR_EXPO 3 125251.0 124769.27 0.3846116997069851 LINEAR_TREND 125251.0 113045.32 9.74497608801526 WINTERS_ADDITIVE 125251.0 125961.14 0.5669735171775071 LINEAR_EXPO 4 140318.0 130774.79 6.801130289770383 LINEAR_TREND 140318.0 115486.46 17.696617682692168 WINTERS_ADDITIVE 140318.0 127615.91 9.052359640245724 LINEAR_EXPO

  10. Scenario 2 Sample Report Forecasts using 3 best model at each Observation. Summary Series Error Model Name 3.34066242459444 Optimum 1 7.5991216842230385 Optimum 2 15.228073593689482 Optimum 3 ObsId Actual Value Forecast Value Error Value Model Name 1 106922.0 108620.52 1.5885598847758218 LINEAR_TREND 106922.0 109083.75 2.0218009390022633 WINTERS_ADDITIVE 106922.0 124301.56 16.2544284618694 LINEAR_EXPO 2 130621.0 122540.29 6.1863789130384905 LINEAR_EXPO 130621.0 114201.32 12.570474885355337 LINEAR_TREND 130621.0 108651.4 16.81934757810766 WINTERS_ADDITIVE 3 125251.0 124769.27 0.3846116997069851 LINEAR_TREND 125251.0 125961.14 0.5669735171775071 LINEAR_EXPO 125251.0 113045.32 9.74497608801526 WINTERS_ADDITIVE 4 140318.0 130774.79 6.801130289770383 LINEAR_TREND 140318.0 127615.91 9.052359640245724 LINEAR_EXPO 140318.0 115486.46 17.696617682692168 WINTERS_ADDITIVE

  11. Architecture & Design Forecasting Engine Script Executor Script Generator Output Extractor Report Generator SAS Forecast-Analyzer MATLAB

  12. Design Cont.. • Script Generator module • A single code generator for all systems. • Template files - A XML specification for defining the syntax of the application specific input script file. • This template file is used by the ScriptGenerator to generate customized input scripts. • Parameter files – specify the parameters. Code generator use it to customize the template file and generate script. • Main classes – CodeGenerator & ScriptExecutor.

  13. Design cont.. • Output Extractor • Factory pattern used. • Output Extractor extracts output from output files generated by backend systems. • Different for different forecasting systems. • User can add his own output extractor through this factory. • ModelMetrics - Store the results for each Expert. • Main Classes – OutputExtractorFactory, SASOutputExtractor, MatlabOutPutExtractor

  14. Use Case • How the users can use the system? • Use command line to give a list of experts (property file) and general arguments, system generates the report. • User writes his own java program using our class API’s to interact with backend systems. • Provides power to exploit the design to meet new emerging requirements. • Use the output generated for further processing.

  15. User Interface::Command Line • Java stats.metric.ForecastEngine [options] • Mandatory options • -I <input file> ( data series) • -p <property file> ( list of experts to be used) • -r <report file> • System generates two reports <report file>.best and <report file>.opt • Optional parameters • -t <count> number of best experts in report. (3) • -s <count> forecasting starts from this observation (31) • -f <count> forecast upto this observation. (31) • -a flag actual values displayed or not . <enabled>

  16. User Interface::Java Program • // create the engine • ForecastEngine engine = new ForecastEngine(); • // get the input parameters • Properties prop = new Properties(); • prop.load(new FileInputStream(<Expert File>)); • HashMap inputParameters = new HashMap(prop); • inputParameters.put(ParameterConstants.BEGIN,"31"); • inputParameters.put(ParameterConstants.END,"32"); • inputParameters.put(ParameterConstants.INCR,"1"); • inputParameters.put(ParameterConstants.INPUTFILE,“<Time Series File>); • // set the input parameter • engine.setInputParameter(inputParameters

  17. User Interface::Java Program • // execute the experts and obtain the results. • engine.execute(); • // Accessing result : Method 1 • ForecastAnalyzer analyser = engine.getForecastAnalyzer(); • ModelMetrics output[] = analyser.getOptimumForcast(ErrorMeasures.APE, 2); • // Method 2 generate the report • ReportFormat format = new ReportFormat(); • format.setFormat( ReportFormat.FORMAT_BESTK ); • AbstractReportGenerator gen = engine.getDefaultReportGenerator(); • gen.setReportFile("c:/Myoutput.txt"); • gen.generateReport( format );

  18. Adding a new System • Write a XML template file for the scripts for the system. • Modify the module-definition.xml to add the node for the new template file • Write a parameter file to define the forecasting expert • Add the entry for the expertfile in the list of experts property file. • If the output format is different from forecasting system already supported • Create a new OutputExtractor implementing the interface provided. • Write a custom forecast Engine.

  19. Custom ForecastEngine MyForecastEngine extends ForecastEngine{ ForecastEngine(args[]) { Super(); OutputExtractorFactory.add(MyoutputExtractor ob); } public static void main(){ ForecastingEngine = new forecastingEngine(args[]); engine.execute(); }

  20. Time Effort Distribution • Study and Research • Study of basics of Time Series Analysis. • Studying the ETS package in SAS • Studying the Neural network package for MATLAB • Exploring for scripting support for AUTOFIT in SAS • Exploring for inbuilt support for adaptive forecasting • Designing scripts for Adaptive forecasting. • Study of language reference for MATLAB and SAS. • Design and implementation • Design has evolved from prototypical level to sophisticated level. • Prototype was built to test how a system can interact with multiple forecasting systems.

  21. Insights Gained • Familiarity to forecasting systems like SAS and MATLAB • Generic designing using OOP • XML usage for defining small scripting requirements. • Using the best experts from a large ensemble leads to better forecasts. • Adaptive forecasting allows forecasting engine to improve with each new observation. • Analysis of whether some forecasting model performs better for some part of the series may be used to forecast better.

  22. Demo • Best K models • Optimal K models

  23. Demo cont. • Adding a forecasting backend in our application. (JAVA backend) • Write a sample java program • To execute forecasting backend. • Make all parameters “hard-coded” (E.g. input file, output file, other parameter). • Write an xml file (say testforcast.xml) corresponding to this sample java file • Replace all parameters with variables. • Store it in resources/templates/

  24. Demo cont. • Add entry of testforcast.xml file in script.xml file. • Write an expert file (testjava.exp) for this new backend. • Add entry in listofExperts.txt • Make a script • To compile and execute the java script. • Job Done.

  25. Future Enhancement • For user • Incorporating more backend/expert systems to make the forecasting engine more robust. • Use SAS 9 which has more options. • For Developer • Incorporate distributed execution support.

  26. Assumptions & Limitations • Due to generic behavior certain assumptions and restrictions are imposed. • Forecasting system should take the input data in a file. • It should generate the output in the file. • Currently multiple forecasting systems must exist on a single system to use them in combination.

  27. References • Time Series Analysis by Box , Jenkins , Reinsel. • Java Online References • SAS online help • ETS Package • Language Reference • Matlab User Guide • Neural Network toolbox manual • Scripting Language

More Related