Integration of Simulation Results into Information Systems

1. Integration of Simulation Results into Information Systems Gio Wiederhold April 2002, updated Nov 2002

2. Information Integration Information Integration provides new Information for improved Decision Making when it • presents more data (mediation 1991) • Risk: much may be irrelevant • captures new relationships (knowledge bases 1977) • Often requires expert inter-domain knowledge • includes current sensor data (fusion, streams 1999) • Data reflect the past only • includes predictions about future courses ******* A new, potentially major topic *******

3. Decision-making (DM) Analyze Alternatives • Current Capabilities • Future Expectations Process tasks: • List resources • Enumerate alternatives • Prune alternative • Compare alternatives now future

4. Prediction Requires Tools Ó E-mail this book, Alfred Knopf, 1997

5. o o o o o o Future information systems Combine data from the past, with current data, knowledge, and predictions into the future Assessment of the values of alternative possible outcomes

6. does not interoperate Databases Planning Science Simulation extensions to move to networked support are also disjoint Distribution DM support is disjoint

7. past now future time Intuition + x17 @qbfera ffga 67 .78 jjkl,a nsnd nn 23.5a Data integration Databases distributed, heterogeneous Current state of DM Support organized support disjointed support • Spreadsheets • Planning of allocations • Other simulations • various point assessments

8. past now future time Information Systems should alsoProject into the Futures Support of decision-making requires dealing with the futures, as well the past • Databases deal well with the past • Sensors can provide current status • Spreadsheets, simulations deal with the likely futures Information systems should be able to combine all three

9. New Resources for Information Application Layer Mediation Layer Foundation Layer decision-makers at workstations value-added services data and simulation resources

10. past now future time Msg systems, sensors Databases, accessed via SQL or XML, CORBA compliant wrappers Simulations, accessed via SimQL and compliant wrappers Interfaces enable integration:SimQL to access Simulations

11. Developer Customer Query Development Interaction Production Interaction Parser Help Schema Commands Help Schema Commands Schema Manager Query manager Initiation and Results of Simulations Use of Access Specs Metadata Manager Error reports Filing of Access Specs Wrapped .. Simulations Metadata Prototype Implementation

12. wrapper wrapper wrapper wrapper Engineering Weather (short-, long-term) Spreadsheets Stanford experiment, supported by DARPA & NISTPhase 1 Architectures Logistics Application Manufacturing Application SimQL access SimQL access SimQL access SQL access Test Data

13. Language - simple for demo SQL SELECT Temperature, Cloudcover, Windspeed, Winddirection FROM WeatherDB WHERE Date = `yesterday' AND Location = `ORD'.   ----> {75, .30, 5, NW} SimQL PREDICT Temperature, Cloudcover, Windspeed, Winddirection FROM WeatherSimulation < name of wrapper accessing web pages > WHERE Date = `tomorrow' AND Location = `ORD'. -----> { (75, .8), (.30,.8), (5, .8), (NW, .8) } Would prefer XML representation

14. Simulation results Simulation parameters Place of SimQL in Objective-based Planning Higher Level Objectives, Intel, OB, ROE, Commanders Guidance & Intent, Etc. Campaign Status 1 Execution Feedback Determine Status 2 Develop * Objectives Phased Sequenced Objectives *: w/Measures 3 Phase & * Sequence Objectives Prioritized Sequenced Tasks 4 Assign Task / Activity Assessment Plan 5 Develop Assessment Plan SimQL Access to Simulations Determine Required Resources Req’mts Resource Constraints 7 Assess and/or Rehearse Plan from JFACC PIP Plan Assessment Feedback

15. Types of simulation services 1. Continously executing: weather prediction • SimQL result reports best match samples 2. Execution specific to query: what-if assessment • may require HPC power for adequate response 3. Past simulations collect results in a base: materials • performs inter- or extra-polations to match query parameters 4. Combinations, i.e., 2. + 3.: top layer simulation using stored partial lower level results: device performance in new setting 5. Human-in-the-loop(mediated by an agent program): SAFs Note • A simulation service program can be written in any language • A simulation service must be compliant to the interface spec.

16. Enabling Interoperation • Simulations should • serve clients via SimQL by • Sharing a Model (research q.) • A query language over the model • a SimQL interface enables • independence of • application development • simulation technology develop’t • reuse of infrastructure • Objective • build information systems combining DBMS, Simulations • even with less performance, • inability to handle all problems, • but enough of them . . . Databases • serve clients via SQL by Sharing a Model (The Schema) A query language over the model the SQL interface enables • independence of application development DBMS technology development reuse of infrastructure Today • most new systems use a DBMS for data storage even with less performance, inability to handle all problems, but enough of them well enough.

17. Internet requirements • Ubiquitous access to simulations of a wide variety of types • Rapid response to parameter changes • often High-Performance computation is needed • distributed simulations with synchronization • Rapid Service Composition • High bandwidth among simulations • Access to multiple services in parallel

18. 0.2 0.3 0.6 0.1 0.07 0.03 0.5 0.5 0.3 0.5 0.2 0.1 time 0.2 0.1 0.1 0.4 prob Use of Simulation Results Simulation results can be composed for alternative Courses-of-actions Composition should include computation and recomputation of likelihoods Likelihoods change as now moves forwards and eliminates earlier alternatives.

19. prob value 0.3 0.1 0.4 0.5 0.1 Next period alternatives 0.1 0.2 0.3 0.3 0.2 0.6 0.1 0.07 0.2 0.13 0.4 time The branches can be labeled with probabilities, then assessed using the outcome with values 100 600 1100 500 200 200 -420 0 -820 -400 1000 2000 5000 1000 0 -6000 -3000 Values 1200 66 134 -1220 & subsequent periods 1266 -1086 past now future

20. ? ? 1266 ? time Msgs sensors Spreadsheets, other simulations, Databases, . . . Integrating data & planning support will make our data reusable and much more valuable A Pruned Bush Re-assess as time marches forward ! 1000 2000 5000 1000 0 100 600 1100 500 200 200 0 1200 66 past now future

21. point-in-time for situational assessment Even the present needs SimQL last recorded observations simple simulations to extrapolate data past now future time • Is the delivery truck in X? • Is the right stuff on the truck? • Will the crew be at X? • Will the forces be ready to accept delivery? Not all data are current:

22. wrapper wrapper Civil Engineering Weather Spreadsheets Recent State of SimQL Research GUI collect language requirements Test Application wrapper

23. SimQL research questions • How little of the model needs to be exposed? • How can defaults be set rationally? • How should expected execution cost be reported? • How should uncertainty be reported? • Are there differences among application areas that require different language structures? • Are there differences among application areas that require different language features? • How will the language interface support effective partitioning and distribution?

24. Why not use DB-like storage? • Volume is large -- • multiple future alternatives • Value is transient • tomorrow all values must be updated • expect high write/read ratio • jut the opposite of typical DB operations

25. Research questions for Decision Support • How to move seamlessly from the past to the future? • How can multiple futures be managed (indexed)? • How can multiple futures be compared, selected? • How should joint uncertainty be computed? • How can the NOW point be moved automatically?

26. Research on Multiple futures Uncertainty in databases Probability estimation Expert Systems uncertainty Fuzzy algebra Prade algebra Planning Models allocation and distribution reduction to current values Risk management beta estimation Databases Planning Science Simulation Distribution Interfaces enables research integration

27. Prediction as a Service • Server is an independent contractor, defines service • Client selects service, and specifies parameters • Server’s success depends on value provided • Some form of payment received for services x,y Databases are a current example. Simulations have the same potential.

28. Summary of SimQL A new service for Decision Making: • follows database paradigm • ( by about 25 years ) • coherence in prediction • displacement of ad-hoc practices • seamless information integration • single paradigm for decision makers • simulation industry infrastructure • investment has a potential market • should follows database industry model: Interfaces promote new industries

29. Definitions of Integration • Data Integration: presenting data from multiple sources so that a suite of applications* can deal with it a single (perhaps virtual) database • Information integration: presenting information, obtained by processing data and metadata from multiple sources so that an suite of application can deal with it as a coherent information resource. * application suite:: a set of applications (collect, maintain, query, analyze) that have a consistent domain model • Application integration: Interoperating processes. The ideal of open systems Standards are more difficult, because of hidden semantics

30. Defining Application Integration ? • Application Integration I [Hergula]: include data extracted (via wrappers or transformers) by functions from multiple applications. • Application Integration II: Use a workflow model to integrate processes into a higher level representation. • Application Integration III: Creating new, higher-order (query, ...) functions by combining existing application functions and made-to-order functions (in SQL, ...)