Lecture 3: Modeling, Simulation, and Prototyping

1. SAE 549: Systems Architecting Lecture 3: Modeling, Simulation, and Prototyping

2. Ways of Studying Systems • Ways to study and build systems in a risk-managed way that satisfy specific requirements • The study and experimentation can take a variety of forms Actual System Physical Prototype Study Analytical Methods Model of System Virtual Prototype Simulation Simulation Math Model

3. A Useful Classification of M&S Static versus Dynamic Deterministic versus Stochastic Continuous versus Discrete Modeling & Simulation

4. Discrete-Event Simulation (DES) • Modeling of a system as it evolves over time using a representation in which the state variables change instantaneously at specific countable number of time points • event: an instantaneous occurrence that may change the state of a system

5. DES Organization • System state • collection of state variables necessary to describe the system at a particular time • Simulation clock • a variable giving the current value of simulated time • Event time • a list containing times at which each event type will occur • Statistical counters • variables that store statistical information about system performance

6. DES Organization (cont’d) • Initialization module • software that initializes the simulation model at time “zero” • Timing routine • software that determines next event from event list and advances simulation clock to time when that event is to occur • Event routine (one per event type) • software that updates the system when a particular type of event occurs • Library routines • software modules used to generate random observations from probability distributions (part of the simulation model)

7. DES Organization (cont’d) • Report generator • software module that uses statistical counter contents to calculate desired measures of performance and produces a report at the end of the simulation run • Simulation controller • software that invokes: • the initialization module • the timing module to determine next event • transfers control to corresponding event module to update system state • checks for termination • invokes report generator at the end of the simulation

8. Parallel Simulation • Concerned with the execution of a simulation model on a tightly coupled computer system (e.g., supercomputer, SMP) • By spreading simulation execution over multiple processors, model execution time can be reduced • A model is decomposed into multiple logical processors (LPs) or submodels • the individual LPs (or groups of them) are assigned to different processors, each goes to work simulating parts of the model • the LPs communicate with each other by sending time-stamped messages or events to each other • a crucial issue is to ensure that events in the overall simulation model are processed in their proper time sequence • each LP can be viewed as a sequential DES model, with its own local state variables, event list, and simulation clock

9. Distributed Simulation • A simulation composed of two or more simulation models located on networked computers • Came into existence as real-time, human-in-the-loop simulation used for training military personnel • e.g. SIMNET (SIMulator NETworking), 1983-1990 • resulted in the creation of a set of protocols for interconnecting simulations—Distributed Interactive Simulation (DIS) standard • next came High Level Architecture (HLA) developed by DoD under leadership of Defense Modeling and Simulation Office

10. High Level Architecture • HLA became the IEEE Standard 1516—a software architectural standard designed to promote reuse and interoperation of simulations • based on the premise that no one simulation could satisfy all uses and applications in defense • HLA can combine the following DoD-defined types of simulation: • live: real people operating real systems (e.g., field exercise) • virtual: real people operating simulated systems (e.g., real people in a tank cockpit simulators fighting simulation-generated opposing forces) • constructive: simulated people operating simulated systems (e.g., discrete-event simulation; wargame)

11. HLA Federation • A collection of interacting individual simulations called federates, a Runtime Infrastructure (RTI), and an interface • RTI provides a set of general-purpose services for federate-to-federate interactions and federation management Simulation Federate Simulation Federate Simulation Federate Interface Runtime Infrastructure (RTI)

12. HLA Characteristics • All interactions among federates go through the RTI • RTI software can be purchased from third party vendors • HLA runtime interface specification provides a standard mechanism for: • federates to interact with the RTI • invoke RTI services • support interactions among federates • respond to RTI requests • The interface is independent of implementation and any federation

13. HLA Characteristics (cont’d) • The interface specification (i.e. runtime services provided by RTI federates and vice versa) • The object model templates (i.e. a standard documentation format) • The rules (i.e. key principles underlying the HLA)

14. Continuous Simulation • The time-based behavior of a system model • the state variables change continuously relative to time • Typically employs differential equations that characterize the rate of change of state variables over time • use numerical techniques (e.g., Runge-Kutta) to integrate differential equations numerically, given initial values for state variables

15. Discrete-Continuous Simulation • For systems that are neither completely discrete of continuous • The three fundamental types of interactions that can occur between discretely changing and continuously changing variables are: • a discrete event may cause a discrete change in the value of a continuous state variable • a discrete event may cause the relationship governing a continuous state variable to change at a particular time • a continuous state variable achieving a threshold value may cause a discrete event to occur as to be scheduled

16. Discrete-Continuous Simulation (cont’d) • Example: • tankers carrying crude oil arrive at a single unloading dock, supplying a storage thank that in turn feeds a refinery through a pipeline (Pritsker, 1995) • unloading tanker delivers oil at a constant rate to storage tank (differential equations) • arrival of tanker, dock closing and opening times (discrete events)

17. Monte Carlo Simulation • Simulation that employs random numbers to solve certain stochastic and deterministic problems • Term originated in World War II when approach was applied to problems related to the development of atomic bomb • Key references: • Hammersley and Hanscomb (1964), Halton (1970), Rubinstein (1981, 1992), Glasserman (2004) • Now widely used to solve certain problems in statistics that are not analytically tractable

18. Spreadsheet Simulation • For problems that are not too complex, discrete-event simulation (DES) and Monte Carlo Simulation can be done using spreadsheets (e.g. Excel) • Excel provides: • a random number generator • the ability to generate random values from basic pdf (e.g. normal, uniform, Poisson) • summary statistics (e.g. μ, σ2) • graphical plots • Products that provide spreadsheet add-ons • @Risk (Palisade Corporation) • Crystal Ball (Decisioneering Inc.)

19. Spreadsheet Simulation (cont’d) • Spreadsheets have the following key limitations (Seila, 2005) • only simple data structures are available • complex algorithms are difficult to implement • limited data storage • longer execution times than sims built using DES packages • Widely used for performing risk analysis in finance, manufacturing, project management, oil and gas discovery • References: • Evans and Olson (2002), Seila et al (2003)

20. Simulation Pros • Only approach for complex stochastic systems that cannot be evaluated analytically using math models • Can estimate performance of both existing/planned systems under different operating conditions • Well-suited for comparative analysis of alternative designs • Enables better control over experimental conditions • Enables study of systems with long time frames (e.g., economic system) in compressed time, or detailed study of system over expanded time

21. Simulation Cons • Produces only estimates of a model’s time characteristics for a particular set of input parameters (need multiple independent runs for each set of input parameters) • Expensive and time-consuming to develop • Can create over-confidence in results for models that do not accurately reflect the system under study

22. Simulation Pitfalls (Law and McComas, 1989) • Failure to have a well-defined set of objectives at the beginning of the simulation study • Failure to have the entire project team involved at the beginning of the study • Inappropriate level of model detail • Failure to communicate with management throughout the course of the simulation study • Misunderstanding of simulation by management • Treating simulation study primarily as exercise in programming

23. Simulation Pitfalls (Law and McComas, 1989) (cont’d) • Failure to have people with knowledge of simulation methodology and statistics on team • Failure to collect good system data • Inappropriate simulation software • Failure to account for sources of randomness in actual system • Using arbitrary distributions as simulation inputs • Erroneously assuming independence in analysis formulae • Using incorrect performance measures • Failure to allow system to reach steady-state behavior

24. Prototyping • Creation of an early or original form of a full-scale model of a structure or a piece of equipment, used in evaluating form, design, fit and performance • Key characteristics include: • a model of a structure or product • used for testing and evaluating form, design fit, performance, and manufacturability • used for both study/analysis and training

25. Virtual Prototype A computer-based simulation of a system or subsystem with a degree of function realism comparable to a physical prototype

26. Virtual Prototyping • The process of simulating the user, the product, and their combined (physical) interaction in software through the different stages of product design, and the quantitative performance analysis of the product (Song et al, 1999) • Virtual prototyping (also called system performance modeling) refers to the design, simulation and testing of new ideas, concepts, products, schemes, or processes in a synthetic but interactive computer environment

27. Realism in Virtual Prototyping • Virtual prototypes can have different degrees of functional realism • The degree of realism is determined by the features or the properties of the target product being modeled and simulated • For example, in training simulators the principle of selective fidelity is employed

28. Virtual Prototyping (A Critique) • Virtual prototyping can potentially cut costs and increase product quality and manufacturing efficiency • Virtual prototyping models (i.e. virtual prototypes) tend to be simplified representations of the real world • an analysis can quickly bring to light design flaws that might otherwise be overlooked • increased accuracy means increased time and cost • These benefits can only be realized if the process of virtual prototyping works efficiently • design and analysis need to be brought together • design outputs fed to visualization/analysis apps • visualization and analysis results fed back to design • speed of iterations is key

29. Systems Focused Virtual Prototyping • Virtual Mockup • digital mockup • no interactivity • potentially supported by CAD/CAE/CAM tools • Virtual Prototype • interactive functionality • potentially supported by SE/CAD/CAE/CAM tools • usability analysis • utility analysis

30. M&S Applications • Systems architecting and engineering • Human-system integration • Test and evaluation • Training

31. M&S in Test and Evaluation • T&E of IT-intensive, complex systems is becoming increasingly difficulty and costly • Live testing has become impractical • too costly • lack of access to other systems and forces • lack of access to other role players when needed • Can no longer afford to wait for physical prototypes to assess designs in terms of functionality, costs, risk, etc. • delays in making fixes/choosing design cost time and money • need to exploit modeling and simulation and virtual prototyping whenever feasible

32. M&S in Test and Evaluation (cont’d) • Increasing system complexity drives T&E complexity and costs • many more interfaces and interactions need to be considered • greater need to assess impacts and unintended consequences • need for continual, frequent testing as subsystems become available incrementally

33. M&S Roles in T&E • Improve systems architecting and engineering • support tradeoffs, eliminate inconsistencies, assess designs using M&S • Enhance planning of live test events • simulate full range of scenarios, actions, and responses • identify critical contexts, capabilities, and interactions to focus interests • Represent and assess impact of changes in system attributes that cannot be examined realistically in live testing • e.g., security issues, environmental impacts, effect of hazards on humans, information operations, civil infrastructure impacts

34. M&S Roles in T&E (cont’d) • Act as surrogates for other systems and effects to cost-effectively explore the battlespace live tests • testing environment comprising composable, federated real systems (e.g., on ranges), data collection assets, and simulations • M&S increasingly becomes key to: • understanding complex system behaviors • focusing live testing for cost-effectiveness • working in “round-trip” fashion with actual testing • conducting upgrades, mishap investigations • changing paradigm from “design, test-fix, test-fix, …” to “design, test through M&S, get it right the first time”

35. Exemplar Modeling Categories Models data-derived representations • cognition • emotion • organization • sociocultural factors • people • objects • physical • phenomenon • vehicles • building • terrain • physical process/flow • (e.g. weather pattern) mathematical representations human-behavioral representations

36. Exemplar Simulation Categories Simulations Live Virtual Constructive • humans • equipment • computer-controlled • setting • e.g., flight simulator • driven by time • (discrete) • humans • equipment • real operational • setting • e.g., wargames • with live soldiers • manning CPs • driven time • (continous) • no human participant • no equipment • computer-controlled events • and physics modeled (e.g. • temperature, pressure, wind • currents, weather) • e.g. science-based sims • (hurricane path construction) • - driven by event sequence

37. Toyota Prius versus Hummer • Prius is the flagship car for environmentally conscious people • People willing to pay a premium to show world they care • Unfortunately, this “green car” is the source of the worst pollution in North America • How?...it takes more combined energy to produce a Prius than a Hummer • Let’s see how ….

38. Analysis • Past government EPA estimates would justify the claim • 60 mpg in city, 51 mpg on highway • However, … these EPA tests were hopelessly unrealistic • limited highway speeds to 55 mph and acceleration of 3.3 mph per second • New tests give a much more realistic rating • highway speeds of 80 mph and acceleration of 8 mph per second • This resulted in dropping Prius’s EPA by 15% to 45 mpg (same as Chevy Aveo which costs less than half of Prius cost)

39. Some Systems Thinking • Prius is partly driven by a battery • The battery contains nickel • Nickel is mined and smelted at a plant in Sudbury, Ontario • The plant has caused such extensive environmental damage that NASA has used the “dead zone” around the plant to test moon rovers! • the area around the plant is devoid of any life for miles • The plant is the source of all nickel found in Prius battery • Toyota purchases 1000 tons annually • Dubbed the Superstore, the factory spreads sulfur dioxide across northern Ontario (every environmentalist’s nightmare) Building a Toyota Prius causes more environmental damage than building a Hummer that is on the road 3x longer than the Prius

40. More Systems Thinking • The nickel produced by the plant is shipped via massive container ship to the largest nickel refinery in Europe • from there, the refined Ni goes to China to produce “nickel foam” • from there, it goes to Japan • finally, the completed batteries are shipped to US completing the around-the-world round trip to produce a single Prius battery • environmentally sound?....no!

41. Smart Questions (that were never asked) • Are there hidden costs? • Can the extended supply chain become a source of pollution and environmental damage? • Are the processes used to produce a hybrid environmentally safe? • Can we inadvertently end up exporting pollution and environmental damage to other countries? • Do our energy calculations take into account the entire supply chain and attendant processes? • Are our assumptions realistic? What is the impact of erroneous assumption X on making the business case? • How would I approach the problem differently using systems thinking?

42. Thank You