The Role and Limitations of Modeling and Simulation in Systems Design

1. The Role and Limitations of Modeling and Simulation in Systems Design Jason Aughenbaugh & Chris Paredis The Systems Realization Laboratory The George W. Woodruff School of Mechanical Engineering The Georgia Institute of Technology November 19, 2004, Anaheim, CA IMECE2004-5981

2. GenerateAlternatives Evaluate Alternatives Select Alternative Information Knowledge Uncertainty: The Challenge of Design, Modeling, and Simulation Predictions of Consequences of DecisionsAre Always Uncertain Analyze the results Model the alternatives GenericDecisionProcess

3. GenerateAlternatives Evaluate Alternatives Select Alternative Information Knowledge Uncertainty: The Challenge of Design, Modeling, and Simulation Designers currently lack appropriate methods for representing and computing with the various types of uncertainty faced in design especially lack of knowledge GenericDecisionProcess

4. Motivation: Complexity Increasing Need more knowledge Need more collaboration Increasingly complex Increasingly multidisciplinary

5. Systems Engineering: A Decomposition Approach The Vee Model Forsberg, K., and Mooz, H., 1992, "The Relationship of Systems Engineering to the Project Cycle," Engineering Management Journal, 4(3), pp. 36-43. Forsberg, K., Mooz, H., and Cotterman, H., 2000, Visualizing Project Management: A Model for Business and Technical

6. System Decomposition:Relating Requirements and Attributes system Requirements Requirements Attributes Attributes subsystems

7. Relating Requirements and Attributes Must match customer requirements Engineers Decide on system Requirements Requirements Attributes Attributes subsystems Have resultant Engineers Design and Build

8. Making “good” decisions Must match customer requirements ??? Engineering Decisions system Attributes Requirements subsystems Have resultant Engineers Build

9. The Role of Modeling and Simulation Must match Customer requirements Engineers Decide on Modeling and Simulation: estimates system Attributes Requirements subsystems Have resultant Engineers Build

10. system Requirements Attributes Attributes subsystems A system Requirements Attributes Attributes subsystems Decomposition is Hierarchical A

11. system Requirements Attributes Attributes subsystems A system Requirements Attributes Attributes subsystems Decomposition is Hierarchical A How do subsystem decisions affect the system attributes?

12. Aggregation of Subsystem Attributes Resulting from complex emergent behavior e.g. queue wait times Increasingly complex Depending on system operation e.g. reliability Depending on system structure e.g. cost Depending on system composition e.g. mass Increasing value of simulation

13. Specific Uses of Modeling and Simulation Models clarify requirements Models help explore robustness system Requirements Attributes Attributes subsystems Models improve communication Simulations reveal emergent behaviors Now I understand! I didn’t think it would do that! I wanted it to behave more like…

14. Limitations of Modeling and Simulation Limitations of knowledge: uncertainty system Integration of multiple models Requirements Attributes Attributes subsystems Representation and propagation of uncertainty Expressing model validity This is what I know: So how accurate are these numbers? Is he even using the right model?

15. How do we deal with uncertainty? • We need formalisms for • Representing uncertainty accurately • Computing with such formalisms • Making decisions based on these formalisms • We need to accurately express what is known • Capture as much of what is known as necessary • Not imply information that we don’t have • Reflect different types of uncertainty

16. Different Types of Uncertainty • Aleatory uncertainty • Inherently random – irreducible • Best represented as probabilitydistribution • Examples: • Manufacturing variability • Epistemic uncertainty • Due to a lack of knowledge • Not accurately represented as • probability distributions • Examples: • Error due to model approximation • Future design decisions

17. A p-box expresses the range of all possible CDFs that are still deemed possible based on existing knowledge. Example: An enveloping of all possible CDFs for normal distributions with variance of 1 and means in the interval [0,1] It represents aleatory uncertainty (variability) via the normal distributions It represents epistemic uncertainty (incertitude) via the interval on the parameters Possible Handling Mixed Aleatory and Epistemic Uncertainty:Probability Bounds Analysis A “P-box” N(0,1) N(1,1)

18. P-boxes: two dimensions of uncertainty Epistemic Precise Variable Deterministic

19. GenerateAlternatives Evaluate Alternatives Select Alternative Information Knowledge Summary: we need more appropriate representations of uncertainty Predictions of Consequences of DecisionsAre Always Uncertain Analyze the results Model the alternatives GenericDecisionProcess

20. GenerateAlternatives Evaluate Alternatives Select Alternative Information Knowledge Summary: we need more appropriate representations of uncertainty Predictions of Consequences of DecisionsAre Always Uncertain Better Representations Analyze the results Model the alternatives Better Selection Better Design GenericDecisionProcess

21. Acknowledgements • Thank you for attending! • This material is based upon work supported under a National Science Foundation Graduate Research Fellowship. • Any opinions, findings, conclusions or recommendations expressed in this presentation are those of the authors and do not necessarily reflect the views of the National Science Foundation. • Additional support is provided by the G.W. Woodruff School of Mechanical Engineering at Georgia Tech.

22. Questions?