Including human behavior in product simulations for the investigation of use processes in conceptual design A SURVEY

1. Including human behaviorin product simulationsfor the investigation of use processesin conceptual designA SURVEY ASME CIE 2006, Philadelphia Wilfred van der Vegte, Imre Horváth Faculty of Industrial Design Engineering

2. Overview • Objective, scope and method of survey • Assessment criteria for reviewing simulation approaches • Types of artifact and human behavior that can be simulated • Categorization of simulation approaches • Brief overview of reviewed simulation approaches with a focus on ‘unfamiliar’ approaches • Conclusions: what is possible,what is not possible yet and what can be done to make it possible

3. Objective of the survey and the research simulation (definition): performing experiments on virtual models Knowledge exploration in orientation phase of new computer tool development my goal: enable designers to perform simulations with fully virtual systems to simulateuse processes of consumer durables the human involved in theuse process is also virtual: even his/her decision-making is simulated simulation of autonomous virtual humans involved in use processes

4. Scope of the survey MethodLiterature study(for scientific achievements)Web search(for commercial achievements) Knowledge exploration in orientation phase of new computer tool development behavior ofproducts to simulateuse processes of consumer durables behavior ofhumans behavior of artifacts behavior ofsurroundings application field: behavior prediction in conceptual design this survey: existing simulation approaches

5. Criteria for the assessment of simulation approaches • Range of use-process behaviors covered (as many as possible, e.g., cognitive, perceptual, mechanical, thermal, …) • Relevance of the scope (with respect to use of consumer products) • Ease of preparing simulations (e.g., direct conversion from CAD) • Speed and computability (real-time vs. delayed) • Ease of interpretation (animations vs. numeric output) • Fidelity of the outcomes (but accuracy is less important) • Combination options and exchangeability of data (assuming that there is no simulation approach that covers the complete range of behaviors on itself)

6. Types of behavior in artifact simulation stability statics deformation stress and strain solid kinetics motion dynamics collision kinematics vibration mechanics deformation hydrostatics acoustics observed physical behavior buckling hydrodynamics physical behavior optics fluid aerodynamics stress and strain thermodynamics pneumatics M, E electrics physical behavior magnetism continuous simulation I interpreted physical behavior hybrid simulation discrete simulation

7. d d e n v a l o s v t r n a i p s n y a d g o r b o Types of behavior in human simulation interface with flow of information flow of matter flow of energy the outside world f r o o i s cognitive kinetic kine- metabolic perceptual control actuator v e a behavior behavior matical p behavior behavior behavior behavior h y e behavior t b sense bowels, central brain central muscles skin, tendons, organs lungs, nervous nervous subcu- bones, heart, system system taneous joints blood tissue, vessels, etc. etc.

8. d d e n v a l o s v t r n a i p s n y a d g o r b o Types of behavior in human simulation mechanical observed physical behavior non- mechanical observed physical behavior interpreted physical behavior f r o o i s cognitive kinetic kine- metabolic perceptual control actuator v e a behavior behavior matical p behavior behavior behavior behavior h y e behavior t b sense bowels, central brain central muscles skin, tendons, organs lungs, nervous nervous subcu- bones, heart, system system taneous joints blood tissue, vessels, etc. etc.

9. Overview of simulation approaches based on models algebraic descriptions algebraic descriptions qualitative models qualitative models finite state machines finite state machines production rules production rules block diagrams block diagrams bond graphs bond graphs 3D schematic models 3D schematic models guided 3D volumetric models guided 3D volumetric models volu- metric volu- metric (semi) autonomous 3D volumetric models (semi) autonomous 3D volumetric models finite-element models finite-element models boundary-element models boundary-element models mesh-based mesh-based skinning models skinning models discretized discretized particle models particle models Taxonomy of model types Typical simulation models logic-based Control models laws-based Behavioral models algebraic Processing models algorithmbased animationoriented Relationship models logical spatial graphical (2D) Object models abstract schematic (3D) boundary Entity models simplifiedboundary volumetric concrete simplifiedvolumetric

10. Common mechanical artifact-simulation models also applied to human simulation algebraic descriptions algebraic descriptions guided human “simulation” (or guided animation): the user guides the manikin using a UI, so that human decision-making does not have to be simulated. Only fixed sequences of action are simulated / animated UGS block diagrams block diagrams bond graphs bond graphs 3D schematic models 3D schematic models guided 3D volumetric models guided 3D volumetric models Yang et al., 2006 finite-element models finite-element models boundary-element models boundary-element models discretized particle models particle models

11. Simulation models that are not commonly used formechanical artifact simulation housing opened release open housing closed housing pushed down push staple loaded close reload Cognitive Processor Long Term Memory paper stapled staple Production Rule Interpreter Short Term Memory Train_Crossing No_Train Auditory Processor • IF-THEN descriptions of how humansprocess information, mostly based onpsychological research results. • Often grouped into functional modules e.g., EPIC Train-In-Crossing Working Memory Entering_Crossing Leaving_Crossing Visual Processor In_Crossing South_Lights North_Gate South_Gate North_Lights On On On On Ocular Motor Processor Off Off Off Off Vocal Motor Processor Tactile Processor Manual Motor Processor qualitative models qualitative models NaturalMotion Ltd. Okino, inc. finite state machines finite state machines Kieras et al., 1997 production rules production rules Thompson & heimdahl, 1999 Koga & Aoyama, 2004 (semi) autonomous 3D volumetric models (semi) autonomous 3D volumetric models Part of the human’s condition-dependent decision making (high-level or low-level) is included in the model. Depending on circumstances during simulation, different courses of action are possible. Typically based on hybrid models skinning models skinning models

12. Simulation approaches that are the most flexible (algebraic descriptions, qualitative model-based simulation)require the most preparation and interpretation effort for designers (no visual models, no link with CAD models of humans and artifacts, no animated output) Of the other, ‘user-friendly’ simulation approaches, 3 categories of approaches together can cover the most relevant forms of human and artifact behavior: finite state machinesfor interpreted behavior(information processing, cognition) rigid volumetric model-based approaches for rigid-body kinetics and kinematics discretized-model based approaches (e.g., FEM) for mechanical deformations and non-mechanical physical behavior Conclusions (1) IF …

13. Conclusions (2) the human interaction chain is simplified cognitive kinetic kine- HUMAN perceptual control actuator behavior behavior matical behavior behavior behavior behavior sense central brain central muscles skin, tendons, organs nervous nervous subcu- bones, information processing by the senses, the brain andthe central nervous system+ controlled conversion to energy by the muscles system system taneous joints tissue, etc. ARTIFACTS flow of energy IF … flow of information

14. How can the three categories of approaches be combined? e t a m t s a c e Legend: t h i i n n i e f s artifact behavior human behavior interpreted physical behavior (information processing) human & artifact behavior cognitive & control behavior d D i 3 s s v c e d o i r l i d e u g o t i i m b r z e c e t i d r r i t c 3 e D m m u o l d o e v l s mechanical deformations, non-mechanical observed physical behavior mechanical deformations, non-mechanical observed physical behavior rigid-body rigid-body kinetics & kinematics kinetics & with deformations but without geometric kinematics effect of deformation on kinematics rigid-body kinetics rigid-body kinetics & kinematics of & kinematics, body parts considered rigid; deforma- cognition, control tions of body parts considered flexible discrete simulation hybrid simulation continuous simulation

15. A possible solution: Nucleus-based modeling(Horváth, 2004) is a novel discretized modeling approach based on particles, that allows full simultaneous simulation of rigid-body kinetics, kinematics, deformations and other physical behaviors. If we can combine nucleus-based simulation with finite state machines, designers might be able to perform hybrid simulations of observed and interpreted physical behavior of humans and artifacts with little preparation effort,obtaining results that can easily be interpreted. Key bottleneck: combining rigid-body mechanics with deformations &non-mechanical behavior

16. End of my presentation.Questions?