FROM SIMULATION TO VR BASED SIMULATION AND TESTING

1. FROM SIMULATION TO VR BASED SIMULATION AND TESTING International Journal of Systems Science

2. Dr. Pavel G. Ikonomov • Pavel.Ikonomov@wmich.edu • Tel: 269-276-3284 • Emiliya Milkova • Emiliya@hotmail.com • Tel: 269-329-3019 International Journal of Systems Science

3. Interactive Computer simulation system Real World Computer Interface Part A Nominal Shape Manufacturing tolerance simulation Properties Designer Manufacturing and Assembly tolerance simulation Tolerance Computer Statistical Tolerance Output Part A Nominal Shape Part B Manufacturing tolerance simulation Part B Input Properties Tolerance Statistical Tolerance Data flow Keyboard Manufacturability and assembly tolerance restrictions Manufacturing and Assembly Data base Work interactively International Journal of Systems Science

4. VR simulation International Journal of Systems Science

5. Virtual Reality Simulation • Virtual reality simulation/prototyping provides complete description: not only of the geometrical, but also the functional capabilities of the future product and process before the actual product is delivered. • Allows external user to test and give comments for product improvement • Virtual reality technology allows instant examination of the accuracy of the design concept early in the development cycle before even the physical model is produced. • VR system can be used for training of the personal before the real process is delivered. • The initial cost of virtual product is much lower than the real product • VRA Implementation is based on real process International Journal of Systems Science

6. Virtual Reality simulation has matured Virtual Reality simulation has matured from high-end supercomputers and special designed interface devices to mainstream daily computing. Electronic Visualization Laboratory's (EVL) development of virtual reality (VR) devices, software libraries/toolkits and applications for collaborative exploration of data over national and global high-speed networks - called "tele-immersion." CAVE became a symbol of high-end VR, high equipment cost and development lead to limited application to small group of research labs. Washington University Virtual assembly project VADE has some advance feature like CAD data import/export, natural hand interaction, and physical based simulation. Problem: High-end computer requirements and difficult to use interface has not lead to wide acceptance of this advance VR system. Why? Cost factor, User Interface, VR simulation design and modeling complexity International Journal of Systems Science

7. Product (beta) testing in software engineering • Software designer: extensive testing of their design product not only within their company, but also by getting feedback from the end-user. • So called beta testing is well-established practice among almost all software companies before they release new software to find bugs, test usability problems, and judge about needed features, new or existing. • Alpha version or alpha release is the first release of a computer program or other product, likely to be very unstable but useful for demonstrating internally and to select customers. It is also called "preview” or “technical preview'' • A beta version, intermediate step of the full development cycle, is not fully debugged or fully functional but satisfies a majority of the requirements. • When a beta is released to the general public it often becomes used almost as widely as the finished product International Journal of Systems Science

8. Why there is need for product simulation and Beta testing (BT) The beta testing for simulation purpose can be used well in advance of the developing of the hardware and software products to employed different options and engineering solution that lead to design optimization before the real products is ready. For example: Companies: to understand where technology is heading, which products appear promising, and--equally important--which software isn't worthy of deployment. Physical science: phenomenological and conceptual model evidence to help them develop robust and valid ideas. To represent their predictions regarding the outcomes of experiments (either hands-on or virtual). International Journal of Systems Science

9. VHDL VHDL: originally developed at the behest of the US Department of Defense in order to document the behavior of the ASICs that supplier companies were including in equipment. Because of this general-purpose nature, it is possible to use VHDL to write a testbench that verifies the functionality of the design, using files on the host computer to define stimulus, interacts with the user, and compares results with those expected. Design simulation can include motion simulation and FEA to test different products from aircraft to micro-miniature switches. For example, according to CAE engineer from Big Three automakers, they cut the cost at least in half using design simulation, optical switcher supplier SiWave Inc. cuts empirical testing by an order of magnitude. Using virtual prototyping, John Deer reduces number of prototypes and saves time; from one to six months for each prototype International Journal of Systems Science

10. What is missing from Virtual reality simulation While 3-D visualization is used to fly trough and play role for design product description and acceptance, it is still only mere 3-D visual presentation with limit interaction or animation missing the way real machines, objects, human and environmental operate together. VR-based applications are used to enable personnel to control and optimize the efficiency of manufacturing systems at different stages of the systems life cycle in real-time. Designers are able to integrate new core concepts within the design process. Usability is a therefore a major issue. The interface provides functions for machine operation, inspection, failure detection and diagnosis as well as cooperative team training Virtual reality system uses natural human interaction for control of the design products. It works in a matter similar to Beta testing and VHDL simulation, it allows not only testing of a new product, better interaction, and experience by user while a the same time allowing feedback of data to the designer. International Journal of Systems Science

11. Product design testing framework Feedback Problems& needs Factory Based Beta Testing Design Product (DP) CAD present. VR beta testing of DP Manuf. simulation VR beta testing of DP Assembly simulation Change DP to VR Simulation Mobile wearable computer VR beta testing of DP testing simulation Multi-modal VR representation of DP VR representation of Design Product VR beta testing of DP Marketing simulation User Based Beta Testing VR beta testing of DP for Sales simulation Automatic log of VR simulation Automatic log of VR simulation VR beta testing of DP Maintenance simulation VR beta testing of DP End-user simulation Feedback problems and needs International Journal of Systems Science

12. Using Virtual Reality Simulation Trough Product Lifecycle Multi-modal VR representation CAVE HMD Desktop Tablet Mobile PDA For each application case, specific and visual and peripheral interface can be build and a feedback module to log information back to the designed can be added. While preserving the main VR beta testing capabilities it adds new dimension to the way user interacts. International Journal of Systems Science

13. Virtual Reality? Interactive Real time Realism Intuitive • 3-D-The ”reality” simulated in a VR system is presented in three dimensions to resemble the real world • Graphics-Most VR systems are based on computer generated imagery • Open GL, DirectX • Real-time • Images generated by the • computer are ideally updated in • real-time, about 30/60 times per • second (30/60 Hz). • Also, the response from the • simulation to the user’s actions is • immediate. • Virtual Reality, ”VR” • Describes the ”sensation” of being immersed in a virtual environment • Heavily ”hyped” by media: ”entering cyberspace,” etc. • Virtual Reality does not have to be identical to ”reality”, not even come close International Journal of Systems Science

14. From interactive PC work to immerse environment The designer works interactively with the computer to design parts in accordance with manufacture and assembly restriction and tolerance requirements. When those requirements are met redesign in accordance with manufacturing and assembly tolerances are applied instead. International Journal of Systems Science

15. New level of Interactive Visual Simulation • Interactive -User interaction (hands grabbing, movement, press of a button, point-and-click) affects the simulation • Visual-The simulation is presented to the user as a sequence of 3Dimages • Simulation-3D Models of existing or imaginative environments are simulated using a computer International Journal of Systems Science

16. Virtual reality application in industry • Visualization • Visualize kinematics • Virtual plants • Interaction • Ease of assembly • Evaluation • Maintenance • Virtual assembly International Journal of Systems Science

17. Assembly/machining path restriction hand hand (tool) Assembly Assembly (machining) restriction Dimensional restriction restriction • Move the holder (tool) position • near thepart • Grasp the part (tool) • Transfer the part (tool) to the • current assembly (part) • Mate the part with the • subassembly (machine) • Retract the holder restriction Assembly Dimensional Assembly restriction restriction Assembly and machining path restriction A new challenge is the requirements deriving from assembly (machining) path restriction. The Assembly and machining path to be considered is a so called basic motion planing problem. It is used to check geometrical feasibility of the planned operation. As the motions are with small amplitude, we can restrict our attention to the workplace as a small volume around the assembly. Example assembly / machining path restriction International Journal of Systems Science

18. CAD Data Application knowledge Modeling Developcustom components Existingsystems Peripherals Visuals Build simulations Integration Application Creating a Virtual Reality application Once the mechanical parts system is designed in CAD software the geometrical data is imported to VR package EON Reality. For the complicated geometry with polygon reduction of the irrelevant parts is done before importing in order to keep performance of the simulation enough high. Depending of the assembly requirements additional tools and machinery like screw driver, wrenches, torque wrenches, push tools, press machine, crane machine and controllers are developed and set. Further room interior: walls, floor, working tables is created and imported to VR package. New 3D GUI user-friendly interface is designed to be comfortable for ordinary user, not accustomed to PC environment. Later setups, behavior model, and the sequence flow and control functions of the assembly system is programmed. International Journal of Systems Science

19. Virtual Manufacturing and Assembly – CAD/CAM First problem is that CAD data is very big, so perforce is very slow so frame rate is so slow that even without assembly interaction with an object, the user has to wait. The other problem is that CAD software are designed for other type of usage, mainly drafting, but when it come to interface, working space and immerse filling it cannot fulfill those needs. Another issue is that that VR peripherals are not designed for and does not work with CAD systems. Certainly some CAD companies have tried to include assembly function in the software, but virtual reality presents can not be achieved in most of the cases. The design shall be evaluated in 3-D environmental to achieve an immerse filing of the of the model, workspace and user movements to allow interaction when the simulation run in real time speed. Why not only CAD/CAM? International Journal of Systems Science

20. Creating VR simulation (1) • CAD Designed model shall be modified for virtual assembly: • Data reduction - As the virtual assembly simulation is dynamic process, using motion, animations, etc., huge data model will cause performance degradation. • Immerse feeling of the model and workspace -Virtual working space shall be designed in the matter that is close to the real situation to carry out an immerse feeling • Peripherals-3d tracing sensors, gloves for new task used in virtual assembly such as grasping and caring objects, tracking the movement of the assembled parts and the user during assembly process • HDM, Stereo Glasses-use of Head Mounted Display or stereo Glasses to achieve an immerse filing in Tree-dimensional environmental. International Journal of Systems Science

21. Creating VR simulation (2) GUI in 3D-Space User Interface (SUI), user position set, forward button, cancel, redo etc. - New easy to use SUI shall be designed, including gesture mechanisms closely resembling real gesture movement of the operator.   Feedback Force- The role for the feedback force is to implement touch feeling and physical quantities such as friction, dumping, and collision detection between the user and virtual objects. Without such physical quantities, simulation is basically orientation to visual perception, that is not enough for person to control and use VR simulation, moreover without such feelings user can get disoriented and lost in VR space. Keeping VA version update and synchronization in virtual enterprise -As the virtual assembly design works in condition of the concurrency, special attention shall be paid to VA data version update and synchronization. International Journal of Systems Science

22. The concept for building Virtual manufacturing and assembly • The system developed has several important differences from usually developed VR based simulation: • It is implemented based on the real assembly practice • Achieves high speed frame rate on ordinary personal computer. • Other very important issue is the user interface interaction with VR environmental in the natural manner. No special knowledge or computer skills are required as the work within the immerse space VR assembly environmental with gloves, HMD and tracking sensors offer an easy non interrupting style of work. International Journal of Systems Science

23. Behavior Model and relations Collision detection hand to part between parts Assembly hierarchy Ordered tree Hand before collision Ghost hand Show force Virtual Reality simulation example-assembly schema • Assembly ordered tree Assembly precedence order constraint shall be implemented in the designed assembly simulation structure to prevent confusion or inaccurate assembly result. • Collision detection prevents part interfere with other part. • Physical constraint are simulated by assembly constraint (connection strength should be simulated) Assembly between parts shall meet constraint such as axis alignment, surface matting and alignment. Constraint are implemented in the assembly simulation to secure and help the assembly operation during the assembly, as the is no possible for him to fulfill those constraints in the real assembly. International Journal of Systems Science

24. Virtual Reality simulation hardware and simulation • Peripherals-3D tracing sensors, gloves for new task • HDM, Stereo Glasses-use of Head Mounted Display or stereo Glasses to achieve an immerse filing in Tree-dimensional environmental. Simulation consideration • CAD Data import and reduction • Immerse feeling of the model and workspace • 3D Space User Interface (SUI), New easy to use and intuitive SUI shall be designed, including gesture mechanisms closely resembling real gesture movement of the operator. • Force Feedback - Version update • Keeping VA version update and synchronization in virtual enterprise International Journal of Systems Science

25. Virtual Reality simulation - hardware and software • One of the benefits of using of the ordinary business PC reduces the cost, but more important is the possibility to use available peripherals to set up a complete system available even for medium size research laboratory. The system we use to develop and run the simulation, consist of PC Pentium 400, 128 RAM. Assembly hierarchy Ordered tree Behavior Model and relations • For complete immersion HMD Virtual research V8 (also Sony Glasstron-Stereo HMD, Polhemus Fastrack Tracking System, Fake Space pinch gloves. • We run EON Virtual reality software that supports most of the peripherals. Key parts of this VR software are: EON studio the main engine, Data Exchange –Most geometry was imported using data exchange from CAD software IGES, STEP and Pro/E file format. The rest of the models were develop in Multigen, 3DSMax and Autocad and imported. International Journal of Systems Science

26. Virtual assembly example • Constraint imposed to the sequence of the assembly does not allow improper order of assembly. • Special 3D-SUI was developed. As a result system work using common human gestures, no special computer skills are required. • Gloves are used with FakeSpace and Fastrack were used to interact VR simulation in natural, manner.Each part was assembled to other using hands to hold and move them. • The hands, head, body movements correspond 1:1 to the real movement International Journal of Systems Science

27. Virtual assembly example (2) • Gloves are used with FakeSpace and Fastrack were used to interact VR simulation in natural, manner.Each part was assembled to other using hands to hold and move them. • The hands, head, body movements correspond 1:1 to the real movement International Journal of Systems Science

28. Virtual Reality simulation systems From simple Virtual assembly simulation to virtual automation plant system International Journal of Systems Science

29. Virtual Reality Application International Journal of Systems Science

30. Virtual Reality Application (2) International Journal of Systems Science

31. Virtual Reality Application (3) International Journal of Systems Science

32. Collaboration work using Peer to Peer network • A new computing network method peer to peer (P2P) that provides an opportunity for direct design collaboration between two clients without the use of a central computer. • There are different activities within the product realization process, such as design, marketing, sales, etc., all of which having different objectives and needs. Collaboration involves feedback from these activities into the product realization process. • Connections between computers are established and • eliminated on demand. International Journal of Systems Science

33. P2P collaboration (2) P2P network consists of both wireless and wire-line Internet and network connections. The method of working by marketing and sales is not often the same as designers, whereas designers typically sit on the computer to generate the product design on the computer, the marketing and sales staff are typically in the field. Marketing and sales staff usually carry mobile devices, like personal digital assistant (PDA), wireless notebook computer, cellular phone, because the task they perform are generally not those performed by design engineers. For example, the objectives of the marketing activity may include selection of color, material Engineering Engineering Workstation Workstation OS1 OS1 Notebook Notebook PDA PDA OS2 OS2 Web Web pad pad Cellular Cellular Internet Internet Opened P2P Opened P2P Phone Phone PC desktop PC desktop Connection Connection OS3 OS3 Open on demand Open on demand Connection Connection for aesthetic features of the product, whereas the objectives of the assembly designer are to ensure that no two components are intersecting in 3 dimensional (3D) space International Journal of Systems Science

34. Conclusion We described framework and applications of Virtual reality simulation trough product lifecycle. The proposed Virtual reality test system allows complete geometrical representation and simulation of the functions of designed products. These products can be tested to provide feedback, and design changes can be done on demand. Further this VR system can be applied at any step of production realization process as well at the end-user site. No special knowledge or computer skills are required to be learned by end user for interaction and control. International Journal of Systems Science

35. Questions? International Journal of Systems Science