Building Virtual Environments

1. Building Virtual Environments Mark Green School of Creative Media

2. Introduction • Now we know a bit about the technology, what do we need to do to start building virtual environments • start looking at the tools and techniques • start by looking at different types of applications, then how we build each type of application

3. Programming • One issue is how much programming we have to do • one solution is to rely on programming for all VE development • for programmers this appears to simplify things • there will always be cases where programming is required

4. Programming • There are some problems with programming: • can only be done by programmers, lose the expertise and skill of non-programmers • not very efficient, some things are best done other ways, example: geometrical modeling • requires significant programming skill, not every programmer can do it

5. Programming • We need to develop non-programming approaches, it’s the only way to move the technology forward • the problem is figuring out how to do it • what is the appropriate set of tools? • How should these tools behave? • How do we design and produce them?

6. Application Taxonomy • In order to build tools we need to know what we are building tools for • need to look at some different types of applications: • see how these applications are developed • identify the development tasks • design and implement tools to support these tasks

7. Application Taxonomy • We want to start off with relatively simple applications and build up towards more complex ones • if we can’t support the development of simple applications, we don’t have a hope for the complex ones • start with the easy problems and work our way up

8. Application Taxonomy • What makes an application complex: • interaction: the level and type of user interaction, the more the user interacts with the application the more complex it is • behavior: do the objects in the environment do things, VE complexity increases with complexity of these behaviors • geometry: large amounts of geometry make an application complex

9. Viewing Applications • These applications are used to view some data in 3D • the data is static, there is no behavior • the user can move around the data, view it from different directions, but cannot interact with the data • simplest application type, read the data, display it while the user moves

10. Developing • The main development problems are reading the data and displaying it • reading the data is the hard one, there are many data formats, some aren’t well documented, hard to write routines to read • need to standardize on a small number of formats, write programs for that format

11. read3ds • One of our standard programs • reads 3ds files, produced by 3D Studio Max and many other programs • somewhat standard format for design programs • two examples: car dashboard originally produce by Rhino, pipes used to demonstrate viewer to an oil company

12. Car Dashboard

13. Pipes

14. Scientific Visualization • Many scientific computations produce large amounts of data, need some way of interpreting it • problems: • many different data formats, depends on the type of computation • many ways of displaying the data • can’t use a simple file reader

15. Scientific Visualization • Lots of scientists don’t like to program, want to reduce the programming they do for visualization • visualization packages that reduce the amount of programming required • one of the best examples is VTK • VTK is based on a model of visualization that makes developing applications easier

16. VTK • VTK views visualization programs as a collection of nodes, data flows between the nodes • some nodes read data, others display data • there is a collection of nodes that transform the data, applying different visualization techniques, where the real power lies

17. VTK • Developing a VTK application consists of selecting the right nodes and then connecting them up • this could be done graphically (some attempts to do so), but is usually done by programming • kitchen example: airflow through a kitchen

19. Pipeline Diagram for Kitchen

20. VTK • We have VTK running on our SGI workstations, could also put it on the PCs • did have a version that worked with our VR software, based on an old version of VTK • haven’t had chance to combine the new version of VTK with our VR software

22. Architectural Walkthroughs • Another example of this type of application • allow users to walk through building, based on output of design software • geometry can be quite complex • make sure that users walk along floor, don’t walk through walls, etc • important niche application

23. Computational Steering • Scientific computations take a long time, up to several weeks • a lot of wasted computation, not sure where to look, so cover all of the possibilities • another approach is to watch the computation as it occurs, use visualization to view partial results

24. Computational Steering • Can guide the computation as it evolves: • optimize parameter values • restrict the size of the computation • zoom in on interesting areas • by interacting with the computation can reduce computing time and increase likelihood of finding something interesting

25. Simple Interaction • The next step is adding some simple interaction: • control of the application • change viewing parameters • not directly interacting with the environment, but with the things around it • use interaction techniques that are similar to 2D ones

26. Simple Interaction • Interaction techniques: • menus • property sheets • sliders for entering numbers • the VE is controlled indirectly by changes to application or viewing parameters, user doesn’t directly interact with objects in VE

27. Simple Interaction • Some design issues: • where do we put the ITs, do they have a fixed position in space (the user may need to move to them), or do they follow the user • don’t want ITs to get in the way, don’t want them to block the objects in the VE • use transparent ITs, they don’t have a background, use line drawings

28. Simple Interaction • No real non-programmer tools • have standard ITs in a library, easy to add to program • development tends to depend too much on the application, so really need to program • not too hard to develop

29. Simple Environments • Define a not very complex but interesting application class • introduce both object behavior and interaction with objects • possible to produce with non-programming tools • use this to motivate our first generation of VE design tools

30. Simple Environments • Divide the environment into components, assemble to produce a complete VE • can develop the components separately, re-use them in other VEs • start by identifying components, then look at the tools that can be used to produce them

31. Simple Environments • Components: • background • static objects • dynamic objects • visual effects • interactions • many of these components need geometry, some need behavior

32. Geometry • We need geometry to define the appearance of all the objects • we know that geometry can be a problem, the more complex the geometry the slower the application • we must be able to control geometric complexity, best choice is polygons and meshes

33. Polygons and Meshes • A polygon is a flat 3D shape, its boundary is a set of lines, called edges • example: triangle, square, quadrilateral, etc • a mesh is a set of polygons that share edges • we will use triangle meshes, each polygon in the mesh is a triangle • meshes can be used to approximate curved surfaces

34. Meshes

35. Geometry • We can do reasonably well with meshes, particularly if we combine them with a scene graph • in a scene graph we can combine meshes, use transformations to put them together • we can animate the transformation to make the objects move, a start at behavior

36. Transformations • Three standard transformations: • translate • scale • rotate • can build more complex transformations by combining them • quickly look at each of the transformations

37. Translate • Move an object in 3D space • move along a straight line • this is a rigid motion, the object doesn’t change shape or orientation • three numbers define a translation, the amount of motion along the x, y and z axis

38. Scale • The scale transformation changes the size of an object • can have separate scale factors for the x, y and z axis • need to have a center for a scale, center is the point that doesn’t move when the object is scaled

39. Rotate • The hardest transformation • changes the orientation of an object • rotate needs a center, point that doesn’t change • it also needs an axis of rotation, the line in 3D that we rotate about • finally it needs a rotation angle

40. Rotate • There are many ways of doing rotation, some are better than others • rotations about the three coordinate axis are the simplest to understand, but don’t behave the way you expect • other approaches are harder to understand, we’ll come back to them later

41. Producing Geometry • We could do this all by programming, but we are trying to avoid that • better to use some modeling and animation program • there are many of these programs, need to find one that will work well for us • need a program that works in terms of polygons and meshes

42. Producing Geometry • Not many programs have good polygon capabilities, ability to control the number of polygons • also need to get the data out of the program and into our VE • this can also be a major problem, since most program use proprietary file formats, or export to formats that lose most of the geometry

43. Producing Geometry • I’ve used 3D Studio Max in the past, good modeler, but hard to export models • the 3DS format is difficult to work with beyond very simple things • Blender isn’t as good a modeler, but its easier to learn • in addition can write Python programs to export the geometry you need in your format

44. Producing Geometry • There are a lot models on the web, download for free, save a lot of modeling time • Many of these models are in 3DS format, or can easily be converted to it • It would be nice if we could use these models as well, concentrate on VR not on modeling

45. Producing Geometry • To use existing models we need to know their structure: • Meshes and their names • Modeling hierarchy • Size and location of the meshes / models • Models from the web will have different sizes, need to be able to scale and position them for our environments

46. Our Approach • We will use pre-built models as much as possible • Try to reduce the modeling effort • Will need to do some modeling: • The glue that holds the environment together • Objects that we can’t find on the web

47. Process • Produce a set of meshes in Blender, may use individual files, or multiple files • load Python program into Blender, execute it, writes the mesh information to a file • our program, SE, will read the mesh files, under script control combine meshes to form objects

48. Process 3DS files Blender Text Editor Dump.py Mesh Files Script SE

49. Process • Use Blender to produce mesh objects, we will use triangle meshes, can produce a wider range of objects • we will quickly look at some modeling operations • make sure all of our meshes are triangles, select all vertices, then use ctrl-T to convert to triangles

50. Process • Script file: a simple text file produced by text editor, describes how to assemble the environment • collection of simple statements: • the mesh files to read • object in the environment, the meshes that define their shape, plus other properties • comments