Visualization III- Interactive 3D Graphics

1. Visualization III-Interactive 3D Graphics Nicholas F. Polys Ph.D. VT Research Computing

2. Goals • Understand Networked Interactive 3D standards • Experience authoring environments (Flux Studio) • Consider the impacts of immersion for computational science

3. Spatial Domain Reprise What is the difference and when would you use one or the other? • Finite Difference • Finite Element

4. Spatial Domains • Where the simulation happens (e.g. FEM) • Meshing the simulation space • Granularity • Runtime • Example: Delaunay Triangulation • http://www.cse.unsw.edu.au/~lambert/java/3d/delaunay.html

5. Example • Salome http://www.salome-platform.org/home/presentation/overview/

6. Visualization Overview User Raw data Tables Visual structures Views Data transforms | Visual attribute | View transforms | Rendering assignment Figure 2.1: Processing in a typical visualization pipeline (from Card et al, 1999)

7. Usability: Information Visualization Visual display of abstract information • Visual Markers (Cleveland & McGill, 1984; Mackinlay, 1986; Card et al, 1999) • Multiple Views (North, 2001; North et al, 2002; Convertino et al, 2003) • Zoom-able Interfaces (Bederson et al, 1996, 2000; Woodruff et al, 1998a-c)

8. Once upon a time

10. Usability: Virtual Environments Visual display of spatial and perceptual data • Immersive and Desktop Platforms Conceptual Learning (Salzman et al, 1999) • Navigating space (Darken et al, 1996, 2002) • Naturalism & Performance (Bowman, 2002; Bowman et al, 2004) • Image Plane (Pierce et al, 1997)

11. State of the Art • Successive layers of abstraction allow developers to design and build at higher levels • Shaders and new rendering algorithms improve realism & performance • Concrete benefits of large format, high-res, and immersive displays

12. Graphics Engines • Games • e.g. Unreal, Delta3D, … • Consumer solutions • e.g. Cortona, Bitmanagement, Octaga, FreeWrl, Xj3D, Flux, H3D… • Multi-User spaces • e.g. ActiveWorlds, Blaxxun,BeThere, SecondLife, … • Industrial grade tookits • e.g. DIVERSE, Paraview & VTK, Amira…

13. Services & Servers • Integrated databases • Interoperable file formats • Referenced resources across the web • Visualization middleware services • Multi-user & persistent worlds

14. Proprietary vs. Opensource Who owns the data? Who owns the tools to access that data? How are bugs/new features accomplished? How much does it cost? Games, Google, Second Life vs. open standards

15. Data Formats • VRML & X3D: • expressive data and runtime behavior • Interoperation with WWW • Import and export from many commercial and free tools • A capable ‘common denominator’

16. Open Standards www.web3d.org • Portability • Durability • IP independence • International recognition and support

17. Foundations • ISO standard, openly published • Multiple implementations including open source codebases • X3D includes Transformation graph and behavior graph Application VRML, X3D Open GL, etc Operating System

18. Source of Specs, Models, Links, Bulleting boards, Blogs, Mailing lists, … http://www.web3d.org

19. Viewers, Browsers, Players • Plug-ins vs. Stand Alone

20. Deep Media Capabilities • ·     Real-time 3D scene graph • ·     Meshes, lights, materials, textures, shaders • ·     Integrated video, audio • ·     Animation • ·     Interaction • ·     Behaviors • ·     Scripts • ·     Application programming interfaces (API)

21. Source of Specs, Models, Links, Bulleting boards, Blogs, Mailing lists, … http://www.web3d.org

22. Web-enabled • Internet and Local Resources • MIME Types • For VRML • x-world/x-vrml .wrl • Content-encoding: x-gzip .gz • For X3D • AddType model/x3d+xml .x3d • AddType model/x3d+vrml .x3dv • AddType model/x3d+binary .x3db • AddEncoding gzip .x3dvz • AddEncoding gzip .x3dbz • Anchor, Inline

23. Application Examples • Pathsim • Carilion • DARPA • Molecular Dynamics

24. Authoring Web3D • Flux Studio (mediamachines.com, ) • White Dune (mac) • Med text editor, X3D-Edit, VRMLPad • Viewers (many, see list at end).

25. Building VEs • Navigation - many control options - walk, fly, examine or none (engines may also support proprietary modes) • Viewpoints - pre-defined camera positions

26. VRML & X3D Scenegraph Basics • Hierarchy of nodes (transformation graph) Nodes / Elements • Events and attribute Data types (behavior graph)

27. DAG!(Directed Acyclic Graph) Important nodes - (Grouping in parent/children hierarchy): • Transform { } • Group { } • Anchor { } • Switch { } • LOD { } • Billboard { }

29. Transform Node: • translation (x y z coordinates, +Y is up, +Z is toward the user) • rotation around an axis (x y z theta; language in rads, Flux in deg) • scale (factor in x y z ) • children (other nodes) • Boundingbox (helps rendering optimizations)

30. Building VEs • Models - primitives (box, sphere, cone, cylinder), extrusions, indexed face set (mesh), line set, point set, elevation grid and text (engines may also support proprietary spline and NURBS geometry) • Materials - diffuse colour, specular, emissive, ambient, shininess, transparency, colour per vertex

31. Shapes Contain: • geometry • appearance

32. geometry • Primitives • Box, Cone, Cylinder, Sphere • Extrusion • ElevationGrid • Indexed Face and Line sets • NURBS

33. Indexed Faces • Coordinate {} • coordIndex [] • creaseAngle: shading across polygons edges of the mesh • Normal [] s (for shape-dependent lighting control) • solid • colorPerVertex []

34. Shapes & Appearances • Appearance {} and Materials {} : • Color: • specular, emissive, and diffuse Colors in RGB, • shininess • transparency • ambientIntensity

35. RGB Material {} diffuseColor 0.678, 0.169, 0.07 specularColor shininess

36. Building VEs • Textures - support for JPEG, GIF, PNG and MPEG1 video. (engines may also have proprietary support for Flash, RealMedia, AVI, multi-texturing and environment mapping) • Lighting - directional, point or spotlight • Environments - background, and fog, hyperlinks (anchor), inlines

37. Building VEs • Audio - fully spatialized 3D audio in WAVE or MIDI format • Textures - support for JPEG, GIF, PNG and MPEG1 video. (engines may also have proprietary support for Flash, RealMedia, AVI, multi-texturing and environment mapping) • Lighting - directional, point or spotlight • Environments - background, switches, hyperlinks (anchor), billboards and fog

38. Auditory Perception • Sound {} • AudioClip {} • MovieTexture{} • pitch • intensity • Spatialized Audio (doppler effect) • Standard formats: .wav, .midi, .mp3, mpeg-1

39. Textures • ImageTexture {} with (or without) alpha channels can be applied and mapped to geometry as fixed or animated maps.  • Standard formats: .png, .jpg, • MovieTexture {} • TextureTransform {} … • PixelTexture {}

40. Lighting Lighting Nodes: on, intensity, ambientIntensity, color • Pointlight {attenuation} • DirectionalLight {} • Spotlight {direction, beamWidth, cutOffAngle} • AMD 1: SFBool global

41. Lighting • Lights have color! • Directional Lights • ‘ Scoped’ by scenegraph • sibling rule • webfolder/vrml/lightingbasis.wrl

42. Building VEs • Performance - LODs (levels of detail), visibility distance culling • Animation - animate position, rotation, scale, points, colour and much more. Scope for many separate animations in one world all with different time lines and triggered by different events. Almost every attribute can be animated. • Sensors - sense user activity such as touch, drag, keypress (plane, cylinder, sphere, and key sensors). Environmental sensors include time, proximity, and visibility.

43. Time & Interactivity • Keyframed animation • Functional animation • Events are ROUTEd between nodes

44. Interpolators • Position • Orientation • Scale • Others: • Colors • Coordinates

45. Sensors • TouchSensor • Dragsensors • Plane • Cylinder • Sphere • ProximitySensor • VisibilitySensor

46. Building VEs • Scripting - Interfaces directly with ECMAScript; also with Java, the web browser (DOM) and any programming language residing on the client/server • Routes - scripts, animations and object properties can be "wired" together in an infinite number of ways to create any effect

47. Scripting Client side • ECMA Script • Loosely typed • Non-compiled • Basic objects such as Math, Date, Browser • Java • Fully typed • Compiled • Industrial-strength classes

48. Immersion • Technology that helps you ‘be there’ … to ‘be present’ • Stereo rendering • View surround • Head and device tracking

50. VR Devices