Tae Soo Yun Dept. of Digital Contents Dongseo University Fall 2002 based on notes from

1. Advanced Topics in Virtual Reality Tae Soo Yun Dept. of Digital Contents Dongseo University Fall 2002 based on notes from Soon Ki Jung, KNU Wohn, KAIST ……

2. Overview Chapter 1. Introduction Chapter 2. Psychological and Cognitive Issues Chapter 3. VR System Anatomy Chapter 4. Virtual Perception Chapter 5. Interaction Chapter 6. Virtual Worlds: Representation, Creation & Simulation Chapter 7. Virtual Worlds: Rendering Chapter 8. Networked VR Systems and Shared Virtual Worlds Chapter 9. Augmented Reality Chapter 10. Applications and Advanced Issues

3. Chapter 1. Introduction 1-1. What is VR? 1-2. Related Areas 1-3. History 1-4. State of the Art and R&D Issues 1-5. Some Novel Applications 1-6. Major Players 1-7. Information Resources

4. 1-1. What is VR? 1. Science vs. engineering 2. Some terminologies 3. VR as a computer technology 4. VR as a media 5. Conceptual model

5. 1. science vs. engineering • What is not VR. • high road - replication of reality low road - 3-D interface / interaction • VR = studies on reality ===> computational reality • analogous to AI • lots of hypes • AI as a science / engineering

6. What is not VR.

7. VR as computational reality • To seek for the computational model of reality. • To apply the model to the VR system. • VR can be understood in the context of modeling efforts. • intelligence • linguistics • emotion • life • reality • compuational “X”

8. 2. Some terminologies • virtual reality • virtual environment • synthetic environment • cyberspace • computer generated environment that is ... • immersive, • interactive, • multi-sensory, • viewer-centered, • 3-D.

9. 3. VR as a computer technology • Technological trend • powerful • smart • physical • computer - human interaction • primary concern --- software other important issues --- hardware, human factors, social issues, infrastructure

10. Computer World Interacting with computers

11. Computer World New Paradigm for HCI

12. 4. VR as a media • All medium attempt to create the “virtual presence”. • Theater, poem, fine arts, novel, telephone, movies, TV, ... • VR vs. existing media • immersive • interactive • 3-D • multimodal • mediated • Information is not sent back and forth. • Mediated environments are created and then experience.

13. effectiveness of communication depends on ... the sense of “being there”. • virtual presence • virtual presence depends on ... • sensory breadth • sensory depth • interactivity

14. 5. Conceptual model • Basic components of VR systems: • machine • interface • human • Issues • The roles of the components • How to put them together? • How to conceptualize (or abstractize) the system? • functional breakdown • human and technological views

15. Sensors (Video Camera/ Virtual Camera) Effectors (CRT Display) Sensors (Eyes) Inter-linkage hardware Tele-operations hardware or simulation computer Human Operator Physical/ Virtual worksite Effectors (Robot/ Virtual effector) Sensors (Joystick/dataglove) Effectors (Hands) By Stephen Ellis, in “What are Virtual Environments?”, IEEE CG&A, pp.17-22, Jan 1994

16. Participant Environment Cognitive/psychological Prior personal experience Perceptual system Effectors Receptors Perception Virtual reality system Individual ergonomics Muscle system Sensors Physical Response Individual sensual sensitivity Action Emotional Response from “A Conceptual Virtual Reality Model”, J. Latta and D.Oberg, IEEE CG&A, vol 14, No. 1, pp.23-29, Jan 1994.

17. Virtual reality system Technical confection Perceptual systems Effector Sensor Participant Real environment Body states Muscle systems Sensor Effector Confection model from “A Conceptual Virtual Reality Model”, J. Latta and D.Oberg, IEEE CG&A, vol 14, No. 1, pp.23-29, Jan 1994.

18. Conceptual Model of VR sensing H-sensor perception cognition motion control H-effector action Human Natural environment

19. Conceptual Model of VR H-sensor perception cognition motion control H-effector ? Human Virtual environment

20. Conceptual Model of VR Virtual environment Human H-sensor perception cognition motion control H-effector V-sensor V-effector sensing action sensing ? action V-human V-vehicle V-bike V-hand etc.

21. Conceptual Model of VR Virtual environment Human H-sensor perception cognition motion control H-effector V-sensor V-effector sensing action sensing ? action Metaphor !

22. Conceptual Model of VR Virtual environment Human H-sensor perception cognition motion control H-effector V-sensor V-effector sensing action P-effector L-effector sensing output device input device action P-sensor L-sensor joystick mouse tracker TV camera microphone 2D - selector pick locator etc. 3D - navigator manipulator etc. V-human V-vehicle V-bike V-hand etc.

23. Chapter 1. Introduction 1-1. What is VR? 1-2. Related Areas 1-3. History 1-4. State of the Art and R&D Issues 1-5. Some Novel Applications 1-6. Major Players 1-7. Information Resources

24. 1-2 Related areas 1. Training simulation 2. Tele-operation 3. Computer graphics 4. Artificial intelligence

25. Training simulation • Differences • Reconfigurable by changing software • May include highly natural environment • Highly interactive and adaptive • Use of a wide variety of human sensing modalities and sensorimotor systems • Highly immersive • Near-field is real; far-field is synthetic

26. Tele-operation • For at least 30 years • Tele-operator • Directly (manually) controlled tele-operator • Tele-robot

28. Computer graphics • Modeling • Motion control (animation) • Rendering • User interface

29. Artificial intelligence • Studies on perception and cognition • Testbed for AI research

30. History Norbert Wiener | Cybernetics <1948> Norbert Wiener defined "cybernetics" as the science of transmitting messages between man and machine, or from machine to machine.

31. History • Fred Waller | Cinerama <1952>

34. History • Morton Heilig, Sensorama <1962> if an artist controlled the multi-sensory stimulation of the audience, he could provide them with the illusion and sensation of first-person experience, of actually "being there." Cinema of the future, “experience theater”

35. History Ted Nelson | Hypertext <1963>

36. History Nam June Paik | Cybernated Art <1967>

37. History Douglas Engelbart | Augmentation <1968> "Our goal of augmenting the human intellect... will exhibit more of what can be called intelligence than an unaided human could... by organizing his intellectual capabilities into higher levels of synergistic structuring." He is best known as the groundbreaking engineer who invented such mainstays of the personal computer as the mouse, windows, e-mail, and the word processor. GUI @ xerox 1970s.

38. History Ivan Sutherland | Ultimate Display <1970> Ivan Sutherland, while conducting his initial research in immersive technologies, Sutherland wrote The Ultimate Display in 1965 in which he made the first advance toward marrying the computer to the design, construction, navigation and habitation of virtual worlds.

39. Sutherland’s 1965 vision • Display as a window into a virtual world … • Improve image until the picture looks real … • Computer maintains world model in real time … • User directly manipulates virtual objects … • Manipulated objects move realistically … • Immersion in virtual world via head-mounted display … • Virtual world also sound real, feel real …

40. Myron Krueger | Responsive <1970> “Artificial Reality” the computer responded to the gestures of the audience by interpreting, and even anticipating, their actions. Audience members could "touch" each other's video-generated silhouettes, as well as manipulate the odd, playful assortment of graphical objects and animated organisms that appeared on the screen, imbued with the presence of artificial life.

41. Alan Kay, Interface < 1972> • Michael Naimark, Surrogate Travel <1979> • Bill Viola, Dataspace <1983> • William Gibson, Cyberspace <1984> • Scott Fisher, Telepresence <1985> • Marc Canter, Macromedia, SoundWorks, Director <1988> • Tim Berners-Lee, WWW <1989> • Pavel Curtis, World Building <1991> LambdaMoo • Daniel Sandin, CAVE <1992> • …….

42. 1st stage : some visionaries • Moton Heilig, Ivan Surtherland, Myron Krueger, William Gibson • 2nd stage : technology development for specific purpose • Training simulator, space exploration, tele-operation • 3rd stage : VR as the general-purpose technology • VR industry, VR academia • Next stage : Toward a scientific discipline • Computational reality • A new computing paradigm • A new media • A new art form • Representation, creation and operation of virtual worlds

43. Questions?

44. 1-4. State of the Art & Issues • Reference • Virtual Reality: Scientific and Technological Challenges”, pp. 35-66, National Research Council, National Academic Press, 1995. • Areas of the study • application domains • psychological issues • VR technologies • evaluation of VR systems

45. 1. application domains • design, manufacturing & marketing • medicine, health care • hazardous operations • training • entertainment, military • experimental psychology • education • information visualization • tele-communication, tele-travel

46. 2. psychological topics • human performance characteristics • alteration of sensori-motor loops • developing the cognitive model • cognitive side-effect

47. 3. VR technologies • Gap between the current technology the required technology (exception -- entertainment, tele-operation) (1) human-machine interface (2) computer generation of VE (3) tele-robotics (4) network

48. human-machine interface • visual channel • auditory channel • haptic channel • motion interface • others • position tracking • video camera • microphone • others

49. visual channel • visual display • HMD • OHD (off-head display) • perceptual effects • mis-registration • sensori-motor alteration • distortion • time-delay • noise

50. research issues • ergonomics • improvement of resolution and fov • wireless • integration of visual, auditory, position tracking • sun glass-like • see-through option • exploiting foveal and peripheral vision