Visual and Haptic Rendering in Co-Space NRF2008IDM-IDM004-002

1. Visual and Haptic Rendering in Co-SpaceNRF2008IDM-IDM004-002 Alexei SOURIN Nanyang Technological University, Singapore

2. R&D Goals • Development of the fast interactive definitions by mathematical functions and algorithmic procedures, which will be small in size and invariant to the rendering platform • Introduction of function-defined physical properties to objects in Co-Space • New visual and haptic rendering pipelines have to be designed • Both the definition and implementation of the physical properties feedbacks have to be done using mathematical functions and/or algorithmic procedures. This will guarantee the invariance of the shared virtual space to the actual rendering platform. • Development of several applications illustrating the usefulness of the proposed approach, including an ability to establish physical contact with other party in Co-Space through haptic devices

3. Application of the Research • We illustrate the proposed concepts by extending the ISO standard Extensible 3D (X3D) and its predecessor Virtual Reality Modeling Language (VRML). • We provide haptic feedback from both standard objects of X3D and VRML, as well as from advanced 3D solid objects and force fields defined by functions. • Our software does not require purchasing any licenses and is available anytime anywhere on any Internet-connected computer. • We will develop interactive modeling tools for making the function-based models for any other possible rendering platforms and shared virtual communities that would constitute Co-Space. • The team is collaborating with the University of Bradford (UK), Bitmanagement Software (Germany), Singapore General Hospital, National University Hospital, and Million Advertising & Silk Screen Pte Ltd.

4. Resources • PI: Alexei Sourin (SCE, NTU)CO-PIs: Olga Sourina (EEE, NTU), Vladimir Kulish (MAE, NTU)Collaborator: Dr. Hassan Ugail, University of Bradford, UK • CamTECH center, SCE, NTU • Manpower: • 2 RF (search in progress) • 3 RSS (search in progress) • Visiting researcher (April 2009) + additional traveling funding from Engineering and Physical Sciences Research Council, UK • Equipment: • Commonly used and prospective haptic devices: SensAble Phantom Omni and Novint Falcon (purchased) • Graphics workstations (PO in progress)

5. Function-based shapes can be used on their own freely mixing with other standard X3D objects in virtual scenes Function-based physics within an invisible geometric container can superimpose with standard X3D objects Function-based appearance and physics can be attached to standard X3D objects Pilot version of the Function-based extension of X3D and VRML

6. Implicit, explicit and parametric functions are used concurrently for defining geometry and appearance of the objects Spatial coordinates and the time Functions are defined as individual formulas, function scripts and external loadable library functions to define: Geometry: curve, surface or solid Appearance: material and geometric texture Material: diffuse, specular, ambient and emissive colors, transparency, shininess Geometric texture: displacement of the underlying geometry Tangible physical properties: surface: tension and friction,density and force fields Transformations: coordinate and shape transformations, including set-theoretic operation Colors, geometric textures and physical properties are interpreted as entities defined in their own domains which are the merged together into one shape Pilot version of the Function-based extension of X3D and VRML

7. Demos and applications Click to play video • Video on the web (2’ 43”) • Project web-page with plug-in for MS Internet Explorer and Mozilla Firefox supporting X3D, VRML and haptic devices (SensAble and Novint), and interactive tool: http://www.ntu.edu.sg/home/assourin/fvrml.htm VIDEO

8. Summary • Strong trend towards virtualization of life and development of open standards for an interoperable 3D Internet. Open standards VRML and X3D are not able to answer all the current demands • Function-based extension of VRML and X3D allows for using any type of geometry, appearance, and operations and adds tangible physical properties to geometric objects • Interim progress and results • 1 PhD and 1 MEng student are involved in the project. 1 RF is shared with another project. Recruitment of RFs and students is in progress • Additional traveling funding for the UK collaborator is obtained from EPSRC, UK. • The pilot software is used in teaching computer graphics students in SCE, NTU. • Beta-testing of the haptic plug-in is in progress • Active collaboration with Bitmanagement Software Gmbh (Germany) in development of the collaborative platform (BS Collaborate) • Interactive exhibit “Haptic Interaction on the Web” showcased at Discovering Engineering’2009 exhibition in NTU (14 March). • Published 2 journal and 1 conference papers.