On-going Research Projects Prof. Hanqiu Sun Dept. of Computer Science & Eng., CUHK hanqiu@cse.cuhk.hk

1. On-going Research ProjectsProf. Hanqiu SunDept. of Computer Science & Eng., CUHKhanqiu@cse.cuhk.edu.hk • Virtual-hand Interfaces • Multi-sensory Surgical Simulation & Training • Interactive Fish Behavioral Animation in VR • Two-handed Task Planning in Virtual Assembly • Web-based Virtual Country Parks in HK • Haptic Sculpting in Virtual Volume Space

2. Virtual-hand Interfaces The whole-hand input interface has three novel properties: • intuitive use of the hand • smooth transition of control modes • reduced interface complexity Hand interface is formed by both postures and gestures. It is difficult to recognize both due to: • the complexity of hand structure • the lack of accurate measure Hand itself is an articulated structure with multiple degrees of freedom (DOFs) (17 active joints --> 23 degrees + 6 degrees)

3. Human Hand

5. Finger Tip Position

6. System Model

9. Accuracy

10. Efficiency

12. State Transition Diagram

14. Virtual Robot Model

15. Physics-based virtual-hand grasping • three states : free, upward, dropping • free : no collision is detected • upward : collision is detected, object is held firmly • dropping : collision is detected, object is slipping from the hand • two stages : Visual Correction and Active Grasping

16. State diagram

17. Visual correction (cont.)

18. Active grasping (cont.)

19. Experimental results (sphere)

20. Experimental results (box)

21. Virtual tools in multi-sensory interaction • Display two models simultaneously for comparison • Two–handed comparison

22. Virtual tools in multi-sensory interaction • Display modes: • Full Volume • Monoplanar • Triplanar

23. Virtual tools in multi-sensory interaction • Fork tool • to position and orientate any object • Clip tool • reveal cross-sections • Crop tool • show sub volume

24. Interactive Medical data Interface

25. Two-handed Interface

26. Interactive Fish Behavioral Animation in VR Behavioral Animation • random movement • collision between fishes • collision between fish and the env. • random swimming • behavior of crowds • speed match • school movement • behavior patterns (chaos models) • food searching behavior • fish preying behavior • danger escaping behavior

27. Fish Model Creation • use of fish skeleton Relation between control points and surface points

33. Behavior of Crowds • Schooling behavior

34. Fish Preying Behavior • similar as food searching behavior • shark attacking the fish

36. Demonstration

37. Two-handed Task Planning in Virtual Assembly Assembly modeling is the process of capturing entities and activity information related to assembling and assembly. Our task planner incorporates with a VR interface for 3D immersive interaction of CAD models and high-level task planning of mating processes. • 3D immersive VR interface • Assembly analysis • Feature/constraint update • Assembly task planning (collision-free interaction, allowable motion)

47. Two-handed Virtual Assembly

48. Geographic Information Systems GIS is a System ofcomputer software, hardware and data, and personnel to help manipulate, analyze and present information that is tied to a spatial location • spatial location– a geographic location • information– visualization of data analysis • system– linking software, hardware, data • personnel– explorers key to the power of GIS

49. Web-based Virtual Country Parks • Provide on-line visualization of multi-scale, multi-dimension, and multi-source data of HK country parks • Promote the ecological tourism and public recreation in our nature, improve the parks with the collection of public suggestions • Interactive network for citizens to know more about our country parks, communication with voice or text, even play in the parks

50. System Architecture • 3-tier architecture system with Internet-enabled environment and database system support