Scalable Peer-to-Peer Networked Virtual Environment

1. Scalable Peer-to-Peer Networked Virtual Environment Master Thesis Oral Examination Dept. of CSIE, Tamkang Univ. Advisor: Dr. Chen Jui-Fa Shun-Yun Hu 2005/01/07

2. Outline • Introduction • Voronoi-based Overlay Network (VON) • Simulation Results • Analysis • Conclusion

3. What is Networked Virtual Environment (NVE)? • Virtual Reality + Internet • 3D environment with people (avatar), objects, terrain, agents • Military simulations (’80)  Massively Multiplayer Online Games (mid-‘90) • Trends: larger scale, more realistic simulation

5. NVE: A Shared Space

6. Issues for Creating NVE • Consistency (events/states) • Responsiveness multiplayer • Security • Scalability • Persistency massively multiplayer • Reliability (Fault-tolerance)

7. The Scalability Problem • Many nodes on a 2D plane ( > 1,000) • Message exchange with those within Area of Interest (AOI) • How does each node receive the relevant messages? Area of Interest

8. A simple solution (point-to-point) N * (N-1) connections ≈ O(N2)  Not scalable! Source: [Funkhouser95]

9. A better solution (client-server) Message filtering at serverto reduce traffic N connections = O(N) server is bottleneck Source: [Funkhouser95]

10. Current solution(server-cluster) Still limited by servers. Expansive to deploy & maintain. Source: [Funkhouser95]

11. Scalability Analysis • Scalability constrains • Computing resource (CPU) • Network resource (Bandwidth) Non-scalable system vs. Scalable system Resource limit x: number of entities y: resource consumption at the limiting system component

12. What Next? • Strategies • Increase resource  More servers • Decrease consumption  Message filtering • Architectures Scale • Point-to-point (LAN) tens 10^1 • Client-server hundreds 10^2 • Server-cluster thousands 10^3 • ? millions 10^6 … Peer-to-Peer

13. What is Peer-to-Peer (P2P)? [Stoica et al. 2003] • Distributed systems without any centralized control or hierarchical organization • Runs software with equivalent functionality • Examples • File-sharing: Napster, Gnutella, eDonkey • Distributed computing: SETI@Home (UC Berkeley) • VoIP: Skype

14. Peer-to-Peer Overlay A P2P overlay network source: [Keller & Simon 2003]

15. Promise & Challenge of P2P • Promises • Growing resource, decentralized  Scalable • Commodity hardware  Affordable • Challenges • Topology maintenance  dynamic join/leave • Efficient content retrieval no global knowledge

16. Issues for Creating P2P NVE • Consistency (events/states) • Responsiveness multiplayer • Security • Scalability • Persistency massively multiplayer • Reliability (Fault-tolerance) • Consistency (topology)  P2P NVE

17. Related Works (1): SimMUD [Knutsson et al. 2004] (Univ. of Pennsylvania) • Pastry + Scribe • Regions • Coordinators (super-nodes) • Fixed-size region • Relay overhead

18. Related Works (2) [Kawahara et al. 2004] (Univ. of Tokyo) • Fully-distributed • Nearest-neighbors • List exchange • High transmission • Overlay partition

19. Related Works (3): Solipsis [Keller & Simon 2003] (France Telecomm R&D) • Links with AOI neighbor • Mutual cooperation • Inside convex hull • Potentially slow discovery • Inconsistent topology

20. Outline • Introduction • Voronoi-based Overlay Network (VON) • Simulation Results • Analysis • Conclusion

21. Design Goals • Observation: • for virtual environment applications, the contents we want are messages from AOI neighbors • Content discovery is a neighbor discovery problem • Solve the Neighbor Discovery Problem in a fully-distributed, message-efficient manner. • Specific goals: • Scalable  Limit & minimize message traffics • Responsive  Direct connection with AOI neighbors

22. Voronoi Diagram • 2D Plane partitioned into regions by sites, each region contains all the points closest to its site • Can be used to find k-nearest neighbor easily Neighbors Region Site

23. Design Concepts Use Voronoi to solve the neighbor discovery problem • Identify enclosing and boundary neighbors • Each node constructs a Voronoi of its neighbors • Enclosing neighbors are minimally maintained • Mutual collaboration in neighbor discovery

24. Procedure (JOIN) 1)Joining node sends coordinates to any existing node Join request is forwarded to acceptor 2)Acceptorsends back its own neighbor list joining node connects with other nodes on the list Joining node Acceptor’s region

25. Procedure (MOVE) 1) Positions sent to all neighbors, mark messages to B.N. B.N. checks for overlaps between mover’s AOI and its E.N. 2) Connect to new nodes upon notification by B.N. Disconnect any non-overlapped neighbor Boundary neighbors Non-overlapped neighbors New neighbors

26. Procedure (LEAVE) 1) Simply disconnect 2) Others then update their Voronoi new B.N. is discovered via existing B.N. New boundary neighbor Leaving node (also a B.N.)

27. Dynamic AOI Crowdingwithin AOI can overload a particular node It’s better if AOI-radius can be adjusted in real time

28. Adjustment Conditions • AOI-radius decrease • Number of connections > maximum allowable connections • AOI-radius increase • Maximum connections not exceeded • Current AOI-radius < preferred AOI-radius • Delay counter • To avoid fluctuations

29. Demonstration Simulation video • General movements (20 nodes, 800x600 world) • Local vs. global view • Dynamic AOI adjustment

31. Outline • Introduction • Voronoi-based Overlay Network (VON) • Simulation Results • Analysis • Conclusion

32. Simulation Method • C++ implementation of Voronoi-based algorithm • World size: 1000 x 1000, AOI: 150 • Trials from 10 – 250 nodes • Connection limit per node: 10 • 1000 time-steps (~ 100 simulated seconds, assuming 10 updates/seconds) • Behavior model • Random movement: random direction • Constant velocity: 5 units/step • Movement duration: random (1 – 25 steps)

33. Consistency Metrics • Topology Consistency [Kawahara, 2004] Number of observed AOI neighbors Number of actual AOI neighbors • Drift Distance [Diot, 1999] Distance between observed position and actual position (average over all nodes)

34. Basic ModelTopology Consistency

35. Basic ModelScalability (1)

36. Basic ModelScalability (2)

37. Dynamic AOI Model

38. Dynamic AOIScalability (1)

39. Dynamic AOIScalability (2)

40. Dynamic AOIScalability (3)

41. Dynamic AOITopology Consistency (1)

42. Dynamic AOITopology Consistency (2)

43. Dynamic AOIReliability (1)

44. Dynamic AOIReliability (2)

45. Outline • Introduction • Voronoi-based Overlay Network (VON) • Simulation Results • Analysis • Conclusion

46. Analysis of Design Consistency (Topology) • Topology is fully connected & consistent enclosing neighbors Responsiveness • Lowest latency  direct connection, no relay Scalability • Resource-growing & decentralized resource consumption Reliability • Self-organizing for small number of node failures

47. P2P NVE Comparisons

48. Problems of Voronoi Approach • Message traffic • Circular round-up of nodes • Redundant message sending (inherent to fully-distributed design) • Incomplete neighbor discovery • Can happen with inconsistent / incorrect neighbor list • Fast moving node

49. Outline • Introduction • Voronoi-based Overlay Network (VON) • Simulation Results • Analysis • Conclusion

50. Conclusion • NVE scalability is achievable with P2P architecture and is a neighbor discovery problem • A promising solution: Voronoi-based P2P Overlay • Leverage knowledge of each peer to maintain topology • Properties • Scalable: fully-distributed, dynamic AOI • Efficient: low irrelevant messages, zero relay • Robust: consistent and self-organizing topology