M ERCURY : A Scalable Publish-Subscribe System for Internet Games

MERCURY:A Scalable Publish-Subscribe System for Internet Games Ashwin R. Bharambe To appear in NetGames’02

Game architectures • Last generation (DOOM) • Broadcast each update to every participant • Game proceeds in locksteps • Current generation (Quake) • Client-server • Server maintains authoritative game state • Sends relevant updates to clients • Generation-X • Mercury!!  Big Boss

What’s wrong with them ? • Very bad scaling properties • Central server • Bottleneck for computation + bandwidth • Single point of failure • Broadcast • Ridiculous on an Internet scale! • Players receive “unneeded” updates and cheat • Scaling and robustness are not plain research curiosities • IDC expects the online-gaming market to be worth $2.3 billion by 2005 • Today's gaming servers are expensive to maintain, inflexible, and prone to crash. • Most online games get stuck at about 6,000 players per server, and players on one server can't talk to those on another, reducing the appeal of an online-gaming community. http://www.redherring.com/insider/2002/0117/724.html

More motivations for distributed game architectures • Building blocks for generic distributed systems • Multiplayer game is a complex distributed application • Communication patterns between game components are similar in spirit to many other applications • Instant messengers, Chat rooms • Service discovery, Stock tickers • Distributed auction systems • MERCURY architecture will be applicable to these distributed applications • You get to play games!!! 

The Problem:Build a Distributed Game which - • Prevents “swamping” at any cost • No Hot-Spots in the system • Minimizes wastage of bandwidth • Avoid flooding at all costs • Is robust • Nodes leaving or joining the system should have minimal impact on other nodes

Outline • Publish-subscribe systems • How games can be modeled as a publish-subscribe system • Architecture of MERCURY • Distributed publish-subscribe system • Simulation results • Future work…

Game requirements • Information needed by a player • Visible and audible events happening in his “arena” • Satellite game information • e.g., enemies, teammates, other terrain, etc. • Downloading textures etc. relevant to the “level” in which he is playing • Big Idea • These interactions can be modeled as a publish-subscribe system • With a slight difference!

What is Publish-Subscribe ? Intelligent Network I am available! Publications Subscription Wanted: a guy! I am too! • Publishers produce events or publications • Subscribers register their interests via subscriptions • Network routes content • publications “meet” subscriptions

int x 100 int y 200 int x >= 50 int x <= 150 int y >= 150 int y <= 250 Example:First-person shooting game Publication (50,250) (100,200) Player (150,150) Virtual World Subscription

The Slight Difference • Traditional pub-sub = filter • No notion of an underlying persistent state • State is very important for a game • Every publication • Is matched against subscriptions and routed • Acts as a write event on the underlying database

Requirements of a pub-sub system for games • Expressive subscription language • Different games - varied interactions • Scalability of the routing mechanism • Scaling with respect to number of subscriptions and publications • Network Efficiency • Avoid flooding of subscriptions or publications • Minimize latency – important for real-time games • Previous pub-sub systems (SIENA, Elvin, Gryphon) • Centralized – OR -- • Use broadcast in one way or other

MERCURY: Subscription Language • SQLish • Type, attribute name, operator, value • Example: int x <= 200 • Types: int, float, bool, string • Operators: • Numeric: <, <=, >, >=, !=, = • String: suffix, prefix, *

MERCURY: Routing protocol • Each node responsible for range of attribute values • Similar to B-tree – distributed! • Division of responsibility • Simple for numeric types • have bounded ranges • String types • Based on first or last few characters; e.g. ‘a-c’ or ‘aa-cz’, etc. • Can support prefix, suffix • Supporting substring efficiently is difficult • For each attribute, nodes logically arranged into circle – call this overlay as Attribute Hub

[0, 105) [240, 320) int x >= 50 int x <= 150 int y >= 150 int y <= 250 [160, 240) [0, 80) Hy Hx [105, 210) [210, 320) int x 100 int y 200 [80, 160) Routing Illustrated • Send subscription to any oneattribute hub • Send publications to allattribute hubs • Each node compares value in message to his range; and routes along the circle Subscription Publication

Routing Challenges • Cannot use hashing • Want to support range queries • Routing hops • O(n) worst case! • Can be brought down to O(log n) using exponentially spaced finger pointers ( similar to Chord ) • Load balance sensitive to distribution of data values

Simulation Results • Workload = FPS-game • Virtual world = square • Player movements: use mobility models from NS Without finger pointers! • Delay between pub. send and recv by subscriber • Load: Number of publications routed by a node

Promising Aspects • Expressive subscription language • Completely decentralized architecture • Scalability • Avoids flooding of subscriptions and publications – reduces network traffic considerably • Distributes publications and subscriptions throughout the network – can reduce swamping effectively • Problem: depends on distribution of data values • Performance • Simulation shows Publication delivery delay scales linearly

Currently working on… • Realistic workloads • Introduce BOTs into Quake – collect traces! • Building a Quake-II+ prototype which uses MERCURY • Study packets between client-server to figure out subscription model • Distribute the server state among different nodes • Diffusive load balancing • Shed off load to neighbours gradually

Questions, Comments, Suggestions, Criticisms …

M ERCURY : A Scalable Publish-Subscribe System for Internet Games