1 / 54

Global Mesh with a Memory: Streamlining Message Routing and Retrieval

Learn how Michael Laing Architect leverages a global mesh with a memory to optimize message routing and retrieval, ensuring efficient communication for millions of users. Discover the benefits of message-based technologies like WebSocket, AMQP, and SockJS, and explore the concepts of idempotent, replicating, racy, and resolving messaging.

cottman
Download Presentation

Global Mesh with a Memory: Streamlining Message Routing and Retrieval

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Michael Laing Architect New York Times michael.laing@nytimes.com

  2. Millions of users • Classes of service: • Gold: replicate/race/resolve • Silver: prioritize • Bronze: queueable A Global Mesh with a Memory Message-based: WebSocket, AMQP, SockJS • Idempotent: • Replicating • Racy • Resolving • If in doubt: • Resend • Reconnect • Reread Event-driven: async using libev

  3. Message: an event with data • Envelope: Routing while in motion & Locating when at rest • Metadata • Body (opaque to us) Message Envelope Metadata Body (may be absent)

  4. Message: an event with data

  5. Publish S3 / CloudFront Init sync Core Message Gateway Device AMQP CQL WebSocket HTTP Cassandra

  6. Subscribe S3 / CloudFront Init Core Message Gateway Device AMQP CQL WebSocket HTTP Cassandra

  7. Cassandra Dismiss Init Core Gateway Device Core Message Gateway Device AMQP CQL WebSocket Cassandra

  8. S3 / CloudFront S3 / CloudFront Connect S3 / CloudFront S3 / CloudFront Message Device Message Device Message Device Message Device Device Message Device Message several Device Message Gateway Device Message Gateway Device Message Gateway Device Message Gateway Core Device Message Gateway Core Core Device Message Gateway Core Message Gateway Device millions dozens millions dozens millions Cassandra Cassandra Cassandra Cassandra Cassandra Cassandra dozens

  9. Envelope – 2 forms of addressing • “Path”: 1) Routing a message to a user 2) Finding a message for a user Message nyt⨍aбrik

  10. Envelope – 2 forms of addressing • “Path”: 1) Routing a message to a user 2) Finding a message for a user • “Postoffice”: Routing a message internally in the nyt⨍aбrik Message nyt⨍aбrik Core Gateway Core Gateway

  11. The Path hierarchy • Path elements are text (utf-8 but “.” is reserved) – the 1stelement is the “category” • “category”: “feeds”, • “2nd element”: “breaking-news” • “3rd element”: “0012345”

  12. The Path hierarchy • Path elements are text (utf-8 but “.” is reserved) – the 1stelement is the “category” • “category”: “feeds”, • “2nd element”: “breaking-news” • “3rd element”: “0012345” • The elements are joined by “.” for routing • “path”: “feeds.breaking-news.00123456”

  13. Deeper into the Path hierarchy • For persistence, the path denotes a sorted “folder” containing messages in reverse datetime order (using the timestamp from the version 1 uuid uniquely identifying each message) • “feeds.breaking-news.56”/bd1961f5-1062-11e4-a630-406c8f1838fa • “feeds.breaking-news.56”/b94e8b45-1062-11e4-900d-406c8f1838fa

  14. Deeper into the Path hierarchy • For persistence, the path denotes a sorted “folder” containing messages in reverse datetime order (using the timestamp from the version 1 uuid uniquely identifying each message) • “feeds.breaking-news.56”/bd1961f5-1062-11e4-a630-406c8f1838fa • “feeds.breaking-news.56”/b94e8b45-1062-11e4-900d-406c8f1838fa • Subscribing to a path is done by “binding”, typically with wildcards: “*” matches any one element, “#” matches any sequence of elements • All breaking-news messages: “feeds.breaking-news.#”

  15. More on subscribing & retrieving • Retrieving from persistent storage can be done by path, e.g. the “latest” breaking-news messages for item 56: • “feeds.breaking-news.56”

  16. More on subscribing & retrieving • Retrieving from persistent storage can be done by path, e.g. the “latest” breaking-news messages for item 56: • “feeds.breaking-news.56” • But retrieval can also be done using trailing wild cards: • “feeds.breaking-news.#” will return the “latest” breaking-news messages for all “current” items

  17. More on subscribing & retrieving • Retrieving from persistent storage can be done by path, e.g. the “latest” breaking-news messages for item 56: • “feeds.breaking-news.56” • But retrieval can also be done using trailing wild cards: • “feeds.breaking-news.#” will return the “latest” breaking-news messages for all “current” items • The Cassandra data store is designed to return hierarchical queries with a single request and in the desired order

  18. A notable simplification: • Paths for subscribing to messages and paths for retrieving persisted messages, including the use of wild cards, are the same, e.g.:

  19. A notable simplification: • Paths for subscribing to messages and paths for retrieving persisted messages, including the use of wild cards, are the same, e.g.: • When a user logs in she is “subscribed” using her ID; messages “published” to her will be received while “persisted” messages and subscription preferences are retrieved (a few 10’s of milliseconds)

  20. A notable simplification: • Paths for subscribing to messages and paths for retrieving persisted messages, including the use of wild cards, are the same, e.g.: • When a user logs in she is “subscribed” using her ID; messages “published” to her will be received while “persisted” messages and subscription preferences are retrieved (a few 10’s of milliseconds) • Once subscription preferences arrive, she will be “subscribed” to them and any corresponding “persisted” messages retrieved • The same paths are used for subscription and retrieval

  21. Special Paths for individual routing • Our subscribers (millions of them) have numeric IDs – using those IDs directly for routing, specifically for the “binding” function, would be inefficient • “id.prefs.09067832” (namespace of 3rd element is too large)

  22. Special Paths for individual routing • Our subscribers (millions of them) have numeric IDs – using those IDs directly for routing, specifically for the “binding” function, would be inefficient • “id.prefs.09067832” (namespace of 3rd element is too large) • Instead we convert the ID to base62 elements and take advantage of the patricia trie search structures built into RabbitMQ and our gateway • “id.prefs.c.2.x.M” (equivalent to the above, used for routing)

  23. Postoffice addressing postoffice Core • The “postoffice” is a logical “bus” that connects all the services in all the nyt⨍aбrik instances globally Gateway Gateway logical view Gateway Core Gateway

  24. Postoffice addressing postoffice Core • The “postoffice” is a logical “bus” that connects all the services in all the nyt⨍aбrik instances globally • It is physically segmented and the segments are connected using RabbitMQ “federation” Gateway Gateway logical view Gateway Core Gateway

  25. Postoffice address elements • Each nyt⨍aбrik service has 3 basic uniquifying elements: • “region”: “us-west-2”, • “instance”: “i-123”, • “pid”: “12”

  26. Postoffice address elements • Each nyt⨍aбrik service has 3 basic uniquifying elements: • “region”: “us-west-2”, • “instance”: “i-123”, • “pid”: “12” • And some additional qualifiers: • “product”: “search”, • “service”: “route”

  27. Postoffice routing key • Each routing key has a “from” address embedded in it: • “region”: “us-west-2”, • “instance”: “i-123”, • “pid”: “12”, • “product”: “search”, • “service”: “resolve”

  28. Postoffice routing key • And a “to” address: • “region”: “us-west-2”, • “instance”: “-”, • “pid”: “-”, • “product”: “search”, • “service”: “route” • Each routing key has a “from” address embedded in it: • “region”: “us-west-2”, • “instance”: “i-123”, • “pid”: “12”, • “product”: “search”, • “service”: “resolve” (the “–” means “any”)

  29. Postoffice routing key • And a “to” address: • “region”: “us-west-2”, • “instance”: “-”, • “pid”: “-”, • “product”: “search”, • “service”: “route” • Each routing key has a “from” address embedded in it: • “region”: “us-west-2”, • “instance”: “i-123”, • “pid”: “12”, • “product”: “search”, • “service”: “resolve” (the “–” means “any”) • And an “action”: “action”: “route”

  30. Postoffice routing key detail • And they are put together as an ordered sequence like this: • <action>.<from address>.<to address>

  31. Postoffice routing key detail • And they are put together as an ordered sequence like this: • <action>.<from address>.<to address> • “route.\ • us-west-2.search.resolve.i-123.12.\ • us-west-2.search.route.-.-”

  32. Postoffice routing key detail • And they are put together as an ordered sequence like this: • <action>.<from address>.<to address> • “route.\ • us-west-2.search.resolve.i-123.12.\ • us-west-2.search.route.-.-” • Meaning: This is a request for a “route” action from a specific invocation of the “search” product “resolve” service addressed to any “search” product “route” service in region “us-west-2”

  33. Postoffice binding • Each service invocation “binds” (subscribes) to the postoffice using its unique address to get messages specifically directed to it, e.g. asynchronous RPC responses • <any action>.<any address>.<my address> • “*.\ • *.*.*.*.*.\ • us-west-2.search.route.i-123.12”

  34. Postoffice binding for services • Each service invocation also “binds” to the postoffice using addresses that will select messages appropriate for its service • <my action>.<my domain>.<my service> • “route.\ • us-west-2.*.*.*.*.\ • *.*.route.*.*”

  35. Postoffice binding for services • Each service invocation also “binds” to the postoffice using addresses that will select messages appropriate for its service • <my action>.<my domain>.<my service> • “route.\ • us-west-2.*.*.*.*.\ • *.*.route.*.*” • All this address manipulation is handled by common methods in the nyt⨍aбrik

  36. Routing in the Core • For load balancing on entry to the nyt⨍aбrik Core Core Message or Core

  37. Routing in the Core • For replication of important (gold service) messages Core Message and Core

  38. Routing in the Core • For distribution to all consumers Core Gateway Device Core Gateway Device

  39. Problems with Core instances • Complex connectivity: N(N-1) federation + clustering + …

  40. Problems with Core instances • Complex connectivity: N(N-1) federation + clustering + … • Many services: input, process, resolve, reject, cache_push, …

  41. Problems with Core instances • Complex connectivity: N(N-1) federation + clustering + … • Many services: input, process, resolve, reject, cache_push, … • Hence, problematic to manage

  42. Problems with Core instances • Complex connectivity: N(N-1) federation + clustering + … • Many services: input, process, resolve, reject, cache_push, … • Hence, problematic to manage • And difficult to autoscale

  43. Possible solution: refactor and simplify • A new component, the Rabbit Router, to focus on connectivity and routing

More Related