1 / 21

Spiral 1 Design

Spiral 1 Design. Joshua Horwitz Mackenzie Sweeney. Architecture Components. Server Communication: AMQP Server-side Application: Python Tornado or NodeJS RabbitMQ as message broker Client-Server Communication: SockJS & RESTful API Client-side Application: HTML, CSS, JS with jQuery

remedy
Download Presentation

Spiral 1 Design

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Spiral 1 Design Joshua Horwitz Mackenzie Sweeney

  2. Architecture Components • Server Communication: AMQP • Server-side Application: Python Tornado or NodeJS • RabbitMQ as message broker • Client-Server Communication: SockJS & RESTful API • Client-side Application: HTML, CSS, JS with jQuery • Browser Communication: PubSubJS

  3. Spiral 1 Proposal • Design/Prototype client pub/sub environment with PubSubJS • Design/Prototype Map abstraction using CMAPI specification • Either modify Mapstraction or implement our own • Maps to support for spiral 1: • LeafletJS (at minimum) • Google Maps V3 • OpenLayers • Design/Prototype UI (for maps) • Design/Prototype Overlay Widget • Extremely useful to test interoperability with other widgets • Can incorporate this as an add-on to our widget • Keep access control in mind, but don’t prioritize

  4. Client Pub/Sub • Implement using global pub/sub library that meets criteria: • JS-native • Widely used • Actively maintained/developed • Our recommendation is PubSubJS // Basic pub/sub operations PubSub.subscribe(‘map.overlay.create’, callback); PubSub.publish(‘map.overlay.create’, payload); // Can also use jQuery syntax $.pubsub(‘subscribe’, ‘map.overlay.create’, callback); $.pubsub(‘publish’, ‘map.overlay.create’, payload)

  5. Map Abstraction Layer • Goal: 0% code-rewriting, 100% interoperability • Possible starting point is Mapstraction (BSD License) • Fork and extend • Or create our own: take what’s useful and improve • Implement CMAPI over all supported maps • At a minimum we should support • LeafletJS (uses OpenStreetMaps) • Google Maps V3 • OpenLayers • Should also support multiple data formats • KMLGeoJSON

  6. How Does it Work? • Create a map template interface • For all maps, implement the same interface (think Java) • User selects an underlying map to use (LeafletJS, Google, etc) • Based on user choice, load in appropriate module • Object guaranteed to have defined template • Can subscribe to appropriate channels with known callbacks • Essentially, its polymorphism • Benefits • No shim required: no unnecessary parsing • One handler, one time • Does not rule out subsequent reloads (swap underlying map)

  7. User Interface • Much of the UI will be provided by the maps provided • Custom widgets (like overlay) should leverage: • HTML5 • CSS3 (responsiveness for different clients) • JS/jQuery • Priorities • Modern feel! • High degree of usability (learn from JC2CUI widget mistakes) • Persistent storage in browser • Get fancy later • Could create our own Grid widget (industry folks hate JC2CUI’s) • Provide cool analytics on mapped data (Jake/David )

  8. Server Pub/Sub (looking forward) • Leverage RabbitMQ • Single exchange routes published messages to rooms • All messages for one room end up at one queue • Provides a pure FIFO structure to receive messages in order • Producers (users): • publish directly to the exchange via SockJS and the RabbitMQ Web-STOMP plugin • Consumer (web server): • handles the user subscriptions and sends the data to users via SockJS

  9. Consumer Model • 2-layer consumer model • Web server • Browser (user) • Walkthrough • Web server consumes messages from room queues • Gets list of users in room from DB • Broadcasts message to all users over SockJS connections • Browsers (users) then receive messages • Channel parsed from message; appropriate callback executed

  10. Benefits of Consumer Model • Obvious benefits • Establishes central authority on room state • Update new users who join room • Recover from disconnects • New features • Collect data on publishing patterns • Save existing session state for quick restore later • This approach has been proven in the commercial world • It works! • No need to modify current CMAPI channel payloads! • HuffingtonPost Live pumps out over 20 million messages/week

  11. AMQP • Developed to solve the messaging interoperability problem in the financial industry • Used now by 100s of critical systems • Open standard • Highly interoperable • Use-cases • Transient pub/sub distribution • Reliable request-reply transactions

  12. RabbitMQ • Market leader MOM for AMQP • Most popular choice on EC2 • Bindings for multiple languages • Java, Ruby, Python, C, C#, Erlang, Node, Perl • Multi-protocol Support • AMP 0-9-1, 0-9 and 0-8 • AMQP 1.0 • MQTT, XMPP, SMTP, HTTP, STOMP

  13. RabbitMQ • Built-in RESTful API & Web-STOMP plugin • The creators have web messaging in mind! • Reliability – can persist data and recover from failures • Built-in clustering and transparent scaling • Self-healing capabilities and extensive tooling/configuration • Built-in access control techniques

  14. RabbitMQ Web-stomp • A simple bridge exposing STOMP protocol over emulated HTML5 websockets • Makes it possible to use RabbitMQ from web browsers • Comes with RabbitMQ • Used in industry (see previous huffington post example) • They handle over 20 million messages/week with web-stomp • EXTREMELY SIMPLE to use

  15. Web-STOMP Code // Stomp.js boilerplate varws = new SockJS('http://' + window.location.hostname + ':15674/stomp'); var client = Stomp.over(ws); send = function(data) { client.send('/topic/bunny', {}, data); }; varon_connect = function(x) { id = client.subscribe('/topic/bunny', function(d) { // do some stuff }); }; varon_error = function() { console.log('error'); }; client.connect('guest', 'guest', on_connect, on_error, /');

  16. So what is this SockJS? • A wspolyfillthat provides a ws-like JS object in ANY browser • Automatic fallback if the channel can’t support websockets • SocketIO does not do this • Also works in browsers behind ws-unfriendly proxies • If it doesn’t work, your network broke the Internet • Well documented scaling and load balancing techniques • Can utilize sticky sessions (JSESSIONID) or prefix-based LB • Used in critical industry systems

  17. Server Application • Must support SockJS • Our recommendation is Python/Tornado • 2nd option recommendation is NodeJS/Express • The approach we are outlining has been proven in the commercial industry as a viable solution • http://www.erlang-factory.com/upload/presentations/791/HuffingtonPost.pdf • Extremely easy to implement • simple prototype has been developed to proof feasibility

More Related