140 likes | 147 Views
High Throughput Application Messaging. NYSE Technologies Data Fabric: Enterprise High Performance Messaging. Michael Schonberg March 15, 2010. Summary. High Performance Messaging in Financial Services NYSE Technologies’ Messaging APIs Data Fabric: A closer look … Data Fabric Benefits Q&A.
E N D
High Throughput Application Messaging NYSE Technologies Data Fabric: Enterprise High Performance Messaging Michael Schonberg March 15, 2010 www.openfabrics.org
Summary • High Performance Messaging in Financial Services • NYSE Technologies’ Messaging APIs • Data Fabric: A closer look … • Data Fabric Benefits • Q&A www.openfabrics.org
Market Data Flow Message path between participants and exchange www.openfabrics.org
Market Data Messaging • Participants receive messages from exchanges • Trades, quotes, updates, news, etc. • Usually from multiple exchanges • Participants send requests as messages • Orders, cancels • Exchange processes the request. • Requests passing through the exchange using messaging • Ultimately generate market data messages that get sent to the participants www.openfabrics.org
Data Fabric in the Exchange The central role of messaging in exchange infrastructure www.openfabrics.org
Messaging in the Market Data Enterprise Radically improved latency High volume/throughput Reduced server footprint www.openfabrics.org
Projected Opra 1 Second Peak Rates Opra rates projected to increase 34% over next 6 months and 85% over the next year. www.openfabrics.org
Other Applications • Generic messaging platform • Publish/subscribe • Request/reply • GMD • Applications in many industries • Logistics and distribution • Airlines • CEP and other enterprise applications www.openfabrics.org
Middleware Agnostic Messaging API (MAMA) Single general purpose messaging API Multiple simultaneous middleware implementations Abstraction without the cost Ultra high performance Nanosecond LDMA latency and ultra low RDMA latency Avoids vendor lock-in API adheres to well known idioms and conventions www.openfabrics.org
Data Fabric Data Fabric TCP For 1 GigE and fan-out to less latency sensitive applications. Data Fabric LDMA Local shared memory. Ultra-low latency between applications on a single machine. Data Fabric RDMA Single digit microsecond latency between machines. 10 GigE and Infiniband www.openfabrics.org
Data Fabric RDMA • The Problem With Sockets • User to kernel context switches and copies • Latency spikes • UDP retransmissions • The Promise of RDMA • OS bypass • IP bypass • Zero-copy • Reliable connection-based www.openfabrics.org
Data Fabric RDMA 2 • Challenges With RDMA • Difficult to program • VMS • Difficult to monitor • User apprehension • Benefits • Lower latency • Higher throughput • Smaller footprint • Power and cost savings www.openfabrics.org
Transport Latency Comparison 1 publisher, 1 subscriber, 1 Million 200 byte messages/second www.openfabrics.org
Q&A • Any questions? www.nyse.com/technologies mschonberg@nyx.com www.openfabrics.org