Distributed Trading System SWE 622

1. Distributed Trading SystemSWE 622 Charles Beach James Wolfe Scott Jones Sharita Green

2. Agenda Prototype Design Code Snippets Evaluations Logistic Measurements Demo

3. System Design Evaluator Trader1 . . . TraderX Logs network Exchange1 . . . ExchangeX

4. Exchange Design Exchange Exec <main> GUI Goods Map<good, price> ExchangeEngine TraderClient Buy/Sell <RMI> Echo Pub/Sub network Trader

5. Trader Design Trader Exec <main> ConfigFile TraderEngine GUI ExchangeInterface Buy/Sell <RMI> Receive Echoes <Pub/Sub> network Exchange

6. Communication • Traders communicate with Exchanges via RMI • Register • Buy • Sell • Exchanges echo transactions to Traders via Pub/Sub.

7. Trader Registration Trader1 Configuration File TraderEngine ExchangeInterface <Exchange1> ExchangeInterface <Exchange2> network RMI: register() ExchangeEngine ExchangeEngine Exchange2 Exchange1 TraderClient <Trader1> TraderClient <Trader1>

8. Exchange Transaction Echoes RMI: buy/sell TraderX Exchange network Pub/Sub ExchangeInterface ExchangeInterface Trader1 TraderEngine TraderEngine TraderN TraderGUI TraderGUI

9. Trader Registration • Calculation of exchange/trader clock offset and network latency Date tempDate = new Date(); long time1 = tempDate.getTime(); // RMI Register method ArrayList tempList = _bRemote.registerTrader(traderName); tempDate = new Date(); long time2 = tempDate.getTime(); // Calculate the clock offset long mid = (time2 + time1)/2; offset = ((Long)tempList.get(0)).longValue() - mid; // Calculate the network latency long netLat = (time2 - time1)/2; TraderLogger.instance().logExchangeTimeOffset(getExchangeName(), offset netLat);

10. Trader processing Echo • Detection out of order echo sequences and stale data // Determine if transaction is the next one expected if(sequenceNumber == client.getSequence()+1) { long transactionTime = timeStamp + client.getOffset(); client.incrementSquence(); // Determine if the data is stale if((_time - transactionTime) >= 10000) { updateGUI(exchangeName, function, product, quantity, price, true); } else { updateGUI(exchangeName, function, product, quantity, price, false); }

11. Exchange Processing of Registrations • Update registration times as traders re-register • Continuously check registration times for “failed” traders public void registerTrader(String traderName) { ... // timestamp registration Date d = new Date(); long registerTime = d.getTime(); // re-register by updating registration time client.setRegisterTime(registerTime); ExchangeLogger.instance().logTraderRegistration(traderName); ... } class CheckTraders extends Thread { ... Iterator itr = _traders.iterator(); Date d = new Date(); long time = d.getTime(); while(itr.hasNext()) { TraderClient client = (TraderClient)itr.next(); if (time > client.getRegisterTime() + 10000) { ExchangeLogger.instance().logTraderFailure(client.getTraderName()); itr.remove(); } }

12. Offset Calculation Trader Exchange Tt1 RMI register Et Tte (ET) Tt2 Trader Calculations Tte = (Tt1 + Tt2) / 2 Offset = Et - Tte

13. Staleness and Out of Order Messages • Problem: Need to identify an un-received message as stale before subsequent “on time” message becomes stale itself. • Solution: • Include in echo message sequence number, time of message and time of previous message. • Registration response provides Trader with the sequence number of last message sent. • Hold “out of order message” in queue until missing message(s) arrive or until previous message(s) are stale. • Previous message staleness calculated by taking “time of previous message” from on hold message, adding offset and comparing against current time. • When current time > 10 sec more than calculated time of previous message, then all previous un-received messages are noted as stale and on hold message can be released (and guaranteed to not be stale).

14. Evaluations PC1 PC2 Exchange1 Exchange2 network Trader1 Trader2 Test Scenario included two Exchanges and two Traders across two machines connected by a local LAN

15. Evaluation 1 Latency between starting to send an order and proceeding with other activities. Exchange logs a timestamp before echoing order and when finished notifying all clients

16. Evaluation 2 Latency between issuing an order and the last trader receiving it’s echo Exchange logs a timestamp before echoing order Each trader logs a timestamp when receiving an order Evaluator computes traders timestamp relative to exchanges based on offset of clocks.

17. Evaluation 3 • Latency between trader failure and the last exchange stopping echoing orders. • Trader logs a timestamp when going offline • Exchange logs a timestamp when detection of trader failure • Evaluator computes traders timestamp relative to exchanges based on offset of clocks. • Amount of time between trader failure and its next registration. • At most 15 seconds

18. Evaluation 4 Storage footprint on exchange required to echo orders as a function of the number of traders With our pub/sub approach, the exchange only sends out ONE message that reaches all Traders. Therefore the storage footprint required to echo orders to traders was not expected to grow significantly with the number of Traders. Function remains CONSTANT

19. Evaluation 4 • Memory Footprint • The exchange used about:       31.3MB with no traders at the very start,           but then drops to 28.29MB if nothing happens      28.33MB when one trader was started      30.59MB on first sell request (still 1 trader)      30.72MB on second sell request (still 1 trader)      31.02MB on seven buy requests (still 1 trader) 31.75MB on resizing (bigger) the server window      32.14MB on resizing (smaller) the window      32.14MB when second trader started      32.16MB on first buy from second trader      32.21MB on first sell of new product from second trader      32.21MB when third trader started      32.28MB after a while with no input (it fluctuates at this point 32.27 - 32.33, but mainly 32.28)       32.38MB on first sell of new product from third trader • Conclusions • Additional Traders have minimal impact on system memory

20. Evaluation 5 Average and max clock skew Trader logs each calculated “time offset” relative to each exchange over 10 second intervals

21. Evaluation 6 Average and max network latency Trader logs each calculated network latency during re-registration.

22. Development Measurements • SLOC count: 2009 • Exchange: 473 • Trader: 557 • Logger: 175 • Evaluator: 804 • Design Effort: 110 man-hours • Design: 30hrs • Coding/Integrating: 75hrs • Experiments/Evaluation: 5hrs

23. Demo

24. Questions