ACTIVE RELIABLE MULTICAST HOW IT WORKS, HOW IT CAN BE USED ON COMPUTATIONAL GRIDS

1. ACTIVE RELIABLE MULTICASTHOW IT WORKS, HOW IT CAN BE USED ON COMPUTATIONAL GRIDS Congduc PHAM SUN's "Gourmandise Cérébrale" SUN Labs Europe, Thursday, February 14th, 2002 http://www.ens-lyon.fr/LIP/RESAM

2. Outline • Introduction • How it works • How it can be used on computational grids

3. Everybody's talking about multicast! Really annoying ! Why would I need multicast for by the way? multicast! multicast! multicast! multicast! multicast! multicast! multicast! multicast! multicast! multicast! multicast! multicast! alone multicast! multicast! multicast!

4. Challenges for the Internet Think about… • high-speed www • video-conferencing • video-on-demand • interactive TV programs • remote archival systems • tele-medecine, white board • high-performance computing, grids • virtual reality, immersion systems • distributed interactive simulations/gaming…

5. From unicast… Sender • Problem • Sending same data to many receivers via unicast is inefficient • Example • Popular WWW sites become serious bottlenecks data data data data data data Receiver Receiver Receiver

6. …to multicast on the Internet. Sender • Not n-unicast from the sender perspective • Efficient one to many data distribution • Towards low latence, high bandwidth data data data data Receiver Receiver Receiver

7. User perspective of the Internet from UREC, http://www.urec.fr

8. What it is in reality… from UREC, http://www.urec.fr

9. Links: the basic element in networks • Backbone links • optical fibers • 10 to 160 GBits/s with DWDM techniques • End-user access • V.90 56Kbits/s modem on twisted pair • 512Kbits/s to 2Mbits/s with xDSL modem • 1Mbits/s to 10Mbits/s Cable-modem • 64Kbits/s to 1930Kbits/s ISDN access • 9.6Kbits/s (GSM) to 2Mbits/s (UMTS) • 155Mbits/s to 1Gbits/s SDH

10. Routers: key elements of internetworking • Routers • run routing protocols and build routing table, • receive data packets and perform relaying, • may have to consider Quality of Service constraints for scheduling packets, • are highly optimized for packet forwarding functions.

11. The Wild Wild Web heterogeneity, link failures, congested routers packet loss, packet drop, bit errors… important data ?

12. Multicast difficulties • At the routing level • management of the group address (IGMP) • dynamic nature of the group membership • construction of the multicast tree (DVMRP, PIM, CBT…) • multicast packet forwarding • At the transport level • reliability, loss recovery strategies • flow control • congestion avoidance

13. Reliable multicast • What is the problem of loss recovery? • feedback (ACK or NACK) implosion • replies/repairs duplications • difficult adaptability to dynamic membership changes • Design goals • reduces recovery latencies • reduces the feedback traffic • improves recovery isolation

14. Active Reliable Multicast How does it work?

15. What is active networking? • Programmable nodes/routers • Customized computationson packets • Standardized execution environment and programming interface • No killer applications, only a different way to offer high-value services, in an elegant manner • However, adds extra processing cost

16. Motivations behind active networking • user applications can implement, and deploycustomized services and protocols • specific data filtering criteria (DIS, HLA) • fast collective and gather operations… • globally better performances by reducing the amount of traffic • high throughput • low end-to-end latency

17. Active networks implementations • Discrete approach (operator's approach) • Adds dynamic deployment features in nodes/routers • New services can be downloaded into router's kernel • Integrated approach • Adds executable code to data packets • Capsule = data + code • Granularity set to the packets

18. A1 active code A1 active code A2 A2 Data Data The discrete approach • Separates the injection of programs from the processing of packets

19. data data data data code data code The integrated approach • User packets carry code to be applied on the data part of the packet • High flexibility to define new services data

20. AL packet An active router some layer for executing code. Let's call it Active Layer

21. Solutions for Reliable Multicast • Traditional • end-to-end retransmission schemes • scoped retransmission with the TTL fields • receiver-based local NACK suppression • Active contributions • cache of data to allow local recoveries • feedback aggregation • subcast • …

22. A step toward active services: LBRM

23. NACK4 Active local recovery • routers perform cache of data packets • repair packets are sent by routers, when available data data data5 data1 data2 data1 data3 data2 data4 data3 data5 data4 data5 data4 data1 data2 data3 data5

24. NACK4 NACK4 data4 NACK4 NACK4 only one NACK is forwarded to the source NACK4 Global NACKs suppression

25. NACK NACK data NACK NACK NACK Local NACKs suppression

26. data4 NACK4 data4 NACK4 data4 data4 data4 NACK4 data4 data4 NACK4 data4 data4 data4 data4 Active subcast features • Send repair packet only to the relevant set of receivers

27. Active Reliable Multicast How can it be used? Computational grids The DyRAM framework Some simulation results Conclusions and perspectives GRID?

28. What is a computational grid? application user from Dorian Arnold: Netsolve Happenings

29. Some grid applications Astrophysics: Black holes, neutron stars, supernovae Mechanics: Fluid dynamic, CAD, simulation. Distributed & interactive simulations: DIS, HLA,Training. Chemistry&biology: Molecular simulations, Genomic simulations.

30. Reliable multicast: a big win for grids Data replications Code & data transfers, interactive job submissions Data communications for distributed applications (collective & gather operations, sync. barrier) Databases, directories services SDSC IBM SP 1024 procs 5x12x17 =1020 224.2.0.1 NCSA Origin Array 256+128+128 5x12x(4+2+2) =480 CPlant cluster 256 nodes Multicast address group 224.2.0.1

31. We see something, but too weak. Please simulate to enhance signal! From reliable multicast to Nobel prize! OK! Resource Estimator Says need 5TB, 2TF. Where can I do this? Resource Broker: LANL is best match… but down for the moment From President@earth.org Congratulations, you have done a great job, it's the discovery of the century!! The phenomenon was short but we manage to react quickly. This would have not been possible without efficient multicast facilities to enable quick reaction and fast distribution of data. Nobel Prize is on the way :-) Resource Broker: 7 sites OK, but need to send data fast…

32. Multicast communications on grids • Dynamic groups are very difficult to handle with the reliability constraint • Mixture of high-throughput (data replication) and low latencies (distributed applications) needs • The application under consideration can have a great impact on the protocol design (i.e. local recoveries) • A one protocol-fits-all solution is difficult!

33. The DyRAM framework (M. Maimour) • Receiver-based: use of NACKs. • No cache in routers, receivers perform local recoveries… • …which are based on a tree structure constructed on a per-packet basis. • Routers play an active role. • Low-overhead active services • Focus on low latency • Load balancing features

34. where to put active components? 1000 Base FX active router active router core network Gbits rate Server active router active router 100 Base TX active router

35. Related works on local recovery • SRM • any receiver in the neighborhood • RMTP, TMTP, LMS, PGM, TRAM • a designated receiver • LBRM • a logging server

36. Active services in DyRAM • Designed to provide low latencies • Session initialization • Early packet loss detection • NACK aggregation • Subcast of repair packets • Dynamic replier election

37. DyRAM and IP multicast • Relies on IP multicast but has few interactions • Runs its own simple session protocol to gather additional topological information at the DyRAM level to enhance the group anonymity imposed by IP multicast

38. @R1,vif 1 @R2,vif 2 @R3,vif 2 @R4,vif 2 @D1,vif 0 Total Replies=5 IP multicast IP multicast IP multicast IP multicast IP multicast INIT INIT Total Replies=3 Reply @ @R5,vif 1 @R6,vif 1 @R7,vif 0 INIT Reply @D1 INIT INIT Reply @ Reply @ Reply @ Reply @ DyRAM: session initialization D0 DyRAM 0 2 1 D1 DyRAM R1 1 0 R2 R3 R4 R6 R5 R7

39. How and where losses can occur • Packet losses occur mainly in edge routers • In this case, all downstream links would most likely be affected by a packet loss • On medium speed LAN, when a packet has been sent on the wire all computers will usually be able to receive it • On very high-speed LAN, computers can be the bottleneck

40. DyRAM: early packet loss detection • The repair latency can be reduced if the lost packet could be requested as soon as possible • DyRAM realizes this functionality by enabling some routers to detect losses and therefore to generate NACKs towards the source • This loss detection service should be located near the source, but not too near!

41. DyRAM: replier election • A receiver is elected to be a replier for each lost packet • Several recovery trees at a given time • Load balancing can be taken into account, several optimizations possible • Uses the topological information gathered during the session initialization

42. NAK 2 from R1 NAK 2 from R2 NAK 2 from R3 NAK 2 from R4 IP multicast IP multicast IP multicast IP multicast IP multicast NAK 2 Repair 2 NAK 2,@ Repair 2 NAK 2 NAK 2,@ NAK 2,@ NAK 2 Repair 2 DyRAM: replier election @R1,vif 1 @R2,vif 2 @R3,vif 2 @R4,vif 2 @D1,vif 0 D0 DyRAM 0 2 1 D1 @R5,vif 1 @R6,vif 1 @R7,vif 0 DyRAM Repair 2 R1 1 0 R2 R3 R4 R6 R5 R7

43. DyRAM: subcasting • Tries to solve the exposure problem • Using the NACK pattern to select relevant links can not avoid exposure • Use of IP addresses is more costly but allows for an exact matching • Several optimizations possible, including a dynamic selection of the appropriate mechanism

44. Routers’ soft state The NACK State (NS) structure which maintains for each lost packet, • seq : the sequence number of the requested packet. • rank : the number of NACK received. • subList : List of the links from which similar NACKs arrived (or IP addresses).

45. Routers’ soft state (cont.) The Track List (TL) structure which maintains for each multicast session, • lastOrdered : the sequence number of the last received packet in order • lastReceived : the sequence number of the last received data packet • lostList : a bit vector that keeps track of received packet Reduces the replier election delay.

46. DyRAM overview One benefit of active networking is to unload the source from heavy retransmission overheads. The backbone is fast, very fast (DWDM, 10Gbits/s not uncommun), so nothing else than fast forwarding functions. The active router associated to the source can perform early processing on packets. For instance our DyRAM protocol uses subcast and loss detection facilities in order to reduce the end-to-end latency. source Any receiver can be designated as a replier for a loss packet.The election is performed by the associated upstream active router on a per-packet basis. Therefore several loss recovery trees can co-exist in parallel at a given time. 100 Base TX active router active router core network Gbits rate active router A hierarchy of active routers can be used for processing specific functions at different layers of the hierarchy. For instance, having an active router at the nearest location from the source/destination could performs very efficient NACK packets suppression 1000 Base FX active router active router DyRAM can increases performances by associating a dedicated active router to a pool of computing resources.

47. Some simulation results • Network model and used metrics • Local recovery from the receivers • DyRAM vs. ARM • DyRAM combined with cache at routers

48. Network model 10 MBytes file transfer

49. Metrics • Load at the source : the number of the retransmissions from the source. • Load at the network : the consumed bandwidth. • Completion time per packet (latency).

50. Local recovery from the receivers (1) • Local recoveries reduces the load at the source (especially for high loss rates and a large number of the receivers). 4 receivers/group #grp: 6…24 p=0.25