1 / 33

meerKAT and the SA connection

meerKAT and the SA connection. Richard Hughes-Jones The University of Manchester www.hep.man.ac.uk/~rich/ then “Talks”. KAT7, meerKAT, & 10% SKA Topologies & Protocols. KAT7. correlator. Carnarvon 80 km. Processing and connectivity. Correlator to HPC “Worst Case”. Jasper Horrell.

yonah
Download Presentation

meerKAT and the SA connection

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. meerKAT and the SA connection Richard Hughes-Jones The University of Manchesterwww.hep.man.ac.uk/~rich/ then “Talks” Bits & Bytes Jodrell, Dec 2007, R. Hughes-Jones Manchester

  2. KAT7, meerKAT, & 10% SKA Topologies & Protocols Bits & Bytes Jodrell, Dec 2007, R. Hughes-Jones Manchester

  3. KAT7 correlator Carnarvon 80 km Bits & Bytes Jodrell, Dec 2007, R. Hughes-Jones Manchester

  4. Processing and connectivity Bits & Bytes Jodrell, Dec 2007, R. Hughes-Jones Manchester

  5. Correlator to HPC “Worst Case” Jasper Horrell Bits & Bytes Jodrell, Dec 2007, R. Hughes-Jones Manchester

  6. HPC to World • protocol overhead 1.5 • 12 m antennas, image full primary beam • 4 bytes per image pixel • image oversampling factor per dimension 4 • 65 k frequency channels • 4 Stokes parameters Jasper Horrell Bits & Bytes Jodrell, Dec 2007, R. Hughes-Jones Manchester

  7. Bits & Bytes Jodrell, Dec 2007, R. Hughes-Jones Manchester

  8. Bits & Bytes Jodrell, Dec 2007, R. Hughes-Jones Manchester

  9. Current Protocol Ideas • Core & 5 km Telescopes to correlator • RF Dark Fibre to correlator F engine • <150 km Telescopes to correlator • RF Fibre to local digitiser (F engine ?) • Infraco fibre to correlator • UDP flow to correlator • Real time • No congestion – BER losses • Correlator to Storage at HPC Carnarvon • 80 km project fibre • Many UDP flows from output of X part of FX to storage • Real time • No congestion – BER losses • HPC to Cape Town & World • > 600 km Infraco fibre • TCP Bits & Bytes Jodrell, Dec 2007, R. Hughes-Jones Manchester

  10. Why not TCP ? Bits & Bytes Jodrell, Dec 2007, R. Hughes-Jones Manchester

  11. TCP (Reno) – Recovery time problem #1 • Time for TCP to recover its throughput from 1 lost 1500 byte packet given by: 2 min 1 Gbit 1500 MTU 80 km KAT-CapeTUK Europe SA 0.8 ms 6 ms25 ms150 ms 33 ms 1.6 s26 s 16min 10 Gbit 9000 MTU 80 km KAT-CapeTUK Europe SA 0.8 ms 6 ms25 ms150 ms 56 ms 2.5 s43 s 26min Bits & Bytes Jodrell, Dec 2007, R. Hughes-Jones Manchester

  12. Remaining flows do not take up slack when flow removed Increase recovery rate RTT increases when achieves best throughput Congestion has a dramatic effect Recovery is slow TCP Reno stream sharing problem #2 Les Cottrell RH-J PFLDnet 2005 • Low performance on fast long distance paths • AIMD (add a=1 pkt to cwnd / RTT, decrease cwnd by factor b=0.5 in congestion) • Net effect: recovers slowly, does not effectively use available bandwidth, so poor throughput • Unequal sharing SLAC to CERN Bits & Bytes Jodrell, Dec 2007, R. Hughes-Jones Manchester

  13. Moving CBR over TCP problem #3 When there is packet loss TCP decreases the rate. TCP buffer 0.9 MB (BDP) RTT 15.2 ms Effect of loss rate on message arrival time. TCP buffer 1.8 MB (BDP) RTT 27 ms Timely arrivalof data Can TCP deliver the data on time? Bits & Bytes Jodrell, Dec 2007, R. Hughes-Jones Manchester

  14. BER and Packet Loss p probability of a bit error n bits per packet Probability of packet loss Time for sending a packet Loss Rate Time between losses Bits & Bytes Jodrell, Dec 2007, R. Hughes-Jones Manchester

  15. BER and Packet Loss Time between losses • Calculated times short • Is FEC in use? Bits & Bytes Jodrell, Dec 2007, R. Hughes-Jones Manchester

  16. South Africa Network Infrastructure Bits & Bytes Jodrell, Dec 2007, R. Hughes-Jones Manchester

  17. Current Academic Network • Run by TENET NREN • Preferred Supplier for TENET is/was Telkom • Access link to Cape Town University is 10 Mbit • Links conjested ! • New NREN called SANREN emerging • HartRAO • Connects nationally at 200 kbit/s • Connects internationally at 300 kbit/s • Cape Town meerKAT office uses ADSL to SAIX SAInternetXchange • Has a VPN to HartRAO • udpmon Cape Town to HartRAO – Had to use port 80 • 1400 byte packets sent OK but could not send this size back • Ping from gig8 UK ping 41.241.5.210PING 41.241.5.210 (41.241.5.210) 56(84) bytes of data.64 bytes from 41.241.5.210: icmp_seq=0 ttl=110 time=425 ms64 bytes from 41.241.5.210: icmp_seq=1 ttl=110 time=439 ms • udpmon to UK failed to connect Bits & Bytes Jodrell, Dec 2007, R. Hughes-Jones Manchester

  18. SKA Spiral Array Bits & Bytes Jodrell, Dec 2007, R. Hughes-Jones Manchester

  19. Infraco Network Infrastructure – Ring Closures Bits & Bytes Jodrell, Dec 2007, R. Hughes-Jones Manchester

  20. FSN Network Infrastructure – KAT Network Bits & Bytes Jodrell, Dec 2007, R. Hughes-Jones Manchester

  21. SKA – Spiral Array – Logical Network SA Sites – Rev 6 Bits & Bytes Jodrell, Dec 2007, R. Hughes-Jones Manchester

  22. International Links Bits & Bytes Jodrell, Dec 2007, R. Hughes-Jones Manchester

  23. SEA ME WE 3 20 SAT-3/WASC SAFE Present Global Connectivity EASSy Bits & Bytes Jodrell, Dec 2007, R. Hughes-Jones Manchester

  24. 1 2 2 3 6 7 4 5 8 9 10 11 SAT-3/WASC 1 Portugal : Sesimbra 2 Spain: Chipiona & Canary Islands : Alta Vista 3 Sénégal : Dakar 4 Côte d ’Ivoire : Abidjan 5 Ghana : Accra 6 Benin : Cotonou 7 Nigeria : Lagos 8 Cameroon : Douala 9 Gabon : Libreville 10 Angola : Luanda 11 South Africa : Melkbosstrand Bits & Bytes Jodrell, Dec 2007, R. Hughes-Jones Manchester

  25. 12 South Africa : Melkbosstrand 13 South Africa : Mtunzini 14 Réunion : Saint Paul 15 Mauritius : Baie Jacoté 16 India : Cochin 17 Malaysia : Penang 16 17 15 14 12 13 SAFE Bits & Bytes Jodrell, Dec 2007, R. Hughes-Jones Manchester

  26. Technical specifications • SAT-3/WASC • Supply contract in force since March 2000 • Supplier: Alcatel Submarine Networks (France) • System length: 14 350km • System type: 4 fibre, OA, WDM. • System capacity: 120Gb/s ultimately • 5.8 m simultaneous telephone calls • Landings in Europe, West - and Southern Africa • System design life of 25 years • SAFE • Supply contract in force since December 1999 • Supplier: Tycom Submarine Cable Systems (USA) • System length: 13 500km • System type: 4 fibre, OA, WDM • System capacity: 130Gb/s ultimately • 6.3 million simultaneous telephone calls • Landings in India, Malaysia, Mauritius, Reunion and South Africa • System design life of 25 years • Cables “Operated” by Telkom • Pricing out of reach for somepartner countries • Can only afford low BW • 1 Gbit TENET to GEANT Bits & Bytes Jodrell, Dec 2007, R. Hughes-Jones Manchester

  27. Traditional Bus Case 2.4 Terrabits CPT to London SAT-3 pricing levels -32% IRR Linear market growth Equity funded ($375m) “Shareblock” 6 Terrabits CPT to London & CPT to Americas (both key knowledge economies) Pricing @ Cost or below IRR Irrelevant Exponential growth Equity & Private funding New Submarine Cable Options Reviewed Bruce Wallace Infraco / Excom Bits & Bytes Jodrell, Dec 2007, R. Hughes-Jones Manchester

  28. Project Funding Structure Bruce Wallace Infraco / Excom Bits & Bytes Jodrell, Dec 2007, R. Hughes-Jones Manchester

  29. Proposed Routes CPT to London & CPT to Forteleza / Rio ~19,300 km’s ~$650 M - $ 700 M Bruce Wallace Infraco / Excom • World Cup 2010 Driver – Cable Needed for TV Bits & Bytes Jodrell, Dec 2007, R. Hughes-Jones Manchester

  30. Any Questions? Bits & Bytes Jodrell, Dec 2007, R. Hughes-Jones Manchester

  31. LightPath Topologies Bits & Bytes Jodrell, Dec 2007, R. Hughes-Jones Manchester

  32. Lab to Lab Lightpaths • Many application share • Classic congestion points • TCP stream sharing and recovery • Advanced TCP stacks Switched LightPaths [1] • Lightpaths are a fixed point to point path or circuit • Optical links (with FEC) have a BER 10-16 i.e. a packet loss rate 10-12 or 1 loss in about 160 days • In SJ5 LightPaths known as Bandwidth Channels • Host to host Lightpath • One Application • No congestion • Advanced TCP stacks for large Delay Bandwidth Products Bits & Bytes Jodrell, Dec 2007, R. Hughes-Jones Manchester

  33. Switched LightPaths [2] • User Controlled Lightpaths • Grid Scheduling ofCPUs & Network • Many Application flows • No congestion on each path • Lightweight framing possible • Some applications suffer when using TCP may prefer to use UDP DCCP XCP … • E.g. With e-VLBI the data wave-front gets distorted and correlation fails Bits & Bytes Jodrell, Dec 2007, R. Hughes-Jones Manchester

More Related