Networking for HEP in the UK

1. Networking for HEP in the UK Introduction to the PPNCG Network Topologies Monitoring Activities Peering to the US and Canada Grid Work SuperJANET4 R. Hughes-Jones ESnet Meeting Kyoto July 2000

2. Introduction to the PPNCG Membership includes HEP and Astronomy users Ensure the community has the required networking facilities Monitor end-to-end performance Investigate new network applications / technologies Provide advice on kit / facilities URL icfamon.rl.ac.uk/ppncg/title.html Acts as an Interface between HEP and UKERNA Collaboration : Multicast, H.323, QoS tests, SuperJANET4 Active Network Monitoring Uses the ICFA monitoring tools and the PPNCG traceping and ftp Monitor MANs / SuperJANET & the networks to CERN, DESY, US Report problems to UKERNA R. Hughes-Jones ESnet Meeting Kyoto July 2000

3. Northern College St Andrews College GSA RSAMD Napier ECA Queen Margaret AbMAN RobertGordon 155 Mbps Cleveland College PoP Site Abertay ClydeNET Glasgow CarlisleCollege EaStMAN 34 Mbps New College Durham BEN MAP Aberdeen St Andrews Stirling GlasgowCaledonia FaTMAN Paisley Strathclyde Heriot Watt UHI Edinburgh Menai Dundee Deeside Northumbria Llandrillo Durham ColegLlysfasi Edinburgh Newcastle UNN Daresbury Keele Central Lancaster Teesside Bangor NorMAN WCoH PO Lab Newcastle Staffordshire Liverpool Hope Bradford Hull NEWI Yale Sunderland Sheffield Huddersfield NNW Southport Halton Edge Hill Gla-BEN Ed-BEN York Lancaster YHMAN U-Net Bolton Inst. Ncl-BEN Scarborough UCL ITE Bangor Sheffield Hallam MCC Leeds Metropolitan LCC Liverpool LJMU Leeds Linc & Humb RNCM Derby Nottingham Manchester Metropolitan MCC-BEN Le-BEN HEAnet Ulster G-MING EMMAN CWC CWC Salford Bishop Grosseteste Leicester UMIST Manchester Leeds Nott-BEN Nott Trent BGS Belfast Be-BEN Newman City CollegeManchester Stephenson UC Northampton UCEB BCFTCS Reading Stranmill Surrey Wolverhampton Rdg-BEN London Royal Holloway St Mary BAS Birmingham Homerton Hinxton Hall UEA Bristol Warwick Bham-BEN MidMAN Westminster Worcester Coll. CWC CWC Cranfield Sandwell Roehampton Powys C-BEN Aberystwyth Westhill Aston Cambridge ITE Monkswood Coventry Kingston Harlech Cf-BEN Anglia Poly QMW MST Radar Essex CoventryTechnical College UEL RCA ULCC-BEN Ceredigion Ex-BEN R-BEN ULCC LBS Cardiff Telehouse Meirion-Dwyfor Newport Bris-BEN Imperial LMAN C&G WCMD Ravensbourne Swansea Institute Glamorgan Bristol Soton-BEN O-BEN Greenwich SWMAN Soton Southbank Falmouth RAL ULCC Goldsmiths Mid KentCollege Carmarthen St Davids Soton City UCL Trinity IOE BWEMAN Dartington Oxford Swansea Exeter Cardiff Institute UWE Birkbeck Afan SOAS Bucks See separate diagram Portsmouth Seeseparate diagram Chichester RVC LSHTM St Mark & John UKERNA BGSExeter SWAN Neath Gorseinon Pembroke ChichesterCollege Bath Sussex LondonGuildhall Merthyr Brookes Bournemouth Aberdare Swansea College Brighton Bath College LondonInstitute HEFCE Plymouth Bridgend Lampeter SuperJANET Connections in the UK R. Hughes-Jones ESnet Meeting Kyoto July 2000

4. SuperJANET Connections in London • 4 * 155 Mbit links to SuperJANET • 2 * 155 Mbit transatlantic links • 3rd link in use 18 Jul 00 • 4th planned by End of 00 R. Hughes-Jones ESnet Meeting Kyoto July 2000

5. UK to Europe TEN-155 April 2000 Peering with CANARIE made in London, uses 10 Mbit of the Dante US line for all Europe R. Hughes-Jones ESnet Meeting Kyoto July 2000

6. Europe and ESnet in the US R. Hughes-Jones ESnet Meeting Kyoto July 2000

7. Monitoring: US Traffic • UKERNA Traffic data Kbit/s. Blue Traffic from US; Maroon Traffic to US • 10 Nov 1999 17 Jul 2000 • 10 minute averages10 minute averages • Last 7 days 1 hour averagesLast 7 days1 hour averages R. Hughes-Jones ESnet Meeting Kyoto July 2000

8. Monitoring: US Traffic - Trends UKERNA – US Traffic Daily totals Mbytes • Traffic Kbit/s. Blue Traffic from US Maroon Traffic to US • 10 Nov 1999 9 Jul 2000 • 10 minute averages From US To US Weekday Weekend Xmas 99 R. Hughes-Jones ESnet Meeting Kyoto July 2000

9. Monitoring: ping plots UK - US • Ping average round trip time & Lost packet % for 100 byte packets • icfamon.dl.ac.uk – ns2.slac.stanford.edu 13 Oct 99 27 Jun 00 R. Hughes-Jones ESnet Meeting Kyoto July 2000

10. Monitoring: traceping UK - US Mar 99 John Macallister PPNCG Oxford R. Hughes-Jones ESnet Meeting Kyoto July 2000

11. Monitoring: traceping UK - US Oct 99 R. Hughes-Jones ESnet Meeting Kyoto July 2000

12. Monitoring: traceping UK - US Jul 2000 R. Hughes-Jones ESnet Meeting Kyoto July 2000

13. Monitoring: ping plots UK - CERN & DESY • Ping average round trip time & Lost packet % for 100 byte packets • icfamon.dl.ac.uk – rsplus.cern.ch icfamon.dl.ac.uk – x4u2.desy.de 27 Jun 00 27 Jun 00 R. Hughes-Jones ESnet Meeting Kyoto July 2000

14. Peering with ESnet • Used CAR and WRED to give priority to Esnet traffic into JANET. • Rtt and packet loss for 100 byte packets. • Tests DL to SLAC • Start 21 Feb 00 Normal Priority CAR+WRED enabled High Priority Config changes to CAR and WRED Further config changes R. Hughes-Jones ESnet Meeting Kyoto July 2000 Robin Tasker PPNCG DL

15. Peering with ESnet and Abilene • rtt and packet loss for 100 byte packets • Plot start 16 May 00 ESnet High Priority DL - SLAC Further config changes Change in bandwidth restrictions from Abilene into JANET Abilene High Priority DL - Stanford R. Hughes-Jones ESnet Meeting Kyoto July 2000

16. The Grid • Government has discussed 165M pounds to develop Computing Grid Infrastructure. • Across Research Councils Initiative – not just HEP • Setting up management structure to co-ordinate PPARC activities. • Have run Globus workshop so US developers could inform the community. • Several sites now working with Globus toolkit. • Web site www.datagrid.org.uk R. Hughes-Jones ESnet Meeting Kyoto July 2000

17. The Grid • Keen Interest in the EU DataGrid (CERN) middleware proposal. • HEP experiments will need world Grid access. • Grid Networking interests include: • High throughput, reliable, very large data transfers • QoS issues – diffserv, RSVP and packet marking • Monitoring, Metrics, and prediction accuracy • Modelling • Physics Applications R. Hughes-Jones ESnet Meeting Kyoto July 2000

18. PAW HBOOK Zebra Toolkit Toolkit Toolkit Campus LAN Toolkit Linux GASS Server SGI GASS Server Sun GASS Server The Grid: PAW with Globus BaBar Tau Analysis • Next Steps: • Working on “root” • Brokers • Data Location • Security + Global farms Andrew McNab Manchester R. Hughes-Jones ESnet Meeting Kyoto July 2000

19. Sigma ~1.2 ms The Grid:Measurements on SuperJANET Latency Mancester - DL • ATM MAN • 7 routers ~0.8ms / router • Router+link 0.4 ms/byte • DL-Liverpool ~10 ms rtt - agreement Richard Hughes-Jones PPNCG Manchester R. Hughes-Jones ESnet Meeting Kyoto July 2000

20. The Grid: Latency on SuperJANET Pink: LANBlue: MAN-SJ CORE Pink: Man-RAL MB Blue: Man-RAL  ~5 s nice exponential  ~ 2 ms 4.5 -> 6.7 ms R. Hughes-Jones ESnet Meeting Kyoto July 2000

21. The Grid: Throughput on SuperJANET Manc – UCL 10 Mbit Manc – Glasgow Manc – RAL 24 Mbit link Manc – RAL 2 Mbit Managed BW R. Hughes-Jones ESnet Meeting Kyoto July 2000

22. The Grid: Multi-stream ftp Oxford to SLAC Single stream 8 ftp streams ~ 0.8 single stream John Macallister PPNCG Oxford R. Hughes-Jones ESnet Meeting Kyoto July 2000

23. The Grid: Multi-stream ftp SLAC to Oxford R. Hughes-Jones ESnet Meeting Kyoto July 2000

24. The Grid: Multi-stream ftp SLAC to Oxford 25 min R. Hughes-Jones ESnet Meeting Kyoto July 2000

25. SuperJANET4 • SuperJANET4 Topology based on Worldcom Trident Network • Backbone initially SDH based • 2.5 Gbit (STM-16) • Upgrade to 10 Gbit in 2001, • 20 Gbit in 2002 • Migration from SDH to DWDM • Technology review in 2003 R. Hughes-Jones ESnet Meeting Kyoto July 2000

26. SuperJANET4: Backbone and Access links • Worldcom supply the transmission • UKERNA layer the IP • Core PoP IP router at Worldcom • Backbone Access Router at MANs • Access Links: • Large MAN 2.5 Gbit -> 10-20 Gbit • Medium MAN 622 Mbit -> 2.5 Gbit • 4 node DWDM development net • Deployment: • Backbone delivery Oct 00 • Routers + acceptance Nov 00 • Manc-ULCC traffic end Dec 00 • All sites Mar 01 R. Hughes-Jones ESnet Meeting Kyoto July 2000

27. Conclusion & Status • Network Links to US and Europe are OK (at the moment !) • Peering with Abilene, ESnet and CANARIE OK, interesting effects with CAR – WRED ! • Considerable UK interest in the Internet2 tests • Grid: active in several areas including networks • SuperJANET4 rollout is very promising R. Hughes-Jones ESnet Meeting Kyoto July 2000