1 / 21

EU DataGrid - Network Monitoring Richard Hughes-Jones, University of Manchester

EU DataGrid - Network Monitoring Richard Hughes-Jones, University of Manchester. DataGrid Network Monitoring. Several tools in use – plugged into a coherent structure: PingER, RIPE one way times, iperf, UDPmon, rTPL, GridFTP, and NWS prediction engine

vianca
Download Presentation

EU DataGrid - Network Monitoring Richard Hughes-Jones, University of Manchester

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. EU DataGrid - Network Monitoring Richard Hughes-Jones, University of Manchester CAIDA Bandwidth Estimation Meeting San Diego June 2002 R. Hughes-Jones Manchester

  2. DataGrid Network Monitoring • Several tools in use – plugged into a coherent structure: • PingER, RIPE one way times,iperf, UDPmon, rTPL, GridFTP, and NWS prediction engine • continuous tests for last few months to selected sites: • DL Man RL UCL CERN Lyon Bologna SARA NBI SLAC … • The aims of monitoring for the Grid: • to inform Grid applications, via the middleware, of the current status of the network – input for resource broker and scheduling • to identifyfault conditions in the operation of the Grid • to understand the instantaneous, day-to-day, and month-by-month behaviour of the network – provide advice on configuration etc. • Network information published in LDAP schema – new self-defining • Cost Function in development – collaboration with WP1 & WP2 • Will be used by UK GridPP and e-science sites and non HEP WPs • Links to GGF and the US work PPDG, IvDGL, GriPhyn CAIDA Bandwidth Estimation Meeting San Diego June 2002 R. Hughes-Jones Manchester

  3. Network Monitoring Architecture LDAP Schema Grid Apps GridFTP Backend LDAP script to fetch metrics Monitor process to push metrics PingER (RIPE TTB) iperf UDPmon rTPL NWS etc Local Network Monitoring Store & Analysis of Data (Access) local LDAP Server Grid Application access via LDAP Schema to - monitoring metrics; - location of monitoring data. Access to current and historic data and metrics via the Web, i.e. WP7 NM Pages, access to metric forecasts Robin Tasker CAIDA Bandwidth Estimation Meeting San Diego June 2002 R. Hughes-Jones Manchester

  4. Clients WEB Display Predictions Grid Broker Analysis LDAP LDAP LDAP LDAP LDAP Web I/f Table Table Table plot plot plot raw raw raw raw raw Scheduler control control Cron script Cron script Cron script Tool Ping Netmon UDPmon iPerf Ripe Network Monitoring Components CAIDA Bandwidth Estimation Meeting San Diego June 2002 R. Hughes-Jones Manchester

  5. PingER IperfER UDPMon MDS backend Site cache Country cache Datagrid cache How Grid Accesses / Uses the Data Paul Mealor UCL CAIDA Bandwidth Estimation Meeting San Diego June 2002 R. Hughes-Jones Manchester

  6. Network Monitoring – Summary Map CAIDA Bandwidth Estimation Meeting San Diego June 2002 R. Hughes-Jones Manchester

  7. Network Monitoring - MapCentre CAIDA Bandwidth Estimation Meeting San Diego June 2002 R. Hughes-Jones Manchester

  8. Network Monitoring: Ping PPNCG: RAL to CERN CERN to RAL CAIDA Bandwidth Estimation Meeting San Diego June 2002 R. Hughes-Jones Manchester

  9. Network Monitoring: Iperf (TCP) UCL to CERN NIKHEF to RAL CAIDA Bandwidth Estimation Meeting San Diego June 2002 R. Hughes-Jones Manchester

  10. UDPmon & Ripe MAN-SARA from 25 May 02 UDPmon Loss Throughput Mbit/s MAN – SARA ~750Mbit/s RiPE 1 way SARA  RAL ms RiPE 1 way RAL  SARA ms CAIDA Bandwidth Estimation Meeting San Diego June 2002 R. Hughes-Jones Manchester

  11. RIPE 1-way & TCP throughput UK-SARA RIPE 1-way time ms Sara  RAL 20 Oct 01 RIPE 1-way time ms RAL  Sara TCP Iperf + prediction Mbit/s UCL  Sara CAIDA Bandwidth Estimation Meeting San Diego June 2002 R. Hughes-Jones Manchester

  12. iperf Pinger & UDPmon UK-Lyon From 25 May 02 – 04 Jun 02 UDPmon Loss Throughput ~140 Mbit/s Man  Lyon Ping Man  Lyon 40 ms rtt Iperf TCP Throughput ~ 47 Mbit/s Man  Lyon 1048576 byte buffer CAIDA Bandwidth Estimation Meeting San Diego June 2002 R. Hughes-Jones Manchester

  13. Ping & UDP throughput MAN-CERNFrom 20 Oct 01 PingER rtt (ms) dl – cern 1000 byte packet Forecast UDPmon throughput Mbit/s man – cern 300 * 1400 byte frames CAIDA Bandwidth Estimation Meeting San Diego June 2002 R. Hughes-Jones Manchester

  14. iperf TCP & UDPmon throughput MAN-SARA From 20 Oct 01 Iperf TCP throughput Mbit/s ucl – sara 262144 byte buffer Forecast in Green UDPmon throughput Mbit/s man – sara 300 * 1400 byte frames CAIDA Bandwidth Estimation Meeting San Diego June 2002 R. Hughes-Jones Manchester

  15. iperf & Pinger UK-Bologna From 20 Oct 01 Iperf throughput ucl – Bologna 262144 byte buffer Forecast in green PingER rtt (ms) dl – Bologna 1000 byte packet Forecast CAIDA Bandwidth Estimation Meeting San Diego June 2002 R. Hughes-Jones Manchester

  16. UDPmon Loss Throughput Mbit/s MAN – SARA Iperf Throughput Mbit/s UCL – SARA 262144 byte buffer Geant Enabled Routing Stable iperf & UDPmon throughput UCL-SARA From 1 Nov 01 – Geant Operational CAIDA Bandwidth Estimation Meeting San Diego June 2002 R. Hughes-Jones Manchester

  17. More Information Some URLs • DataGrid WP7 Networking: http://www.gridpp.ac.uk/wp7/index.html • DataGrid MapCentre http://ccwp7.in2p3.fr • Motherboard and NIC Tests: www.hep.man.ac.uk/~rich/net/nic/GigEth_tests_Boston.ppt • PPNCG Home page with Stop Press: http://ppncg.rl.ac.uk/ • PPNCG Page for monitoring Grid, HEP & Astronomy Sites http://icfamon.dl.ac.uk/ppncg/astronomy.html • IEPM PingER home site: http://www-iepm.slac.stanford.edu/ • IEPM-BW site: • http://www-iepm.slac.stanford.edu/bw CAIDA Bandwidth Estimation Meeting San Diego June 2002 R. Hughes-Jones Manchester

  18. MB - NG E-science core project • Project to investigate and pilot: • end-to-end traffic engineering and management over multiple administrative domains – MPLS in core diffserv at the edges. • Managed bandwidth and Quality-of-Service provision. (Robin T) • High performance high bandwidth data transfers. (Richard HJ) • Demonstrate end-to-end network services to CERN using Dante EU-DataGrid and to the US DataTAG. • Partners:CISCO, CLRC, Manchester, UCL, UKERNA plus Lancaster and Southampton (IPv6) • Status: • Project is running with people in post at Manchester and UCL. • Project Tasks have been defined and Detailed planning in progress • Kit list for the routers given to Cisco • Test PC ordered • UKERNA organising core network and access links – SJ4 10Gbit upgrade • Strong Links with GGF CAIDA Bandwidth Estimation Meeting San Diego June 2002 R. Hughes-Jones Manchester

  19. RAL OSM-4GE-WAN-GBIC MCC OSM-4GE-WAN-GBIC OSM-1OC48-POS-SS RAL OSM-4GE-WAN-GBIC OSM-1OC48-POS-SS UCL OSM-4GE-WAN-GBIC OSM-1OC48-POS-SS SuperJANET4 Production Network MB - NG UCL OSM-4GE-WAN-GBIC MB – NG SuperJANET4 Development Network (22 Mar 02) WorldCom SJ4 Dev C-PoP Warrington 12416 OC48/POS-SR-SC MAN OSM-4GE-WAN-GBIC Leeds Gigabit Ethernet 2.5 Gbit POS Access 2.5 Gbit POS core MPLS Admin. Domains SJ4 Dev ULCC 12016 SJ4 Dev C-PoP Reading 12416 SJ4 Dev C-PoP London 12416 OC48/POS-SR-SC OC48/POS-LR-SC ULCC WorldCom WorldCom Dark Fiber (SSE) POS CAIDA Bandwidth Estimation Meeting San Diego June 2002 R. Hughes-Jones Manchester

  20. The EU DataTAG project • EU Transatlantic Girid project. • Status: Well under way – People in post, Link expected Jul 02 • Partners: CERN/PPARC/INFN/UvA. IN2P3 sub-contractor • US Partners: Caletch, ESnet, Abilene, PPDG, iVDGL … • The main foci are: • Grid Network Research including: • Provisioning (CERN) • Investigations of high performance data transport (PPARC) • End-to-end inter-domain QoS + BW / network resource reservation • Bulk data transfer and monitoring (UvA) • Interoperability between Grids in Europe and the US • PPDG, GriPhyN, DTF, iVDGL (USA) CAIDA Bandwidth Estimation Meeting San Diego June 2002 R. Hughes-Jones Manchester

  21. NL SURFnet Fermi SLAC UK Abilene ESNET SuperJANET4 IT GARR-B Alcatel Alcatel GEANT Juniper Light Switch Giga Switch Juniper Light Switch Giga Switch Cisco 6509 Cisco Cisco Juniper Juniper Starlight (Chicago) 2.5 Gbit PoS lambda CERN (Geneva) DataTAG Possible Configuration multi-platform multi-vendor CAIDA Bandwidth Estimation Meeting San Diego June 2002 R. Hughes-Jones Manchester

More Related