SLAC monitoring Web Services

1. SLAC monitoring Web Services Paola Grosso SLAC Networking Services ESCC Jan 28-29 2004 IEPM Web Services

2. Outline of presentation • What are Web Services? • SOAP • XMLRPC • NM-WG (Network Measurement Working Group) schema • SLAC Web Services: IEPM implementation • Present and future collaborations and project evolution IEPM Web Services

3. Web Services “A web service is any service that is available over the Internet, uses a standardized XML messaging system, and is not tied to any one operating system or programming language” IEPM Web Services

4. XML XML = eXtensible Markup Language: • It’s designed to structure, store and send information; • Tags and structures are used-definable; • Conversion of data to XML allows exchange of data in previously incompatible formats; • XML data is in flat files, so that it’s software and hardware independent. IEPM Web Services

5. XML-RPC, SOAP and WSDL XML-RPC: • Implements remote procedure calls over HTTP; • It’s simple to use (i.e does not have advanced features as SOAP). Simple Object Access Protocol (SOAP): • It’s a communication protocol based on XML; • It’s platform and language independent. Web Services Description Language (WSDL): • It’s an XML document; • Used to describe and locate Web Services. IEPM Web Services

6. GGF Network Measurements Working Group The group worked on: defining standards for describing network measurements taken in Grid environments, so to allow exchange of measurements with a common “dictionary”. http://www-didc.lbl.gov/NMWG/docs/draft-ggf-nmwg- hierarchy-01.pdf Grid services need to know: 1 what and where are the best source/s to copy data from; 2 if parallel streams need to be used, and how many; 3 what are the ideal TCP window and buffersizes. IEPM Web Services

7. NM-WG schemas • A “request” schema, defining how the measurements can be requested to the server: http://www.hep.ucl.ac.uk/~pdm/nmwg/2004-01-08-Dante-NM-WG-RequestSchema2.xsd • A “response” schema, defining how the network measurements have to handed back to the requester/client: http://www-didc.lbl.gov/NMWG/schemas/NM-WG_Schemas_for_OGSI.html IEPM Web Services

8. A request NetworkMeasurementRequest Subject Time Characteristics Methodology The network entity for which a measurement is requested, i.e a router, a node, a path The time period for which we are retrieving a measurement. The name of the characteristic requested The methodology requested http://www.hep.ucl.ac.uk/~pdm/nmwg/2004-01-08-Dante-NM-WG-RequestSchema2-examples/Dante-NM-WG-RequestSchema2-demo-NMWG1.xml IEPM Web Services

9. A response NetworkMeasurement Characteristics Subject Methodology Results The name of the network property that has been measured. The host and The networks That are being tested. What test was run, and how it was run, in order to collect the measurement. The actual value of the measurement IEPM Web Services

10. DelayType BandwidthAvailableType PathLossType BadwidthAchievableType PathDelayJitterType BandwidthType PathReorderingType A response result A response can contain results of type: Information on the network bandwidth Information on the path characteristics http://www-didc.lbl.gov/NMWG/schemas/response_Jan2004/sample.xml IEPM Web Services

11. SLAC WSDL file The description of the available service is in the SLAC IEPM WSDL file: http://www-iepm.slac.stanford.edu/tools/soap/wsdl/IEPM_profile.wsdl The document provides: • Operation that can be performed GetBandwidthAchievableTCP • Binding of the operation to an action IEPM#GetBandwidthAchievableTCP • Definition of input and output parameters Target and time BandwidthAchievableType IEPM Web Services

12. IEPM group at SLAC “The IEPM group at SLAC is monitoring network connectivity and End-to-end performance for sites involved in High Energy Nuclear and Particle Physics” IEPM Bandwidth to the World monitors ~35 sites around the world and analyzes the data to determine variations and anomalies http://www-iepm.slac.stanford.edu/index.html IEPM Web Services

13. IEPM BW tests The tests that are periodically running are: • Ping and traceroutes • Iperf = Single and multistream • BBftp = Multistream file transfer software, used at SLAC to move physics data offisite • ABwE = the Available Bandwidth Estimator (developed by J. Navratil) • GridFTP = Data transfer protocol optimized for WANs IEPM Web Services

14. IEPM BW data The data is available on the Web: http://www.slac.stanford.edu/comp/net/bandwidthtests/antonia/html/slac_wan_bw_tests.html For Web Services we started to store the data in an Oracle database: • Easier retrieval than compared to flat files • Easier data correlation in complex queries IEPM Web Services

15. SLAC Web Services servers We have two servers running at SLAC: • SOAP server • XML-RPC server Implementation is fairly similar. Both servers dispatch incoming requests from clients to a Perl module (i.e IEPM): The Perl module connects to the database back-end where the data is stored. XML-RPC #!/usr/local/bin/perl use SOAP::Transport::HTTP; SOAP::Transport::HTTP::CGI -> dispatch_to('IEPM') -> handle; #!/usr/local/bin/perl use XMLRPC::Transport::HTTP; XMLRPC::Transport::HTTP::CGI -> dispatch_to('IEPM') -> handle; SOAP IEPM Web Services

16. SOAP client The client can contact the server to extract data, providing: • End node name • Time period (start-end) • Method name #!/usr/local/bin/perl -w use SOAP::Lite; my $node = "node1.cacr.caltech.edu"; my $timePeriod="20031201-20031205T143000"; my $measurement = SOAP::Lite ->service('http://www-iepm.slac.stanford.edu/tools/soap/wsdl/IEPM_profile.wsdl') ->GetBandwidthAchievableTCP("$node", "$timePeriod"); IEPM Web Services

17. XML-RPC client Works in the same way as the SOAP client, with the same input parameters and few modication to the calling code: #!/usr/local/bin/perl -w use XMLRPC::Lite; my $node = "node1.cacr.caltech.edu"; my $timePeriod ="20031201-20031205T143000"; my $toolName = "iperf"; my $streams = 8; my $measurement = XMLRPC::Lite -> proxy('http://www-iepm.slac.stanford.edu/cgi-wrap/advisor.cgi’)-> call('IEPM.GetBandwidthAchievableTCPMultiStream',"$node", "$timePeriod","$toolName", "$streams") ->result; IEPM Web Services

18. IEPM “canned” client To allow people to see the available data, without having to code their own client, we provide a public client interface to our Web Services: http://www-iepm.slac.stanford.edu/tools/soap/IEPM_client.html Data can be: Printed Plotted Time periods can be: User Specified Pre-defined: 24 hours 7 days 30 days IEPM Web Services

19. Collaborating Need somebody else at the other end, a consumer of the data… So far: • FNAL IEPM Web Services. They have implemented servers to serve the data collected there. (Maxim Grigoriev) • <add url to FNAL IEPM data) To come: • Monalisa interface to IEPM via web services. (Iosif LeGrand) • <add URL to Monalisa> IEPM Web Services

20. Future developments • Upgrade the service to use the most recent request/response schemas. • Provide access via Web Services to PingER data. • Integrate our measurements into the Advisor tool, for end-to-end performance debugging. • Work with Grid developers at SLAC to allow to use the service for best path determination. IEPM Web Services

21. Conclusion Questions and comments? The SLAC Web Services page: http://www-iepm.slac.stanford.edu/tools/web_services Our email address: iepm-l@slac.stanford.edu IEPM Web Services