Installing and Configuring the perfSONAR Services

1. Installing and Configuring the perfSONAR Services

2. COURSE OBJECTIVES • By the end of this course you will be able to: • Describe key perfSONAR services. • Install, configure and deploy the following perfSONAR services: • The Lookup Service • The BWCTL Measurement Point • The SSH / Telnet Measurement Point • The RRD Measurement Archive • The SQL Measurement Archive • Identify how to interact with the Service Desk.

3. THE PERFSONAR ADMINISTRATION GUIDE • Please note that these slides do not contain detailed instructions about how to install the perfSONAR services. • For detailed instructions, please refer to the perfSONAR Administration Guide.

4. COURSE OUTLINE • Module 1 – perfSONAR Technical Overview • Module 2 – General Installation Considerations • Module 3 – Installing and Configuring the Lookup Service • Module 4 – Installing and Configuring the BWCTL MP • Module 5 – Installing and Configuring the SSH / Telnet MP • Module 6 – The Service Desk • Module 7 – Installing and Configuring the RRD MA • Module 8 – Carrying out an Installation on Debian • Module 9 – Installing and Configuring the SQL MA • Module 10 – Feedback on perfSONAR Installation and Configuration

5. MODULE 1: perfSONAR TECHNICAL OVERVIEW

6. WHAT IS PERFSONAR? • perfSONAR is: • A project consisting of a variety of organisations and individuals • A set of protocols that: • Assume a set of services based on defined roles. • Define their communication syntax and semantics. • Allow anyone to develop an implementation of a service. • A set of code • Service implementations

7. WHAT ARE THE PERFSONAR SERVICES? • The perfSONAR services form an interoperable, distributed performance measurement middleware framework. • perfSONAR stands for PERformance Service Oriented Network monitoring Architecture.

8. DESIGN OBJECTIVES • perfSONAR is designed to be: • Decentralised and Scaleable • Large number of networks and services, large volume of data • Each domain can set its own security policy • Dynamic and ‘Self-Discovering’ • Add and remove components during operation • Components ‘automatically’ become aware of one another • Secure • Will not put participating networks at risk of attack or congest them • Modular • Allows discrete module development

9. THREE-TIER ARCHITECTURE • The perfSONAR framework: • Is middleware. • Is distributed between domains. • Facilitates inter-domain performance information sharing. • perfSONAR services ‘wrap’ existing measurement tools.

10. WHAT IS A SERVICE? • A Service is a tightly defined, independent entity that has a well defined interface and can be accessed directly.

11. ARCHITECTURAL PRINCIPLES • Postulate: all measurement systems contain a combination of: • Measurement tools • Data Storage • Security and Policy implementation • Topology information • Visualization • Services have been identified that are: • Based on these ‘roles’ • Based on requirement for other functionality such as • Service discovery, resource protection, data formatting

12. perfSONAR divides measurement system tools into generic ‘families’ or ‘services’. Each service has a protocol. THE PERFSONAR SERVICES FRAMEWORK (1)

13. THE PERFSONAR SERVICES FRAMEWORK (2) Client Authentication Service Lookup Service Transformation Service Measurement Point Service Measurement Archive Service Resource Protector Service

14. THE CLIENT Client Authentication Service Lookup Service Transformation Service For example the perfSONAR Visualisation User Interface Measurement Point Service Measurement Archive Service Resource Protector Service

15. THE LOOKUP SERVICE Client Authentication Service Lookup Service Transformation Service Registers services including their capabilities. Facilitates complex searches. Participates in network of Lookup Services Measurement Point Service Measurement Archive Service Resource Protector Service

16. THE LOOKUP SERVICE (1) • Purpose: all other services must register with the lookup service in order to participate in the framework.

17. THE LOOKUP SERVICE (2) • Other services (including measurement points) register their existence with a lookup service, by delivering ‘lookup information’: • Location • Type of Service • Each domain has an instance of the lookup service • These instances (will) communicate with one another • Clients find other services by querying the lookup service. • All the client needs to know is the URL of a Lookup Service

18. THE AUTHENTICATION SERVICE Client Authentication Service Lookup Service Transformation Service Provides authentication for clients and protects privacy. Can be federated. Measurement Point Service Measurement Archive Service Resource Protector Service

19. THE AUTHENTICATION SERVICE • Purpose: ensures client-privacy and domain security by using role-based authentication and authorisation.

20. THE MEASUREMENT POINT SERVICE Client Exposes measurement tools and publishes their data. Authentication Service Lookup Service Transformation Service Measurement Point Service Measurement Archive Service Resource Protector Service

21. MEASUREMENT POINTS • Measurement Points: • Belong to domains (domain = a network) • Each measurement point implementation maps to a tool that provides one or several metrics • Examples: • One-way-loss • Jitter • TCP throughput • Show commands on routers

22. THE MEASUREMENT ARCHIVE SERVICE Client Authentication Service Lookup Service Exposes measurement databases and file stores. Publishes measurement data Avoids queries to multiple Measurement Point Services Transformation Service Measurement Point Service Measurement Archive Service Resource Protector Service

23. MEASUREMENT ARCHIVES • Purpose: expose measurement data held in databases or file systems. • Wrapper for any type of storage mechanism (SQL Databases, RRD files, etc) • Provides access to recent and stored data • Collects information from Measurement Points, Transformation Services or other Measurement Archives. • i.e. it ‘subscribes’ to these other services • Can also write to databases and file systems

24. THE TRANSFORMATION SERVICE Client Transforms data in a variety of ways (e.g., aggregation, filtering, correlation). For future development. Precise role needs to be defined. Authentication Service Lookup Service Transformation Service Measurement Point Service Measurement Archive Service Resource Protector Service

25. THE RESOURCE PROTECTOR SERVICE Controls the comsumption of limited resources (e.g. network bandwidth). Client Authentication Service Lookup Service Transformation Service Measurement Point Service Measurement Archive Service Resource Protector Service

26. THE PERFSONAR SERVICES FRAMEWORK (3) • Each service has a specific function. • Each instance of a service belongs to an administrative domain.

27. PROTOCOLS • perfSONAR has developed a set of protocols for sharing performance data. These: • Assume the services set out in the framework. • Define their communication syntax (schema) and semantics (business logic). • Allow anyone to develop an implementation of a defined service. • Are compliant with the Global Grid Forum’s Network Measurement Working Group (NM-WG) schema specification. • Are based on XML over SOAP. • You can see the protocols as ‘rules and tools’ for participating in the perfSONAR framework.

28. GENERIC SERVICES AND SERVICE IMPLEMENTATIONS

29. SERVICE IMPLENTATIONS • The perfSONAR project has also developed a set of service implementations that use the defined protocols. • Some of these have been developed by JRA1, within the GEANT2 project for the perfSONAR pilot: • The Lookup Service • The BWCTL Measurement Point • The SSH / Telnet Measurement Point • The RRD Measurement Archive • The SQL Measurement Archive

30. DATA COLLECTION, NORMALISATION AND SHARING VIA THE FRAMEWORK

31. FRAMEWORK FACILITATES NORMALISED END-TO-END PERFORMANCE DATA (SIMPLIFIED DEPICTION)

32. THE PERFSONAR PILOT: MEASUREMENT AND ARCHIVING SERVICES

33. THE PERFSONAR PILOT

34. THE ROADMAP • Enhanced Lookup Service: • Lookup Services (ideally one per domain) peer with one another for increased ease of data discovery. • Authentication Service: • Collaborating with JRA5 and Edugain to produce an implementation of the service. • A data subscriber (client) and a data producer (server) can communicate directly across domains provided that they are authorised to do so. • Further service implementations using Netflow information or packet capture features

35. TECHNICAL OVERVIEW: SUMMARY • perfSONAR will provide an infrastructure to: • Locate data sources • Authenticate and authorise clients • Protect resources and ration their usage • Retrieve, normalise, transform and share data • Only a partial infrastructure is implemented in the pilot • perfSONAR is a flexible and open framework: • perfSONAR services can ‘wrap around’ existing data collection tools

36. THE PERFSONAR SERVICES FRAMEWORK - RECAP • How do we expose measurement tools? • Measurement point services • How do we expose measurement databases and file stores? • Measurement archive services • How do we transform data (aggregate, correlate, filter etc.)? • Transformation services • How do we locate all these services and their capabilities? • Lookup Services • How do we protect resources? • Resource Protection Services • How do we ensure a client is allowed to access a service? • Authorisation and Authentication Services

37. MODULE 2 – GENERAL INSTALLATION CONSIDERATIONS

38. GENERAL INSTALLATION PREREQUISITES (1) • All of the perfSONAR services require the following: • Recommended operating system: Redhat Linux / Fedora. • You can install on other platforms, but perfSONAR has not been tested on these • Installations on Windows are not supported • ‘wget’ command must be available on the OS • Required by installers to download software • Perl module: LWP • Perl version 5.6.1 or higher • These prerequisites must be manually installed before you begin installation of the perfSONAR services.

39. GENERAL INSTALLATION PREREQUISITES (2) • All of the perfSONAR services except for the BWCTL Measurement Point (a non-Java application) require the following: • Java Developers’ Kit (JDK) version 1.5 or higher • Already installed for you on the training server • Note that the RRD MA must have JDK version 1.5 (not any other) • Apache Ant 1.6.x • Tomcat application server – Jakarta Tomcat • These prerequisites must be manually installed before you begin installation of the perfSONAR services. • Note: Tomcat can be automatically installed by the bundle installer, but it recommended that you download Tomcat from the Apache web-site and manually install it before running the bundle installer.

40. OTHER INSTALLATION PREREQUISITES • In addition to the General Installation prerequisites, each service has one or more prerequisites that are specific to it. • Some must be installed manually by you before you begin to install a service: • These are listed in subsequent course modules and are also documented in the Installation Manual. • Some can be installed automatically by the perfSONAR Bundle Installer: • These are documented in the perfSONAR Administration Guide.

41. THE ROLE OF THE ADMINISTRATION GUIDE • The perfSONAR Administration Guide: • Lists all pre-requisite software. • Provides step-by-step instructions explaining how to install the perfSONAR services. • Will be used extensively during this course. • Will be handed out to you in hard copy. • Should be used as your guide when you are installing the services on your own servers. • During the exercises, please point out any inconsistencies or errors in the guide to your trainer.

42. THE ROLE OF THE SERVICE DESK • The Service Desk is being set up as a single point of contact for all issues relating to perfSONAR installation, configuration and use. • More information about the service desk is provided in module 6 of the course.

43. INSTALLATION STEPS • To install the perfSONAR services: • Unzip and untar the bundle installer (perfSONAR-2.0.tar.gz) • Empty the CLASSPATH variable. • Execute the bundle installer • Follow the on-screen dialogue • For detailed instructions, refer to the perfSONAR Administration Guide.

44. HOW DOES THE BUNDLE INSTALLER WORK? • The bundle installer will ask you whether you want to: • Install a new service • Modify or test an existing service • Give feedback to the perfSONAR team • If you choose to install a new service: • The installer will ask you to choose from the six available services • It will then: • Automatically download the appropriate installation files and execute the installation steps in sequence • Ask you ‘interactive questions’ about the installation

45. CONFIGURING THE SERVICES – STITCHING • Before they can be used, several of the perfSONAR service implementations require a type of configuration known as ‘stitching’. • Stitching: • Is the process of configuring metadata for your service. • Metadata is data that describes other data. • E.g. data units, interface name, direction of traffic etc. • Usually involves the creation of a ‘metadata configuration file’.

46. STITCHING IN CONTEXT (1) Stitching allows a service to understand the data it is dealing with. Examples: Which router? Inbound or outbound traffic? What data units? Etc.

47. STITCHING IN CONTEXT (2) Stitching allows a service to understand the data it is dealing with. Examples: Which link? Which interfaces? Location of interfaces? Etc.

48. THE ADVANTAGES OF STITCHING • Ultimately, stitched metadata tells the perfSONAR framework precisely what kind of data your service sends or can receive. • Why this approach? • The ability to perform stitching makes service implementations flexible. • E.g. the RRD Measurement Archive can handle data from RRD files in different networks that are structured in different ways. • Stitching allows service implementations to deal with multiple ‘flavours’ of the same kind of data. • E.g. a single instance of the SQL Measurement Archive service can deal with both inbound and outbound traffic since you can mark each data source as providing information about either inbound or outbound traffic.

49. OTHER CONSIDERATIONS • If you are installing multiple services in the same Tomcat instance, using the same eXist XML database (as in training): • Each service requiring XML database access must have a different username and password • You must ensure that each service is allocated a unique name for its own collection • Must be manually created for the Lookup service • Ensure that you only install one instance of ant on your machine • Putting multiple ant installations in the same path causes problems

50. BACKUPS • Once you have successfully installed and configured a service, back it up. • Backup the ‘webapps’ directory related to the service • Located by default within your perfSONAR bundle directory • Can be used to restore the service • Backup your metadata configuration files