1 / 58

perfSONAR Information Discovery

February 11 th 2010, APAN 29 – perfSONAR Workshop Jeff Boote, Assistant Director R&D. perfSONAR Information Discovery. Outline. Motivation Information Services Architecture Global Lookup Service Home Lookup Service Topology LS Operations Registration Discovery Synchronization

debbiehall
Download Presentation

perfSONAR Information Discovery

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. February 11th 2010, APAN 29 – perfSONAR Workshop Jeff Boote, Assistant Director R&D perfSONAR Information Discovery

  2. Outline • Motivation • Information Services Architecture • Global Lookup Service • Home Lookup Service • Topology • LS Operations • Registration • Discovery • Synchronization • Protocol

  3. Motivation • Network measurements are useful for a local domain, but are also pivotal in solving end to end problems. • Many network problems are found to be at the demarcation of a network • Most forms of communication must cross outside of the home domain • Each domain should strive to make historical data available • À la carte operation frees network staff from special requests • Data availability will drive the production of automatic tools • Finding specific information can be challenging • Finding a specific type of measurement • Finding data for a given domain or IP subnet • Finding data classified by a VO or community

  4. Motivation • The perfSONAR information services were designed to record the location of data categorized on the following criteria: • Data Type, e.g. by classification such as latency or bandwidth. Or by tool such as ping or iperf • Domain name • Topology reference (For L3: IP Address (v4 or v6)) • Keyword, e.g. “tagging”. Users may assign arbitrary metadata to measurement data to signify membership in a VO (USATLAS, eVLBI) or other forms of association (Internet2, DOE). • Services are autonomous – they know to register to some (any close) information service. • Information services are autonomous as well – they are self organizing to distribute information.

  5. Information Services Architecture • The architecture of the perfSONAR Information Services consists of the following components: • Global Lookup Service • Home Lookup Service • Topology • Clients and Services • Each component is designed in the SOA fashion to do one specific job. • All components speak the same XML protocols for interoperability

  6. Information Services Architecture

  7. Information Services Architecture

  8. Information Services Architecture

  9. Information Services Architecture

  10. Information Services Architecture

  11. Information Services Architecture

  12. Information Services Architecture

  13. Global Lookup Service • The Global Lookup Service is a federation of individual instances • Recommended that the total number be kept small and geographically placed • Form an ‘information cloud’ • Each will synchronize with others to exchange information • Each server is a peer, and within a set amount of time knows what other servers know • To “bootstrap” where to find the gLS services, a ‘hints’ file is used • Similar to DNS • Current location: http://www.perfsonar.net/gls.root.hints • Each hints file = a new cloud, can be many at any given time

  14. Global Lookup Service • The gLS performs periodic operations: • Synchronizing with others • Summarizing internal data • Cleaning the databases of expired data • Data has a TTL – it is automatically cleaned if not renewed • The gLS maintains 2 databases: • ‘Registered’ information – this comes from hLSs • ‘Summarized’ information – this is a combination of everything registered • Query Types are structured, or ad-hoc • XQuery interface to arbitrarily query the databases • Well formed query description for ‘discovery’ need • Domains • IP Addresses • Data Type • Keywords

  15. Global Lookup Service • gLS servers only accept registrations from hLS servers • MA/MP etc. services do not register to the gLS • gLS servers respond to the following messages: • LSRegistrationRequest • Register data from an hLS • LSDeRegistrationRequest • Deregister data for an hLS • LSKeyRequest • Retrieve the registration key for an hLS • LSKeepaliveRequest • Renew a registration for an hLS • LSQueryRequest • Query the database of the gLS

  16. Home Lookup Service • The Home Lookup Service manages the registration information of the services • Services register their name/address • Services register all of the measurement data they maintain • Each domain should deploy an hLS • Use multiple hLSs if many services are required (e.g. load on hLS will dictate splitting – recommended < 10 services per hLS) • hLSs will contact a ‘close’ gLS by using the hints file • hLSs may contact multiple gLSs if desired (gLS synchronization does not make this a requirement)

  17. Home Lookup Service • The hLS performs periodic operations: • Synchronizing with others • Summarizing internal data • Cleaning the databases of expired data • Data has a TTL – it is automatically cleaned if not renewed • The hLS maintains 2 databases: • ‘Registered’ information – this comes from services • ‘Summarized’ information – this is a combination of everything registered • Query Types are structured, or ad-hoc • XQuery interface to arbitrarily query the databases • Well formed query description for ‘discovery’ need • Domains • IP Addresses • Data Type • Keywords

  18. Home Lookup Service • hLS servers only accept registrations from services • hLS servers respond to the following messages: • LSRegistrationRequest • Register data from a service • LSDeRegistrationRequest • Deregister data for a service • LSKeyRequest • Retrieve the registration key for a service • LSKeepaliveRequest • Renew a registration for a service • LSQueryRequest • Query the database of the hLS

  19. perfSONAR Information Discovery February 11th 2010, APAN 29 – perfSONAR Workshop Jeff Boote, Assistant Director R&D For more information, visit psps.perfsonar.net

  20. Topology Service • The topology service stores and normalizes network topology • XML format – similar to measurements • Each topology element features IDs and IDrefs to link state • Supported elements: • domain • network • node • port • link • path • service

  21. Topology Service • Topology descriptions can be entered in two ways: • By hand via configuration files • Individual elements registered by services • Queries are supported: • XQuery interface to answer arbitrary topology questions • The TS registers the domains it has control over to the LS infrastructure.

  22. Topology Service • hLS servers respond to the following messages: • TSQueryRequest • Query the TS for information • TSAddRequest • Add new information to the TS • TSUpdateRequest • Update the time to live for existing information in the TS • TSReplaceRequest • Replace existing information in the TS

  23. LS Operations • The following operations will be described • Registration • Discovery • Syncronization • Each operation may involve different actors • hLSs • gLSs • Services • Clients • There is a time sensitivity to each operation, and it may be performed periodically.

  24. LS Operations - Registration • There are 2 Types of Registration: • Service to hLS registration • hLS to gLS registration • Services will register: • Service name, contact information • All measurement “Metadata”, e.g. the description of each measurement data set. • hLSs will register: • Service name, contact information • Summarized data for each registered service (e.g. all “Metadata” for a given service will be summarized into a set format)

  25. LS Operations – Service Registration

  26. LS Operations – Service Registration

  27. LS Operations – hLS Registration

  28. LS Operations – hLS Registration

  29. LS Operations - Registration • Registration accomplishes 2 goals: • Services identify themselves into the LS infrastructure • Services ‘renew’ the registration, e.g. establish themselves as being active • Other messages go hand in hand with registration: • DeRegistration • Remove an entire registration • Remove data from the registered data set • Keepalive • Renew a registration • KeyRequest • Request a key for a registered data set

  30. LS Operations - Discovery • Discovery is the process of locating registered information • Discovery is normally a two part process: • Find who to ask the question • Ask the question • The first layer in the discovery process is the gLS: • gLS maintains a list of all hLSs and their summaries • Can answer which hLS has more information for a given discovery query • The second layer is the hLS: • hLS maintains a list of some services and their summaries • Can answer which service has more information for a given discovery query

  31. LS Operations - Discovery • Typical data request: • “I want SNMP data for the Chicago -> New York link on the Internet2 backbone” • Breaking down the components: • SNMP Data • Internet2 Domain • Specific interfaces (OC192 between Chicago and New York) • How to ask the question to the hLS/gLS: • Who has SNMP Data for the Internet2 Network? • The answer from each will be different: • From gLS: which hLS(s) to ask • From the hLS: which service(s) to ask

  32. LS Operations - Discovery

  33. LS Operations - Discovery

  34. LS Operations - Discovery

  35. LS Operations - Discovery

  36. LS Operations - Discovery

  37. LS Operations - Discovery

  38. LS Operations - Discovery • hLSs and gLSs accept the same kinds of query message • XQuery • Allows complex use of this language to request almost anything about the data • Discovery • Structured to limit the querying to specific elements • Domains • IP Ranges • Data Types • Keywords • Full recursion is possible with the query model, but not currently supported.

  39. LS Operations – Synchronization • gLS servers are independent yet federated • Any domain can deploy one • Services may register to any gLS • gLSs exchange data sets with each other, and ‘register’ things they may not know about • Synchronization relies on: • Hints file containing which gLSs participate in a “cloud” • gLSs maintaining a database of the services that have registered directly with them (e.g. “authoritative” registration source for a given service) • gLSs running a periodic “broadcast” to all other gLSs containing the hLS instances they are authoritative for. Other gLSs will then choose to either register or ignore this information.

  40. LS Operations – Synchronization

  41. Protocol • There are 5 Major messages in the gLS/hLS Protocol • LSRegistrationRequest • LSDeregistrationRequest • LSKeepaliveRequest • LSQueyRequest • LSKeyRequest • Each is based on the standard perfSONAR message format: • Message container • At least one metadata element • At least one data element, linked to the metadata element

  42. Protocol - LSRegister • LSRegister(Request/Response) • Register content into the hLS (or gLS) • Metadata(s) = Service Structure • Data(s) = Metadata associated with service • Different operations based on context • ‘New’ – If the service is not in the LS, it is added to the LS as a new entity • ‘Update’ – Use the ‘key’ to add new items to the LS • ‘Clobber’ – Send the key and changes to the service element (N.B. this nukes existing dataset)

  43. Protocol - LSRegister

  44. Protocol - LSRegister

  45. Protocol - LSRegister

  46. Protocol - LSDeregister • LSDeregister(Request/Response) • Remove content from the hLS (or gLS) • Metadata = Key for a service • Data = nothing, or specific service metadata • ‘nothing’ case is a data trigger • ‘Selective’ case is a pS-PS addition (EU *LS code does not support this)

  47. Protocol - LSDeregister

  48. Protocol - LSDeregister

  49. Protocol - LSKeepalive • LSKeepalive(Request/Response) • Updates the time in the control structure to allow this service to still live in the hLS (or gLS) • Metadata = key • Data = nothing (trigger)

  50. Protocol - LSKeepalive

More Related