GEC16 Plenary Session: GENI Solicitation 4 Tool Context

1. GEC16 Plenary Session: GENI Solicitation 4 Tool Context Marshall Brinn, GPO March 20, 2013

2. Overview • GENI Solicitation 4 seeks tools to help researchers use GENI to perform their experiments • This presentation outlines the essentials of the software context in which such tools will operate • Resources • Services • Interfaces

3. GENI Software Context Review Federation: A collection of people and institutions who agree to share resources and abide by common procedures in order to share resources in a reliable, mutually beneficial manner. Operations Center: Processes and tools monitoring activity on GENI resources for adherence to policies. Clearinghouse: Set of services establishing federation-level authentication, authorization and accountability of experimenter use of federation resources. Tools: Software capabilities that interact with federation resources on behalf of experimenters Aggregates: Software entities that represent federated resources in transactions with experimenter tools. Resources: Physical resources (compute, network, storage) made available to the federation by means of a participating aggregate. Experimenter: A researcher seeking to perform network experiments on customized data plane. Grey boxes are real-world entities, represented in software by Purple boxes.

4. Tools: Adding Value on Top of GENI Services • We expect tools will help operators work with GENI by providing value-added services such as: • Visualization: Allowing for visualizing the state of the network, resources or experiment • Aggregation: Combining information that is held in separate resources or services to provide convenient “macro” views. • Streamlining: Making it easier to formulate and submit complex resource or orchestration requests • Monitoring: Performing background checks and generating regular reports and alerts (as needed) • Uniformity: Presenting different underlying services or API’s in uniform manner Such value-added services are built on top of the basis of existing GENI services

5. Tools for Experimenter Support Tools to help experimenters will tend to contribute to one or more of these categories. • Topology: Construct a custom data plane on which to build and run an experiment • Browse: Determine what resources are available and what their capabilities are • Allocate: Construct a data plane topology from available resources • Program: Install/configure software to support my experiment • Experiment: Deploy and run an experiment and capture and analyze results. • Instrument: Establish points at which to collect different metrics of system performance • Orchestrate: Manage the distributed start/stop/pause steps of experiment operations • Monitor: Collect measurements on the running experiment • Analyze: Gather and analyze data collected from experiment

6. GENI Services: Overview • Authentication: GENI uses SSL certificates signed by a GENI trust root. • Anything trusted by GENI (directly or indirectly) may have such a certificate • GENI Tools (notably the GPO Clearinghouse Portal) support single-sign-on via credentials supplied by Shibboleth InCommon • Tools may want to be constructed to fit into such a single-sign-on environment. • We also support SSO using OpenID credentials.

7. Topology Services: Details • GENI Resources are presented by Aggregates which speak the AM (Aggregate Manager) API • http://groups.geni.net/geni/wiki/GAPI_AM_API_V3 • Client/Resource negotiation represented by three kinds of RSpecs (Resource Specifications): • Advertisement: What do you provide? • Request: What does the client need/want? • Manifest: What was actually allocated to client? • Include non-compute aggregates such as ION, FOAM and WiMAX • Including GMOC, Slice Authorities GENI services are advertised in the federation Clearinghouse Service Registry

8. Experiment Services: Details • There are two families of Experiment Support tools: • GIMI: http://groups.geni.net/geni/wiki/GIMI • Based on OML (ORBIT Measurement Library) and iRODS data grid repository • GEMINI: http://groups.geni.net/geni/wiki/GEMINI • Based on PerfSONAR, integrating other key GENI I&M projects including LAMP and INSTOOLS • There are archiving services (e.g. iRODS), topology services (e.g. UNIS) • Each provides mechanisms for instrumenting, capturing and archiving experiment measurements

9. Challenges and Caveats • There are variations among implementations of AM API: • Different versions (V1, V2, V3) • Different RSpec extensions • Different compute environments • There are two independent families of experiment support tools: • Different orchestration mechanisms • Different instrumentation and collection protocols One theme you will hear at this GEC is “Unified Experimenter Experience” which attempts bridge these gaps

10. Summary • There is a significant need and opportunity to develop tools to help experimenters • There are a lot of services that exist from which useful tools can be constructed • These services are not as uniform as we'd like: we're trying to unify them as much as possible, but it may fall on the tool developer to bridge some of these gaps, at least in the short term. Consult the GENI WIKI http://groups.geni.net/geni for details on the essential GENI services

11. Notes / Backup

12. NOTES • GMOC • XMLRPC and Signed Docs • Service Registry and other CH functions • Show CH diagram • AM API • Advertisement Rspec (images, VM’s, H/W) • Slice Authorities • IDP’s • Single-Sign-On and Shib • Resource Allocation • Resource Measurement Repositories/Directories

13. Notes • Credentials for common authorization • InCommon/OpenID for SSO • GMOC • OpenFlow switches • Non-compute aggregates: FOAM, ION, WiMAX • L2 vs L3 • Topology information available on WIKI pages • What services are there? What are their attributes? How do you talk to them? • Tools are wrappers around services that make them easier to use (wrapping, caching, joining, pre-computing) • What do the services provide? What does the experiment want? Tools can bridge the gap.