GENI I&M Update: Gathering, Transferring and Sharing MD

1. GENI I&M Update:Gathering, Transferring and Sharing MD • Goals • Architecture Overview • Process • Functional Services • Measurement Data Flows/Transfers • Measurement Traffic Flows • Principal Use Cases • Types of Services • Control Framework Implications • Measurement Data Objects and Descriptors • Controlling Access to Measurement Data Objects • Current Status • Current I&M Design and Prototyping Efforts • I&M Service Deployments • References: • I&M work in progress: http://groups.geni.net/geni/wiki/GeniInstMeas • I&M capabilities catalog: http://groups.geni.net/geni/wiki/GENIIandMCAPCAT • I&M architecture document: http://groups.geni.net/geni/wiki/GeniInstrumentationandMeasurementsArchitecture • Available I&M tools: http://groups.geni.net/geni/wiki/GeniInstMeas#AvailableGENIIMTools

2. GENI I&M Architecture Overview:Measurement Data (MD) Flows/Transfers • MD packet flows and file transfers are carried from one I&M service to the next • “Stitching” is done as I&M services are setup and configured • Authorization mechanisms are required to protect MD • Flows/transfers follow a wide range of protocols and schema, including both pull and push methods

3. Measurement Data (MD) Flows/Transfers:Current Protocols and Schema Opt 3 Opt 1 • Current range of protocols and schema: • Option 1: Bytes of MD via SNMP • Option 2: File of MD via SCP, FTP or 3rd-party FTP • Option 3: XML-formatted MD via HTTP (e.g., perfSONAR) • Option 4: Tuples of MD via Custom OML Protocol • Option 5: Tuples of MD via IPFIX over SCTP • Option 6: NetCDF formatted files of data using LDM over TCP Opt 5 Opt 3 Opt 5 Opt 2 Opt 5 Opt 4 Opt 2 Opt 2

4. GENI I&M Architecture Overview:Principal Use Cases • Use Case 1: Experimenter gathering measurement data (MD) from their slice • Use Case 2: Operator (or Service Provider) gathering measurement data (MD) from GENI infrastructure • Use Case 3: Experimenter gathering measurement data (MD) from their slice and from GENI infrastructure • Use Case 4: Experimenter (or Operator) moving measurement data (MD) to an archive, with an option to share with other Researchers

5. Use Case 1: Experimenter gathering MD from their slice • Each I&M service is setup as part of the Experimenter’s slice • Includes: • Type 1 service: dedicated to a slice • Type 4 service: with portal to share MD • The Experimenter that collects the MD “owns” the MD, and sets the policy for how it can be stored or disposed

6. Use Case 2: Operator (or Service Provider) Gathering MD from GENI infrastructure • An Operator (or Service Provider) can gather MD from GENI infrastructure into their slice • For their own use • To share with other operators, e.g., share with GMOC • The Operator (or Service Provider) that collects the MD “owns” the MD, and sets the policy for how it can be stored or disposed, consistent with policy of Infrastructure and/or GENI • Includes: • Type 2 service: common service with multiple slivers • Type 3 service: common service with MD to multiple slices

7. Type 2 I&M Service:Common service with multiple slivers • Each Type 2 I&M service: • Common service, assembled, configured and managed by a Service Provider (or Operator) in their sliver using Control Framework (CF) mechanisms • Service Provider runs an AggMgr interface and offers slivers (e.g., MP service slivers, or UW service slivers, as shown above) to other Operators (or Experimenters) via CF mechanisms • Each sliver is a resource with an rspec • Example: Measurement System for packet capture from Univ Wisconsin • Example: User Workspace (UW) service prototype from CNRI

8. Type 3 I&M Service:Common service with MD to multiple slices • Each Type 3 I&M service: • Common service assembled, configured and managed by a Service Provider (or Operator) in their sliver using Control Framework (CF) mechanisms • Service Provider (or Operator) registers Descriptors of available MD Objects (e.g., MD flows, as shown above) with Measurement Information (MI) service • Example: perfSONAR MP and MI services • Operator (or Experimenter) finds and requests a flow from the I&M service • Authorization mechanism required to make flow available to selected operators/experimenters. • Obligated to follow policy established by MD “owner”

9. Use Case 3: Experimenter gathering MD from their slice and from GENI infrastructure • An Experimenter can gather MD from their slice and from GENI infrastructure • Combines Use Case 1 with Use Case 2 • Includes Type 2 service: common service with multiple slivers • Policy required to make sliver available to selected experimenters • Includes Type 3 service: common service with MD to multiple slices • Access mechanism required to make flow available to selected experimenters • Includes Type 1 service: dedicated to a slice • Configured to stitch flow into experimenter’s I&M service

10. GENI I&M Architecture Overview: Controlling Access to Measurement Data (MD) Objects • MD Objects have associated Measurement Data Object Descriptors (MDODs, commonly known as “metadata”) • Each MDOD typically includes policy elements, to specify how the MD Object can be shared or disposed. • The I&M service holding the MD Object is obligated to follow the policy for sharing • Measurement data flows and transfers between Type 1 services within a slice must be “stitched” as slice is setup • An access control mechanisms is required to prevent others from getting the MD • A Type 3 service can register measurement data for others to discover and share MD via a standardized interface • An access control mechanism is required to admit selected GENI experimenters and operators • It should reuse accepted certificate/credential arrangements • It must follow policy in MDOD associated with Digital Object • A Type 4 service can share measurement data via a web portal • An access control mechanism is required to admit selected GENI experimenters and operators • It should reuse accepted certificate/credential arrangements • A Digital Object Archive (DOA) service is required to make digital object available to selected users within GENI, selected users outside of GENI, and/or all others • It must follow policy in MDOD associated with Digital Object