GEC16 Service Developers Roundtable: Strawman Unified I&M Tools and Services

1. GEC16 Service Developers Roundtable:Strawman Unified I&M Tools and Services Marshall Brinn, GPO March 19, 2013

2. Overview • Goal: Provide researchers with an intuitive, powerful set of tools to support experimentation • Problem: We have two different families of tools (GIMI, GEMINI) that have different architectures, interfaces and are (currently) focused on different compute platforms • Solutions: • Make all services available on all platforms • Provide common ‘base’ interface that allows experimenters to manage experiments in a platform-agnostic manner

3. I&M Proposed Simple Common API # Let's keep it very simple. # Something like: # Open a session for storing metrics session_url <= open_session(session_name) # Close a session for storing metrics status <= close_session(session_url) # What are the metrics I can ask for on this platform? metric_names <= available_metrics() # What are the attributes for a given metric attributes <= metric_attrributes(metric_name) # Start metric capture # Specify attributes tailoring the nature of capture (frequency, detail e.g) metric_capture_id <= start_metric_capture(session_url, metric_name, attribs) # Stop metric capture success <= stop_metric_capture(session_url, metric_capture_id) # Gather all data for a given session # Optionally filtered by attributes of matching metric types data <= grab_session_data(session_url, attributes=None)

4. I&M Proposed Simple common API • Clearly this is a simple API, and one can do much more with either GIMI and GEMINI than is available through this API. • Haven’t discussed common analysis or orchestration, e.g. • Nor have we touched on common resource topology allocation • There is considerable non-uniformity in this area as well  • But it is a starting point and we can add to it as we go • Just getting this basis established will be a major win for the experimenter community • Supporting common scripts that can run on ANY GENI topology provided from ANY GENI aggregates.

5. Backup / Notes

6. Strawman Combined Tool • 2 slides: MSB • Roundtable discussion • 15 minutes: Ilia, Rob, Tom, Niky, MSB • Discuss common graphical resource assignment tool, and relationship to portals • Bring in discussion of Solicitation 4

7. Orchestration • Instrumentation • Monitoring • Archiving • Analysis

8. I&M • We know that there are two different I&M tool suites being developed and deployed: GIMI and GEMI • Each is developed on one platform, and is slated to be ported to the other platform this year (though personally I think this is a stretch). • I would like to define a common set of API’s that will be available on any GENI image supplied by GENI racks.

9. I&M : Proposal • I suggest we write a simple API for generic I&M support • And then write implementations of this API that run on any given image. • The image may support this implementation with GIMI or with GEMINI (or perhaps a hybrid)

10. I&M [3] • There’s lots more to do in this area (Orchestration, data analytics, etc.) • But I think this is a very good start and if we only do this much we’ll have done a lot. This effort is likely to be less in the hands of the Architects than in the members of the different I&M teams. But I want us to monitor and encourage this kind of common functionality.

11. Notes • Common API for writing, requesting, tagging • Different underlying implementations • From PORTAL • PORTAL : SSO integration of separate tools • Allocation time : AM API (initialization scripts) • Execution time : Scripts, services within slivers • LCD + Value Added differentiators