Using Omni to Build Tools

1. Using Omni to Build Tools $ omni.pycreatesliveraliceslicemyRSpec.xml INFO:omni:Loadingconfig file omni_config INFO:omni:Using control framework pgeni INFO:omni:Sliceurn:publicid:IDN+pgeni.gpolab. expires within 1 day on 2011-07-07 INFO:omni:Creating sliver(s) from rspec file INFO:omni:Writing result of createsliver for INFO:omni:Writing to ‘aliceslice-manifest-rspe INFO:omni: ----------------------------------- INFO:omni: Completed createsliver: Options as run: aggregate: https://www.emulab. framework: pgeni native: True Args: createsliveraliceslicemyRSpec.xml Result Summary: Slice urn:publicid:IDN+pgeni Reserved resources on https://www.emulab.net/p Saved createsliver results to aliceslice-man INFO:omni: =================================== Sarah Edwards, GENI Project Office 23 July 2013

2. Outline • What is omni? • Using omni as a script to build other tools • Example WIP: Awesome Omni • What’s next?

3. What is omni? • Omni is a command line client tool for use with the GENI Aggregate Manager (AM) API • Supports reserving resources from any aggregate which supports the AM API • Supports complete resource reservation workflow • Distributed with gcf which is a set of developer tools, including an AM, which allow for testing the AM API client aggregate • Flack • Omni(-based): • omni • portal.geni.net • gcf-am.py • Flavors: • InstaGENI • ExoGENI • FOAM (OpenFlow) • AMSoil-based

4. Strengths and Weaknesses In addition, to the usual strengths and weakness of CLIs… Strength Weakness Changes fairly frequently User must be aware of differences in v1/v2 vs v3 Lots of command line options (-h is very long) Packaging System exit codes • Reliably at the forefront of implementing changes to the AM API • Easy and powerful to script with • e.g. readyToLogin.py • Returns easy to use python objects (no parsing CLI output)

5. (Not So) Recent Additions • Multiple aggregates specified in one command (use –a multiple times) • omni-configure.pyconfigure omnifor ProtoGENI, Portal and PlanetLabuser account • readyToLogin.py--no-keysgets login info even if don’t have ssh keys – works well with multi-user slices • stitcher.pydynamically reserve cross-domain VLANs • Aggregate nicknames In an upcoming release • RSpec nicknames • Specify in omni_config vm= http://...../onevm_rspec.xml • Look in default directory or url default_rspec_location= http://rspecs.geni.netdefault_rspec_extension= rspec agg nickname RSpec nickname omnicreateslivermyslicemyspecialtopology -a ig-utah

6. omni-configure.py • Distributed with omni • Not built on omni (ienot an omni script) • Automatically generates your omni_config • Supports ProtoGENI, PlanetLab, and GENI Portal accounts • Reasonable defaults for the common case • Just run: omni-configure.py • All defaults can be overridden with command line options

7. readyToLogin.py • Distributed with and built on omni • Automatically determines your ssh login info • Supports ProtoGENI, ExoGENI resources $ readyToLogin.py egtest10 -a pg-ky my-node's geni_status is: ready (am_status:ready) User sedwards logs in to my-node using: ssh-p 33594 -i…/geni_key_portalsedwards@pc64.uky.emulab.net ssh -p 33594 -i…/geni_cert_portal_key sedwards@pc64.uky.emulab.net ssh -p 33594 -i…/geni_cert_portal_keysedwards@pc64.uky.emulab.net $ readyToLogin.pyegtest10 -a pg-ky --no-keys my-node's geni_status is: ready (am_status:ready) User sedwards logs in to my-node using: ssh -p 33594 sedwards@pc64.uky.emulab.net

8. Omni scripting Easily execute and parse the output of omni calls from within another script #!/usr/bin/envpython import omni # Get a parser from omni that understands omni options parser = omni.getParser() parser.set_usage("%s [options] username"%sys.argv[0]) options, args = parser.parse_args(sys.argv[1:]) <snip> # (1) Run equivalent of 'omni.pylistmyslicesusername’ text, sliceList = omni.call( ['listmyslices', username], options ) <snip> # (2) For each returned slicename run equivalent of: # 'omni.pyprint_slice_expirationslicename’ for slicename in sliceList: text, expiration = omni.call( ['print_slice_expiration', slicename], options ) printStr += "%s\n"%(str(expiration)) printStr+= "="*80 return printStr Returns python objects gcf/examples/expirationofmyslices.py ================================================================================ User sedwards has 2 slice(s): Slice urn:publicid:IDN+ch.geni.net:tutorial+slice+portal99 expires on 2013-07-13 18:54:28 UTC Slice urn:publicid:IDN+ch.geni.net:tutorial+slice+portal01 expires on 2013-07-12 14:03:25 UTC ================================================================================

9. Omni scripting Inherit command line options from omni while adding custom options #!/usr/bin/envpython import omni # Get a parser from omni that understands omni options parser = omni.getParser() <snip> # Add additional optparse.OptionParser style options for your script as needed. # Be sure not to re-use options already in use by omni parser.add_option("--myScriptPrivateOption", help="A non-omni option added by %s"%sys.argv[0], action="store_true", default=False) # options is an optparse.Values object, and args is a list options, args = parser.parse_args(sys.argv[1:]) if options.myScriptPrivateOption: # do something special for your private script's options print "Got myScriptOption” <snip> Easily inherit omni’s command line arguments… …while adding your own Get -a, -c, --api-version for free gcf/examples/myscript.py

10. Tools built using omni scripting For experimenters gcf/src/stitcher.py • Tool which supports stitching (as seen in yesterday’s plenary) gcf/examples/readyToLogin.py • determines ssh command to login to any node gcf/examples/remote-execute.py • remotely runs commands on the nodes in your slices For developers gcf/acceptance_tests/AM_API/am_api_accept.py • AM API acceptance tests • verifies compliance with the AM API specification • supports AM API v1, v2, and v3 • Looking for an example code snippet? Look here For operators • Aggregate health checks run by GPO Lab’s nagios are simple omniscripts

11. Tools using omni under the covers GENI Experimenter Portal (https://portal.geni.net) • runs omni under the covers to talk to aggregates GENI Desktop and GIMI • Each use omni for parts of their GENI Portal integration HyperNets a.k.a. ShufengHuang’s PhD Thesis

12. Best Practices Using Omni to Build a Tool • Cache slice credential omni.pygetslicecredmySlice-o # save slice cred to a file omni.pycreatesliver--slicecredfilemyslice-cred.xml • Cache user credential omni.pygetusercredmySlice -o # save slice cred to a file omni.pylistmyslices --usercredfilejdoe-usercred.xml • GetVersion cache by default reduces calls to GetVersion but may or may not be appropriate to your situation To not use the cache use --NoGetVersionCache To use the cache use --ForceUseGetVersionCache To age out the cache --GetVersionCacheAge To set the location of the cache file --GetVersionCacheName • Cache a list of aggregates used in each slice • So only have to query used aggregates

13. What next for omni? Possible next steps? What would be most useful? • Packaging and installing omni -> easier to build tools using omni • Completed initial installation on Windows • Unix package • Cache user & slice creds -> fewer/quicker calls • Ability to run multiple commands in one invocation • Exit codes returned when run from the command line make it hard to shell script with omni • Smooth over difference in v1/v2 vs v3  easier for users • Common CH API support • Multi-user slice support: dynamically query CH for users and keys for each slice instead of using a statically configured list • Others?

14. Where to get it? Where to get omni/gcf? Can be downloaded from the gcf wiki: http://trac.gpolab.bbn.com/gcf/wiki Have an idea for an improvement? Send a suggestions to gcf-dev@geni.net Or, make a ticket (requires a wiki account) http://trac.gpolab.bbn.com/gcf/newticket

15. Awesome Omni:Or how I learned to stop worrying and love Omni Work In Progress inspired by Niky Riga

16. Motivation • Omni originated as a developer test tool and has been adapted to support advanced experimenters • Omni’s user interface reflects this history • Many (~30) commands and many (~40) options • The output is thorough (aka long and verbose) • omni-configure.py, readyToLogin.py, and stitcher.py are separate tools

17. Imagine that omni were more awesome. Imagine it worked like this….

18. Configure and setup awemniinit * awemniconfigured!                        * * Try:                              *   awemnihelp                             * awemnihelp * .....                                  

19. A complete workflow awemni create vm as topo1 at utah_rack * Created one virtual machine called ”topo1" at aggregate "utah_rack” * To login/ssh, try:                                          *   awemni connect node1                                             * To release resources try:                                           * awemnidelete node1 awemniconnecttopo1 atutah_rack * ”topo1" isbooting...wait a minute and tryagain awemniconnecttopo1 atutah_rack * ”topo1" isready...logging in * vm> hostname vm.utah.geniracks.com vm> exit awemnideletetopo1 * Deleted virtual machines  

20. Or even more simply… awemni create vm * Created one virtual machine named "novelSlice” *   at each of: utah_rack, mass_rack * To login, try:                            *   awemniconnectnovelSliceatutah_rack * To release resources try:                       * awemnideletenovelSlice awemniconnectnovelSliceatutah_rack * "novelSlice" isready...logging in       vm> hostname vm.utah.geniracks.com vm> exit awemnideletevm * Deletedvirtualmachines

21. Reserving a custom stitched topology  With stitching this becomes even easier… awemnicreate nikysawesometopology .....

22. What’s needed to do this? • Have omni support nicknames for RSpecs • A way to store a small amount of current state A wrapper to: • parse the natural language-ish syntax into the underlying omni • parse the omni outputs into something natural language-ish • pull in the functionality of omni-configure.py to support init • pull in the functionality of readyToLogin.pyto support connect • parse RSpec into something human readable • Testing

23. Demo