Systematic Experimentation in GENI

1. Systematic Experimentation in GENI

2. Agenda Motivation Sarah Edwards, Xuan Liu, Niky Riga, Vic Thomas, GPO Case Study: Virtual Router Failure Restoration Xuan Liu, UMKC/GPO Tools and Techniques Custom images and install scripts InstaGENI, Gary Wong, University of Utah ExoGENI, Paul Ruth, RENCI Porting ExoGENI Images to InstaGENI Ezra Kissel, Indiana University Manipulating RSpecs with geni-lib Nick Bastin, Barnstormer Softworks How to Collaborate Sarah Edwards, GPO Q&A Experimenter Drop-In (next session)

3. Systematic Experimentation in GENI:Motivation Sarah Edwards, Xuan Liu, Niky Riga, Vic Thomas GENI Project Office

4. GENI-FIRE 150 node topology Clear plan Automation Scale and Visualization Courtesy of Thierry Rakotoarivelo, NICTA, GEC 20 Demo

5. Virtual Computer Networking Lab Start small Courtesy of Cong Wang and DivyaBhat, UMass Amherst, GEC 20 Demo

6. Intelligent Data Movement System Long-lived slice for stitched, shared VLAN Dynamically add/remove nodes as needed Courtesy of Ezra Kissel, Indiana University, GEC 20 Demo

7. Why doesn’t this work?(Anonymous) Past Bug Reports OpenFlow multi-site repeatable incremental test controller in one AM? Multi-site Hadoop Used control plane to send high volume data between racks OpenFlow multi-site Interfere with rack ! GRE works Stitch test controller in one AM

8. Systematic Experimentation Running an experiment on a testbed requires best practices and methodology from a combination of: • Science (Scientific method) • Programming • System Administration Today, talking about strategies and techniques for bringing up experiments in a systematic way.

9. Why experiment systematically? DebuggingEasier to debug experiments. Easier to get help debugging. Especially for complex experiments. Scalabilityand Repeatability A well designed experiment will be repeatable and systematically scalable Valid Develop a solid understanding of and validate your experimental setup RigorousPerform a scientifically rigorous (and publishable) study

10. General Principle #1: One thing at a time Experimentation 101 andDebugging 101 Only change one thing at a time Examples: • Software/image • Configuration • Number of nodes/links • Geographic distribution of nodes/links

11. General Principle #2: Start Small Start by building smallest possible topology by hand. Automate as needed. Test and measure as you go. router node OVS switch router client server host host host master router worker worker

12. General Principle #3: Save what you do • Log all of your experimental artifacts for every experiment that works • RSpec • image • install script • custom software • measurements • etc • Use version control to store your artifacts • Always know the last configuration that worked

13. Experiment Workflow Summary Workflow How often should I loop?

14. Process Recommendation D. Increase scale E. More nodes C. Orchestrate and Instrument F. More aggregates B. Automate topology creation at a single aggregate A.Build (smallest possible) topology by hand at a single aggregate Automate

15. Process: Start Small A. Build smallest possible topology by hand at a single aggregate B. Automate topology creation at a single aggregate • Store RSpecs, software, etc in version control • TOOLSUse install scripts and custom images • Determine how to update software on your nodes Repeat A/B as needed. C. Orchestrate and Instrument your experiment procedure TOOLSGENI Desktop and LabWiki Automate as needed. node master worker worker

16. Process: Scale D. Increase scale (number of nodes, volume of traffic, etc) TOOLSFlack for small scale. geni-lib for larger scale server client server client client client client client

17. Process: Split across aggregates E. Split an existing topology across second aggregate • How connect across aggregates matters (stitching vs OF etc) TOOLSstitcherfor VLAN stitching and GRE tunnels. ExoSM for Intra-ExoGENI stitching. Flack for GRE tunnels. F. Split across multiple aggregates server client client client client client Colors represent different aggregates Inter-aggregate link

18. Avoiding Common Pitfalls • Don’t start with a large or inter-aggregate topology • In general, if you are keeping your experiment up because you can’t recreate your setup, then something is wrong. • GENI is not a simulator • Common Issue: OpenFlow controllers used on an multi-aggregate topology with loop MUST be tested in a single aggregate topology with loop

19. Case Study

20. Process Summary D. Increase scale E. More nodes C. Orchestrate and Instrument F. More aggregates B. Automate topology creation at a single aggregate A.Build (smallest possible) topology by hand at a single aggregate Automate

21. Overview of Tools and Techniques • Automation • Custom images on InstaGENI/ExoGENI • Converting ExoGENI images to InstaGENI • Install script on InstaGENI • Postboot script on ExoGENI • Measurement with LabWikiand GENI Desktop • Repeating and sharing experiments with apt • Scaling Up • geni-lib • Inter-aggregate experiments • stitcher • OpenFlow Tutorials Now Now Now Drop-In Now Tutorial Tutorial Now

22. Agenda Motivation Sarah Edwards, Xuan Liu, Niky Riga, Vic Thomas, GPO Case Study: Virtual Router Failure Restoration Xuan Liu, UMKC/GPO Tools and Techniques Custom images and install scripts InstaGENI, Gary Wong, University of Utah ExoGENI, Paul Ruth, RENCI Porting ExoGENI Images to InstaGENI Ezra Kissel, Indiana University Manipulating RSpecs with geni-lib Nick Bastin, Barnstormer Softworks How to Collaborate Sarah Edwards, GPO Q&A Experimenter Drop-In (next session)

23. How to Collaborate

24. Project Membership example Projects have 1 Lead and any number of Admins, Members, and Auditors Expiration Typical Class Typical Research Project http://groups.geni.net/geni/wiki/GENIConcepts#Project

25. Working with multiple members in a slice Members of all slices in a project: • Project Leads (Professor) • Project Admins (Post-docs, researchers) Other can be added manually Professor Slice Admin Research Asst Post-Doc Slice Member Slice Lead http://groups.geni.net/geni/wiki/GENIConcepts#Slice

26. Slice Access Being a member of a slice means you can act on a slice: • Add resources • Check status • Delete resources • Renew resources With any tool!

27. Slice Access: Logging in to resources To ensure access in collaborator’s resources: Option 1: Make resource reservation from Portal or omni Option 3: Use a common public key Add slice member’s accounts to existing resources: $ omni -V 3 poa SLICE geni_update_users --useSliceAggregates –-useSliceMembers Alternatively… $ addMemberToSliceAndSliversmyslice username Slice membership does not guarantee ability to login to resources! Only works on InstaGENI/ProtoGENI

28. Questions?

29. Special Topic Experimenter Drop-In In this room after the break • Writing ExoGENI velocity scripts • New ExoGENI image snapshotting feature • Converting ExoGENI images to InstaGENIimages • Manipulating RSpecs with `geni-lib` • Sharing experiments with `apt` Thanks to: • Paul Ruth, RENCI • Nick Bastin, Barnstormer Networks • Rob Ricci, University of Utah