ORCA Status Report for Spiral 2 and Roadmap for Spiral 3 GEC9

1. ORCA Status Report for Spiral 2 and Roadmap for Spiral 3GEC9 Ilia Baldine, Jeff Chase, AnirbanMandal, YufengXin

2. Bella 2.2 release (mid Nov) • For sure: • Bug fixes, bug fixes, bug fixes • Source tree re-org • Single project with trunk/ branches/ tags/ instead of multiple ones • PG RSpec-> NDL converter • XMLRPC controller for SM with full(er) RSPec support • Registry actor liveness confirmation • Better GUI to support multiply-connected topologies • Patches for Euca 1.[56].x to support multiple dataplane interfaces • Implemented Eucalyptus 2.0.0 with NEuca support • Demonstrating multi-connected topology embedding in Euca site • Minor NDL extensions • Add support for port-to-port provisioning • Maybes: • Advance reservations • Limited portal Shib support • I2 ION support • MAX/Dragon substrate support (PL interoperability) • Rudimentary Euca image management • SSH proxy • Cisco3400 driver/handler (U of H)

3. Bella 2.2 features in detail • XMLRPC controller • sdasdf • Eucalyptus with NEuca extensions • Eucalyptus experiment topology embedding • Orca Actor registry • NDL-OWL Format conversions Presentation title goes here

4. XML-RPC Controller • Goal: Provide an XML-RPC interface to ORCA for experiment control tools and external users • Tools/users would use this interface to • discover available resources • instantiate experiment slivers • check the status of the instantiated slivers • renew the lease for existing slivers • destroy and shutdown slivers • Controller exposes a GENI-AM API like xml-rpc interface to users • Caveats • Can’t support RSpec yet • Converter from/to NDL to/from RSpec to be integrated in near future • Credential management missing, only authorization is through the ORCA portal running the xml-rpc controller

5. Slice Manager (SM) Overview Broker ORCA Framework request Experiment control tools / Users ticket delegate XML – RPC calls redeem Authority/AM lease

6. Slice Manager (SM) Interface Details ORCA Framework 0. GetVersion () Experiment control tools / Users XML - RPC 1. ListResources () 2. Advertisement NDL 3. createSliver (Request NDL) 4. Manifest handle - sliceID 5. sliverStatus (sliceID) 6. Manifest sliver details and status 7. renewSliver (sliceID, time) 8. deleteSliver (sliceID) 9. ShutDown (sliceID)

7. NEuca Overview • NEuca (pronounced nyoo-kah) - extensions to Eucalyptus that permit • Creating and configuring virtual machines with multiple network interfaces attaches to VLANs or physical interfaces on the host node • Perform arbitrary post-boot actions on guest VMs • Implemented as a set of patches for Eucalyptus 2.0.0 and guest-side tools pre-loaded on the image (Linux derivatives supported) • Permits using Eucalyptus-manage interface for management, while using high-performance separate backplane for data communications • Developed as part of the larger initiative in enabling science applications on clouds • NSF SDCI award #1032573 • Does not affect the normal functionality of a private Eucalyptus cloud • In GENI context network experiments can coexist with computational work in a private cloud Network Extensions to Eucalyptus

8. Overview Network Extensions to Eucalyptus

9. Usage • Create an .INI-formatted configuration file • Pass to the instance using –user-data-file option • File contains • Global definitions • Guest interface configuration • Shell script to be executed post-boot • NEuca-py tools installed on the image can help retrieve the information for use by applications Presentation title goes here

10. [global] slice_id=6D3DE7CC-7188-49B2-AF48-F21497450D84 ; optional slice identifier [interfaces] ; Interface specification has the form of either ; eth1=vlan:[hostethX]:[vlan tag on ethX]{:guestip address} or ; eth1=phys:[hostethX]{:guestip address} ; interfaces must start at eth1 and can be named in any order, however numbering gaps are not allowed ; e.g. if configuration file specified eth1, eth2 and eth4 only eth1 and eth2 will be processed eth1=vlan:eth0:20:192.168.1.3/24 ; eth1 attaches to eth0.20 on host and has IP 192.168.1.3/24 eth2=vlan:eth0:19:192.168.2.3/24 ; eth2 attaches to eth0.19 on host and has IP 192.168.2.3/24 [instanceConfig] ; The script must not have any blank lines ; use tab or spaces in the beginning of the line to indicate the continuation ; of the script script=#!/bin/sh # Test script yum install make Presentation title goes here

11. Status and near-term plans • Implemented in Eucalyptus-2.0.0 • Installed at Duke, RENCI, UNC-CH, UMass • Integrated with ORCA • Launching science application experiments (Umass plenary demo) and topology embedding (ORCA demo) • Discussions with Eucalyptus Inc about including NEuca in upcoming open-source releases • Near-term enhancements • Using NEuca to implement SSH proxies for guests • Support for SR-IOV for better performance Presentation title goes here

12. NEuca site topology embedding • Use ORCA in conjunction with NEuca to enable local network experiments in a Eucalyptus cluster • Allow to embed random node topologies (consisting of VMs) into the Eucalyptus cluster • Cluster can still be used for computational tasks without any changes Presentation title goes here

13. Future topology embedding work • (Much) Harder problem: embed experiment topology in networked (Layer 2) collections of clusters • Some work in Spiral 3. Presentation title goes here

14. ORCA Actor Registry • Snapshot of available actors from various ORCA containers • Users can use this for resource discovery and other queries • Two new properties need to be set during container startup to register new actors • registry.url=http://geni-test.renci.org:11080/registry/ • registry.method=registryService.insert • Registry contains information about actors - brokers, authorities (AM) and slice managers (SM) • Actor name, guid, type, description, public keys, certificates and substrate information • For an authority (AM), the description of the resources under it’s control is available in the form of NDL

15. Slice Manager (SM) Slice Manager (SM) ORCA Actor Registry ORCA Container 2 ORCA Container 1 Broker Broker Authority/AM Authority/AM Register Register ORCA Actor Registry Experiment control tools / Users Registry Server XML-RPC Web Server Browser/jsp

16. Actor Registry • Users can visit http://geni-test.renci.org:11080/registry/actors.jsp to view current set of actors registered with the registry • Users can also programmatically query the registry using an XML-RPC interface String url = "http://geni-test.renci.org:11080/registry/"; XmlRpcClient client = new XmlRpcClient(url); Vector params = new Vector(); params.clear(); String result = (String) client.execute("registryService.getActors", params);

17. Actor Registry Snapshot

18. Spiral 3 features (Camano 3.x) • Two releases • Camano 3.0 ~03/30/2011 • SSH Proxy for substrates behind firewalls/NAT • RSpec -> NDL-OWL conversion • XMLRPC for GENI AM API 1.0 with RSpec support • Mixed mode testing environment • More robust topology embedding • Camano 3.1 ~08/15/2011 • Layer 3 peering for slices • GUSH integration • Shibboleth authorization for GENI attributes • Advanced scheduling reservation policies • ProtoGENI handler and policy • I2/MAX/Dragon handler and policy

19. Spiral 3 goal: ProtoGENI interoperability • Connectivity plan and implementation • Leverage StarLight • Need other peering locations (work with NLR/I2) • Decide on naming conventions/translations • First allow ProtoGENI to reserve Orca resources • XMLRPC handler with RSpec support implementing GENI AM API • Camano 3.0 • Allow Orca to reserve resources from ProtoGENI aggregates • GUSH integration for Orca • ProtoGENI handler and reservation policies • Camano 3.1 Presentation title goes here

20. Clearinghouse and intra-cluster connectivity • Cluster D broker still up however… • GPO recommends sites operate their own brokers • IMHO StarLight, LEARN, NLR and BEN should be under one broker • Sparse resources • Crucial interconnects • Avoid deadlocks during multiple experiments • Advance reservations should help • Actor registry • Registration API will evolve to support heartbeats • Query API should remain stable and usable by other clearinghouse-like projects tracking resources • Connectivity • VLANs to StarLight • VLAN 533 to UMass (perhaps getting more VLANs in the future) • VLAN 1804, 1810 OSU, WSU – short-circuited for now, will be Orca-managed • VLAN 2701, 2702 – via Gloriad to Korea (FIRST@PC OF testbed) • LEARN site connectivity to NLR and between the sites is being negotiated ‘as we speak’ Presentation title goes here

21. RENCI/Duke-supported experiments • Programmable Packet Networks over Dynamic Circuit Substrate • Xiaowei Yang, Duke University • Layer 3 programmable peering, topology embedding • Fine-scale probing for Very High Speed Networks • JasleenKaur, UNC-CH • Port-to-port multi-domain provisioning • Experiments in controlled and reproducible cross-traffic for network experiments • Kevin Jeffay, Don Smith UNC-CH • Port-to-port multi-domain provisioning • Working closely with ERM/IMF projects Presentation title goes here