BigDB : Proposed Big Switch Contribution to Open Daylight Rob Sherwood

1. BigDB: Proposed Big Switch Contribution to Open Daylight Rob Sherwood

2. Outline • Why? Lessons Learned… • BigDB Overview • Demo • Potential Implications for Open Daylight • Code Integration Strategy

3. Lessons Learned from Floodlight And Beacon, NOX, FlowVisor, etc… • Floodlight has a lot of developers and a lot of APIs • Two years of growth, most active OSS SDN controller community • Problem #1: API divergence/chaos • Open source makes for a wild west of API design • Mismatched names, types, encodings, return values • Java APIs don’t match REST APIs which don’t match storage schemas • Documentation became a PITA • Problem #2: Applications too tightly coupled to state • Applications needed lots of refactoring to change state properties • Consumers of state would hang if the state producers died • State read caching and write compressing was per application • Problem #3: Table-based schemas were a PITA • Apps had to do lots of join()s ; Cascading deletes were bug-prone • Lots of foreign key references; permanent state inconsistencies very easy

4. Solution: BigDB – an API Indirection Layer • Auto-generate all APIs from a formal language: YANG • Modules define their state in YANG • Modules can re-use and extend types and schemas from each other • Common accessor/search/manipulation pattern: CRUD + xpath • Auto-generate API documentation from YANG • Each piece of data/state is annotated with storage properties • Hints for which pluggable back-end • Decouple applications from actual state location • Simplifies application development and scaling • All schemas are tree-based (not table like SQL) • Simplifies data dependency descriptions: switch  port link status • Many fewer joins; cascading deletes are now subtree deletes

5. BigDB Example Workflow Define Yang Schema (2) Install Module Into BigDB (3) Access REST API List switches { key"dpid”; leafdpid { typedpid-string; description”...” } //… leafactive { typeboolean; description ”...” } leaf alias { type string; configtrue; description”...” } } #!/usr/bin/python importurllib, json URL=“/api/v1.0/switches” defjson_get(host, url): # … defshow_switches(): data = json_get(localhost, url) for sw in data[‘switches’]: print “Switch %s is %s” % ( sw[‘alias’], “up” if sw[‘active’] else “down”) def main(): show_switches(); BigDB

6. BigDB Details • Schemas are extensible • One App could define a switch and another App could supplement it • Searches in BigDB are xpath-like • curl -g 'http://localhost:8082/api/v1/data/controller/core/switch[dpid="00:00:00:00:00:00:00:01"] • (Mostly) Standards compliant YANG parser • Auto-generate REST and Java API support now – more possible • Currently integrating RBAC support with pluggable auth modules • Supports “weak references” for non-tree references • Think symbolic links in a file system

7. BigDB for Replication and Scale-out BigDB Isolates Applications Application Bar Application Foo “PUT /Apps/Foo/state=1” “GET /Apps/Foo/state” “/…/state=1” BigDB BigDB Foo.Yang Foo.Yang Memory Backend NoSQLDistributed Database Added support for publish/subscribe state change notifications “/Apps/Foo/state=1” Controller One Controller Two

8. BigDB Demo • Extending the YANG Schema • show_switches.py (direct from ppt slide)

9. Implications for Open Daylight • BigDB is a uniform language for defining APIs – north and south • Learn the lessons from floodlight! • Keep APIs consistent from the start, before more get written • Keep Apps decoupled from state location and properties • Pluggable back-end storage system • Currently have adapters for in-memory DB • Have a “dynamic” backend – think “/proc” • Could easily adapt to JBOSS Infinispan backend • (Active development) Pluggable authentication modules for RBAC

10. Integration Strategy • BigDBCode • ~25 KLOC (without tests, etc.) • Mostly stand-alone in the net.bigdb java package • Can run as stand-alone or “captive” mode with controller • I can see benefits both ways – open to opinions • Lots of existing floodlight schemas could be leveraged • API compatibility if we decide that’s useful • Any additional back ends to plan?

11. Conclusion • I’ve designed a lot of SDN controllers • IMHO, BigDB is the “right” answer to a lot of these issues • Proposal: BigDB/YANG should become the de facto API description tool for the other Open Daylight components • If we choose not to go down this router, we risk: • API divergence • Apps too tightly coupled to each other’s state • Hard-to-manage state consistency models