1 / 34

Middleware Lab SS 2011

Middleware Lab SS 2011. BubbleStorm API for Java Max Lehn, Christof Leng , Robert Rehner. Agenda. Building the BubbleStorm libraries Getting the sources & necessary tools Building the BubbleStorm libraries BubbleStorm Java API Introduction A BubbleStorm Application

kira
Download Presentation

Middleware Lab SS 2011

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Middleware Lab SS 2011 BubbleStorm API for Java Max Lehn, ChristofLeng, Robert Rehner 03.05.2011 | Department of Computer Science | Databases and Distributed Systems

  2. Agenda • Building the BubbleStorm libraries • Getting the sources & necessary tools • Building the BubbleStorm libraries • BubbleStorm Java API Introduction • ABubbleStorm Application • Frontend & backend • Simple example application 03.05.2011 | Department of Computer Science | Databases and Distributed Systems

  3. BubbleStormfor Java • BubleStorm is written in Standard ML • Java applications access the library through a JNI wrapper Java application Java BS Java API JNI wrapper C C library interface BubbleStorm library Standard ML 03.05.2011 | Department of Computer Science | Databases and Distributed Systems

  4. Getting the Sources • Youneed a Subversion client • E.g., Tortoise SVNfor Windows,kdesvnorthecommandlineclientfor Linux • Check out theBubbleStormprojectfromhttps://www.dvs.tu-darmstadt.de/svn/projects/BubbleStorm/Middleware/trunksvnco https://www.dvs.tu-darmstadt.de/svn/projects/BubbleStorm/Middleware/trunk • Youshouldbynowhaveaccountstoaccesthe SVN repositoy 03.05.2011 | Department of Computer Science | Databases and Distributed Systems

  5. Necessary Tools • MLtoncompiler (includes a GNU toolchain) • SQLitelibrary • Java 6 JDK (not just JRE!) • Apache Ant …seeinstallationmanual on ourwebsiteformoredetails Wesupport Linux and Windows (Mac OS X shouldwork, too) 03.05.2011 | Department of Computer Science | Databases and Distributed Systems

  6. Building the BubbleStorm library • Shell:> cd <BubbleStorm checkout>/bindings/c/bubblestorm> make • libbubblestorm.so / bubblestorm.dll • Note: You need at least 1GB RAM to build the project, and it may take a couple of minutes 03.05.2011 | Department of Computer Science | Databases and Distributed Systems

  7. Building the JNI library • Shell:> cd <checkout>/bindings/java/bubblestorm> ant • libbsjni.so / bsjni.dll 03.05.2011 | Department of Computer Science | Databases and Distributed Systems

  8. Testing the Java Library • Copy all shared libraries to <checkout>/bindings/java/bubblestorm • Linux: libbubblestorm.so, libbsjni.so • Windows: sqlite3.dll, bubblestorm.dll, bsjni.dll • Run a single bootstrap node listening on port 8585:> cd <checkout>/bindings/java/bubblestorm> ant -Dargs="bootstrap localhost" runtest • Join the network (2nd console):> ant -Dargs="join localhost 8586" runtest (binds to port 8586 and connects to localhost:8585) 03.05.2011 | Department of Computer Science | Databases and Distributed Systems

  9. The BubbleStorm API 03.05.2011 | Department of Computer Science | Databases and Distributed Systems

  10. CUSP • CUSP is BubbleStorm's transport protocol • Reliable transport based on UDP • Congestion & flow control • Encryption • Channels and streams: • Channels are the 'physical' links between pairs of nodes • Streams are the logical transport units visible to the application • A channel can contain any number of streams • Streams are cheap to create and use once the channel is set up • Your application may not use much of CUSP directly, but you will see some parts of it UnidirectionalStreams Channel 03.05.2011 | Department of Computer Science | Databases and Distributed Systems

  11. CUSP API Briefly: • EndPoint is the local connection point • The first thing you have to create for using CUSP • Binds to a local UDP port • advertise() to accept incoming streams • contact() to connect to a remote host • Address represents a network address (e.g. localhost:8585) • InStream reads data from a remote host • OutStream writes data to a remote host 03.05.2011 | Department of Computer Science | Databases and Distributed Systems

  12. BubbleStorm: Creation • The core component of a BubbleStorm node: class BubbleStorm • Create instance using BubbleStormFactory Class BubbleStormFactory • static BubbleStormBubbleStormFactory.create( float capacity, Address bootstrap,int port,booleanencrypt) Creates a Topology object that is not connected to the network • capacity: relative capacity of this node (default: 1.0) • bootstrap: bootstrap address for connecting the network (optional) • port: local UDP bind port (CUSP) • encrypt: enable encryption (CUSP) 03.05.2011 | Department of Computer Science | Databases and Distributed Systems

  13. Join & Leave Class BubbleStorm • void createRing(Address localAddress)void createRing(Address localAddress, JoinHandlerjoinHandler) Creates a new network by building a ring with itself • localAddress: the address using which other nodes can reach this node • joinHandler: is notified when the join process has completed (quick, but not immediate!) • void join(JoinHandlerjoinHandler) Joins an existing network; addresses of nodes used for joining are pulled from the application's host cache (previous slide) • joinHandler: is notified when the join process has completed • void leave(LeaveHandlerleaveHandler) Leaves the network 03.05.2011 | Department of Computer Science | Databases and Distributed Systems

  14. Bubble Types a (data) a (publish) b (query) b (subscribe) • Each bubble has an application-defined type • Bubble types are connected (matched) to ensure a certain number of nodes in the overlap of two bubbles • Examples: • data/query • publish/subscribe • but also: n-way, self-intersection 03.05.2011 | Department of Computer Science | Databases and Distributed Systems

  15. Bubble Types (2) Recall: Replication Types • Fading, Managed, Durable BasicBubbleType PersistentBubbleType InstantBubbleType FadingBubbleType ManagedBubbleType Future: DurableBubbleType 03.05.2011 | Department of Computer Science | Databases and Distributed Systems

  16. Bubble Master Bubble typesarecreatedusingBubbleMaster obtainedusingBubbleStorm.bubbleMaster() • InstantBubbleTypenewInstant(intid, floatpriority) • InstantBubbleTypenewInstant(intid) Eachbubble type has a unique, application-defined type ID (id) • FadingBubbleTypenewFading(intid, floatpriority,BubbleHandlerWithIdbubbleHandler) • FadingBubbleTypenewFading(intid,BubbleHandlerWithIdbubbleHandler) The bubblehandlerisinvoked on incomingdatabubbles (Managedbubble type comingsoon) 03.05.2011 | Department of Computer Science | Databases and Distributed Systems

  17. Bubble Type Methods BasicBubbleType • void match(PersistentBubbleTypewith, double lambda,BubbleHandlerhandler) Matches bubble type with another • with: the matched bubble type (must be a persistent bubble type) • lambda: certainty factor / desired number of intersecting nodesP(at least 1 result) = 1-exp(-lambda)lambda = 4  P ≈ 98.17% • handler: is notified on intersection 03.05.2011 | Department of Computer Science | Databases and Distributed Systems

  18. Bubble Type Methods (2) PersistentBubbleType • void matchWithId(PersistentBubbleType with, double lambda,BubbleHandlerWithId handler) LikeBasicBubbleType.match, but objectsareassociatedwith an ID Persistent objectshave IDs foruniqueidentification IDis a 160-bit identifier 03.05.2011 | Department of Computer Science | Databases and Distributed Systems

  19. Bubble Creation InstantBubbleType • void createBubble(byte[] data) Creates an instant bubblewithgivenpayloaddata (i.e., pushesthedataintothenetwork) FadingBubbleType • void createBubble(ID id, byte[] data) Creates a fadingbubblewithgiven ID andpayloaddata 03.05.2011 | Department of Computer Science | Databases and Distributed Systems

  20. Data/Query • Data is stored with persistent bubbles • Queries are sent as instant bubbles • Addition to stores: query responses • A node that receives a query and has matching data calls back to the querier • The results are delivered 'off-the-band', i.e., through direct CUSP connections b a results store query 03.05.2011 | Department of Computer Science | Databases and Distributed Systems

  21. Publish/Subscribe • Subscriptionsaresentas persistent bubbles • Publications aresentas instant bubbles(in thesimplestcase; publicationsmaybepersistedaswell) • Responses are not senttothequerier, but tothesubscriber b a notifications subscribe publish 03.05.2011 | Department of Computer Science | Databases and Distributed Systems

  22. Notification Off-the-band notificationandresultdeliveryissimplifiedusingtheNotificationmechanism • Peer B • Decodethenotificationinformation, obtain a notificationsender • Send a notification • Peer A • Create a notificationobjectwhichservesas a call-back receiver • Encodeand send thecall-back information (address, port, …) • Notificationhandlerisinvoked • Optional: Close notificationobject 03.05.2011 | Department of Computer Science | Databases and Distributed Systems

  23. Notification (2) Class NotificationFactory • static NotificationReceivercreate(EndPointep, Handler handler) • static Pair<NotificationSender, Integer> decode(byte[] data, EndPointendpoint) Class NotificationReceiver • byte[] encode() • voidclose() Class NotificationSender • voidsend(ID id, Handler handler) • voidsendImmmediate(ID id, byte[] data) 03.05.2011 | Department of Computer Science | Databases and Distributed Systems

  24. Query Simple queries are even more simplified using the Query class Class Query • static Query create(BubbleMaster master,PersistentBubbleTypedataBubble,intqueryBubbleId, float lambda, Responder responder) Creates a Query object, implicitly creating and matching the query bubble type 03.05.2011 | Department of Computer Science | Databases and Distributed Systems

  25. Query (2) Class Query (cont.) • Stoppable query(byte[] queryData, ResponseReceiver receiver) Starts a query: • opens a notification receiver • creates a query bubble • delivers incoming responses to the application The query has to be explicitly stopped by the application via the returned Stoppable 03.05.2011 | Department of Computer Science | Databases and Distributed Systems

  26. BubbleStorm API Wrap-Up Importantclasses: • BubbleStorm • BubbleMaster • BasicBubbleType, PersistentBubbleType • InstantBubbleType, FadingBubbleType • Notification • Query 03.05.2011 | Department of Computer Science | Databases and Distributed Systems

  27. A BubbleStormApplication 03.05.2011 | Department of Computer Science | Databases and Distributed Systems

  28. Frontend and Backend Sends requests(store, query, …) ApplicationFrontend User Interface Bubble Query BubbleStormmiddleware Handler Responder ApplicationBackend Stores data,processes queries Frontend and backend should not needtointeractdirectly! 03.05.2011 | Department of Computer Science | Databases and Distributed Systems

  29. Sample Application (The following is a condensed version of net.bubblestorm.demo.BSTest) • Initialize the BubbleStorm libraryBubbleStormFactory.init(); • Create CUSP EndPointEndPoint endpoint = EndPoint.create(port); • Create BubbleStorm instance, get bubble masterBubbleStormbs = BubbleStormFactory.createWithEndpoint( 1.0f, bootstrapAddr, endpoint);BubbleMasterbm = bs.bubbleMaster(); 03.05.2011 | Department of Computer Science | Databases and Distributed Systems

  30. Sample Application (2) • Instantiate backend (implements the bubble handler for store operations and the query responder)ValueStorevalueStore = new ValueStore(); • Create data bubble typeFadingBubbleTypedataBubble = bm.newFading(1, valueStore); • Create Query object (implicitly creates and matches query bubble)Query query = QueryFactory.create(bm, dataBubble, 2, 4.0f, valueStore); • Join the networktopology.join(joinHandler); • Implement the join handler… 03.05.2011 | Department of Computer Science | Databases and Distributed Systems

  31. Project Layout • All java-related sources are located in<bubblestorm>/bindings/java • 3 subfolders • common (BSBase-Jni): basic JNI functionality, helper classes • cusp (CUSP-Jni): CUSP bindings • bubblestorm (BubbleStorm-JNI): BubbleStorm bindings • Each of them contains an Eclipse project which you can (should) open • To browse the API classes • To look at the demo application(s) • Your application references (at least) the BubbleStorm-JNI project • Note: There is a sync (default) and an async version of the CUSP bindings –you should only look at sync 03.05.2011 | Department of Computer Science | Databases and Distributed Systems

  32. Run configuration • To run BubbleStorm applications, you need to set the library search path for the shared libraries • libbubblestorm.so, libbsjni.so / bubblestorm.dll, bsjni.dll, sqlite3.dll • Windows: set the PATH environment variable to the containing folder(s) • Linux: set the LD_LIBRARY_PATH environment variable • You can set these variables in Eclipse's run configuration or in an ant script (see bindings/java/bubblestorm/build.xml) 03.05.2011 | Department of Computer Science | Databases and Distributed Systems

  33. Deliverables for the Next Meeting • Understand the BubbleStorm Java API (Javadocs in the source code) • Run the demo program • Build the first version of frontend and backend of your application 03.05.2011 | Department of Computer Science | Databases and Distributed Systems

  34. If there are Questions • The BubbleStorm classes have Javadoc comments (hopefully helpful) • Please post all BubbleStorm-related questions/comments to the mailing list:bubblestorm-devel@dvs.informatik.tu-darmstadt.de … andifyouhavequestionsnow, askthemnow 03.05.2011 | Department of Computer Science | Databases and Distributed Systems

More Related