SAGE – Architecture and API

1. SAGE – Architecture and API July, 2007 GCB SAGE workshop

2. Early Tile-Display Software: JuxtaView • Visualization of 2D datasets with predictive pre-fetching. • Slow user interaction and inefficient data access Scripps – Bathymetry and digital elevation NCMIR – microscopy (2800x4000 24 layers) July, 2007 GCB SAGE workshop

3. Early Tile-Display Software: MagicCarpet • Smart mip-mapped 2D data access • Very interactive • Needs datasets to be pre-processed July, 2007 GCB SAGE workshop

4. Visualization of 3D datasets Transfer function and UI Early Tile-Display Software: Vol-a-Tile July, 2007 GCB SAGE workshop

5. The Sage Pixel Streaming Architecture • Stream uncompressed pixels generated by apps to display walls • Designers tried to achieve a desktop-like environment for tile displays July, 2007 GCB SAGE workshop

6. Adaptive Rendering • Adapt to the display client • Laptop, high-resolution monitor, tiled display • Rendering capabilities • Pixel drawing, polygon rendering • Network capacity • Adapt to output resolution • “Resize” and “move” events • Pixel up-scaling or down-sampling • Increase rendering resolution July, 2007 GCB SAGE workshop

7. Pixel Sources • Visualization applications • Software rendering • Hardware rendering • Legacy applications • RDP, ARD, VNC • TeraVision (hardware capture) • Video streaming July, 2007 GCB SAGE workshop

8. Remote sensing Live video feeds Volume Rendering High-resolution maps 3D surface rendering Remote laptop Working in Display-Rich Environments July, 2007 GCB SAGE workshop

9. SAGE Components • Capture the pixels • Partitioning of the images • Routing the pixels • Layout on the display • User interaction July, 2007 GCB SAGE workshop

10. Basic Pipeline • With SAGE we separate the rendering and display. • Rendering machine is connected to display driven by thin client by fast network July, 2007 GCB SAGE workshop

11. Pipeline Gets Complicated… • More complicated scenarios • Rendering machine sends pixels to multiple displays. • There are multiple rendering machines (cluster) • Assume that the rendering is done remotely across big fat networks • Provide source (rendering software) with user interaction feedback July, 2007 GCB SAGE workshop

12. Ideal Scenario July, 2007 GCB SAGE workshop

13. Why Raw Pixel Streaming • Pixels are transmitted raw (without compression) because networks are getting faster/cheaper. • We don’t spend time compressing and decompressing on the CPU • New SAGE supports DXT compression July, 2007 GCB SAGE workshop

14. Related Work • WireGL / Chromium • Streams OpenGL primitives • Slow. User interactive apps suffer. • Not designed for fat long distance networks • Complex configuration • XDMX • Highly researched. Very flexible. • Streams X primitives • Slow • Not designed for fat long distance networks • No support for mullions July, 2007 GCB SAGE workshop

15. Related Work … contd. • IBM’s SGE • Hardware switched solution • Streams raw pixels • Not scalable • Expensive and legacy. • TeraVision • Hardware and open source software solution • Streams raw pixels from any video source • Scalable • Expensive • No support for dynamic routing. July, 2007 GCB SAGE workshop

16. Architecture … • Free Space manager provides central control between apps, UI and system • SAGE applications send their streams directly to the display nodes July, 2007 GCB SAGE workshop

17. Free Space Manager • Central control unit of SAGE. • Setups the entire system based on configuration files • Communicates with UI clients • Window movement and resize require messages to be passed between the Free Space Manager and apps • Sends SAGE status messages to apps. July, 2007 GCB SAGE workshop

18. SAGE Application Interface Library (SAIL) • Library for writing SAGE applications • Interfaces with the Free Space Manager to receive and send system messages • When a SAGE application gives SAIL a new video frame, it sends pixels directly to the appropriate portions of the display tiles. July, 2007 GCB SAGE workshop

19. More Details • Frame syncing • Every frame displayed within SAGE is synced with a TCP message. • So 60 fps = 60 messages per second • Windowing system • FrenchWindows • Collaborative features • UI can be launched and used by multiple users at the same time • New SAGE (v2.x) provides multiple mouse pointers. July, 2007 GCB SAGE workshop

20. Basic Code to ‘SAGEify’ an Existing App sailConfig scfg; // Setup SAGE structure scfg.cfgFile = "sage.conf"; scfg.appName = “myapp"; scfg.rank = 0; sageRect renderImageMap; renderImageMap.left = 0.0; renderImageMap.right = 1.0; renderImageMap.bottom = 0.0; renderImageMap.top = 1.0; scfg.imageMap = renderImageMap; scfg.colorDepth = 24; scfg.pixFmt = TVPIXFMT_888; scfg.rowOrd = BOTTOM_TO_TOP; sageInf.init(scfg); // Init SAGE while (1) { .. <render video frame> .. sageInf.swapBuffer( rgbBuffer ); // send rendered video frame to SAGE } July, 2007 GCB SAGE workshop

21. SAGE UI • Connects to SAGE over the network. • Main window is divided into two sections: • top which represents the display the UI is connected to • bottom which holds information about the applications currently running. July, 2007 GCB SAGE workshop

22. SAGE UI • Resize or move application windows like on a desktop • Change ‘depth’ order of windows July, 2007 GCB SAGE workshop

23. SAGE UI • Maximize the windows • Preserve (or not) the aspect ratio of rendering app. July, 2007 GCB SAGE workshop

24. SAGE UI • Performance monitoring built in • Bandwidth (Network and Display) • FPS (Network and Display) • Number of nodes used by an app July, 2007 GCB SAGE workshop

25. SAGE UI • Performance graphs • Built in logging • Global option accessible via ‘Performance’ menu July, 2007 GCB SAGE workshop

26. SAGE UI • Session recording and playback July, 2007 GCB SAGE workshop

27. SAGE UI • Can connect to multiple displays at same time July, 2007 GCB SAGE workshop

28. SAGE UI • Chat features as a back channel July, 2007 GCB SAGE workshop

29. SAGE Application Model • Application • Rendering component • User interface component • Layout on the screen controlled by the FreeSpace Manager • Pixels captured by SAIL • SAGE Application Interface Library • Streaming network protocol July, 2007 GCB SAGE workshop

30. SAIL – The SAGE Application Interface Library • The SAGE programming API. • Simple interface allows you to specify a SAGE displayand connect to it • glSwapBuffer like call July, 2007 GCB SAGE workshop

31. OpenGL App SAGEified int main(int argc, char *argv[]) {      // application code       glPixelStorei( GL_UNPACK_ALIGNMENT, 1 );       sailConfig scfg;       scfg.cfgFile = "sage.conf";       scfg.appName = "render";       scfg.rank = 0;       scfg.ip = argv[2];       scfg.resX = 400;       scfg.resY = 400; sageRect renderImageMap;       renderImageMap.left = 0.0;       renderImageMap.right = 1.0;       renderImageMap.bottom = 0.0;       renderImageMap.top = 1.0;       scfg.imageMap = renderImageMap;       scfg.colorDepth = 24;       scfg.pixFmt = TVPIXFMT_888;       scfg.rowOrd = BOTTOM_TO_TOP; sageInf.init(scfg);       cout << "sail initialized " << endl;       glutMainLoop(); } // headers for SAGE #include "sail.h" #include "misc.h" void reshape(int width, int height) {        // blah blah } //display function void redraw(void) {        // draw code        glReadPixels(0, 0, winWidth, winHeight, GL_RGB,        GL_UNSIGNED_BYTE, rgbBuffer); sageInf.swapBuffer((void *)rgbBuffer); glutSwapBuffers(); • } July, 2007 GCB SAGE workshop

32. Write a Native SAGE App scfg.imageMap = renderImageMap; scfg.colorDepth = 24; scfg.pixFmt = TVPIXFMT_888; scfg.rowOrd = TOP_TO_BOTTOM; sageInf.init(scfg); // create zoom adjusted image buffer buffer = new unsigned char[(extent.w / extent.zoom) *                       (extent.h / extent.zoom) * (3)]; // clear the zoom adjusted image buffer memset(buffer,0,(extent.w / extent.zoom)*(extent.h / extent.zoom)* 3); // Main while loop while (!quit) { // Generate pixels and copy into ‘buffer’           // swap buffer        sageInf.swapBuffer(buffer); } • // for SAGE • #include <sail.h> • #include <misc.h> • // SAGE Stuff • int winWidth, winHeight; • sail sageInf; // sail object • // initialize SAGE • sailConfig scfg; • scfg.cfgFile = "sage.conf"; • scfg.appName = “myApp";  • scfg.rank = rank; • scfg.ip = NULL; • scfg.resX = (extent.w / extent.zoom); • scfg.resY = (extent.h / extent.zoom); • sageRect renderImageMap; • renderImageMap.left = 0.0; • renderImageMap.right = 1.0; • renderImageMap.bottom = 0.0; • renderImageMap.top = 1.0; July, 2007 GCB SAGE workshop

33. Building and Installing SAGEFrom Source • Dependencies • Compilation • Configuration July, 2007 GCB SAGE workshop

34. SAGE Dependencies • QUANTA 0.4 (www.evl.uic.edu/cavern/quanta) • Readline (runtime and development packages): from GNU project, http://cnswww.cns.cwru.edu/~chet/readline/rltop.html • SDL libraries for the display side, http://www.libsdl.org • Some test programs need GLUT to compile (atlantis, atlantis-mpi, ...), http://freeglut.sourceforge.net July, 2007 GCB SAGE workshop

35. Compiling SAGE • Modifying makefiles •  Open the appropriate Makefile in the "sage/src" directory and edit the following lines: • Set QUANTA_DIR  to the directory where you built QUANTA • Set the appropriate flag for your shell preference: • If you use bash, set MYFLAGS –DSAGE_BASH • If you use csh, set MYFLAGS –DSAGE_CSH July, 2007 GCB SAGE workshop

36. Compiling SAGE .. Contd. • Execute make install in the "sage/src" directory. • Execute make install in the "sage/src/nwProtocol" directory. • Execute make install in sage/app and sage/app/atlantis. July, 2007 GCB SAGE workshop

37. Edit Configuration Files • Go to sage/bin • Edit fsManager.conf • Edit sage.conf • Edit stdtile.conf (or whatever tile config file specified in sage.conf) July, 2007 GCB SAGE workshop

38. Sample sage.conf displayBinDir SAGE_DIR/bin appBinDir SAGE_DIR/bin appList render { configName local nodeNum 1 Init 100 100 1000 1000 exec 127.0.0.1 render 0 127.0.0.1 nwProtocol tvTcpModule.so } atlantis { configName local nodeNum 1 Init 100 100 1000 1000 exec 127.0.0.1 atlantis 0 127.0.0.1 nwProtocol tvTcpModule.so configName UDP nodeNum 1 Init 100 100 1000 1000 exec 127.0.0.1 atlantis 0 127.0.0.1 nwProtocol tvUdpModule.so } endList July, 2007 GCB SAGE workshop

39. Sample sage.conf … contd. tileConfiguration stdtile.conf receiverBaseSyncPort 12000 receiverBufNum 20 receiverStreamPort 21000 fullScreen 1 sailBaseSyncPort 11000 July, 2007 GCB SAGE workshop

40. Sample stdtile.conf • For 1 node only : TileDisplay Dimensions 1 1 Mullions 0.625 0.625 0.625 0.625 Resolution 1280 1024 PPI 90 Machines 1 DisplayNode Name localhost IP 127.0.0.1 Monitors 1 (0,0) July, 2007 GCB SAGE workshop

41. Stdtile.conf For Multiple Nodes DisplayNode Name yorda5-10 IP 10.0.8.125 Monitors 1 (1,1) DisplayNode Name yorda6-10 IP 10.0.8.126 Monitors 1 (1,0) DisplayNode Name yorda7-10 IP 10.0.8.127 Monitors 1 (2,2) DisplayNode Name yorda8-10 IP 10.0.8.128 Monitors 1 (2,1) DisplayNode Name yorda9-10 IP 10.0.8.129 Monitors 1 (2,0) TileDisplay Dimensions 5 3 Mullions 0.625 0.625 0.625 0.625 Resolution 1280 1024 PPI 90 Machines 15 DisplayNode Name yorda1-10 IP 10.0.8.121 Monitors 1 (0,2) DisplayNode Name yorda2-10 IP 10.0.8.122 Monitors 1 (0,1) DisplayNode Name yorda3-10 IP 10.0.8.123 Monitors 1 (0,0) DisplayNode Name yorda4-10 IP 10.0.8.124 Monitors 1 (1,2) July, 2007 GCB SAGE workshop

42. July, 2007 GCB SAGE workshop