Progress on System Architecture for Extreme Devices

1. Massive Cluster Clusters Gigabit Ethernet Progress on System Architecture for Extreme Devices David Culler http://www.cs.berkeley.edu/~culler U.C. Berkeley Endeavour Retreat 1/20/200

2. Outline • The Very Large • Millenium Cluster-of-cluster resources available to you • System Architecture • The Middle • Kiosks, laptops, and PDAs • The Small • Embedded Servers • Low power wireless • Architecture for Zillions of devices Endeavour Sys. Arch

3. Large Resource Deployment • In place: • Full 64-PIII Linux Cluster w/ Myrinet & ethernet (Gb rdy) • demonstrated on Ninja DDS • several remote 16-PIII clusters • DLIB 4-P + 1/4 TB cluster • 1/2 TB storage server • Gb connection to dept, OC48 to NTON • Rootstock cluster dissemination facility • REXEC econ-based remote execution facility • 100 KVA PDU • Demonstrated • intercampus network, routing, config (as per CNS) • full Gb/s with 2 source, 2 sinks • Deploying • 45 x 4 of main cluster (2/4 GB mem, 18 GB disk, Gb ether) Endeavour Sys. Arch

4. Vineyard Cluster Architecture • dissemination, resource control, remote execution, and communication Applications / Services(ISPACE/Kiosks) Mgmt / Monitoring PBS I/O MPI VEXEC TOOLS REXEC - VIA / GM, GbE - Multicast - NT / Linux (2.2.x) - Stride Scheduler Rootstock Distribution Endeavour Sys. Arch

5. 2. Make the CS “graft” - specify IP address - pckg removes - dchp, dns, nis,... sanity check and build - resolv.conf, /etc/hosts, ... constructs cluster build (lease) download CS build floppy 3. CS power-on build - xfer and localize DT - add local admin scripts - node build floppy Cluster leased builds K 4. Node power-on build - local stock from CS Dissemination: Rootstock Cluster System Distribution Center cluster stock - build - os - drvrs - mill SW - os mods 1. Cluster Stock - Rootstock build pages - Full Current Linux - all fixes and pckgs - SSL, SSH - Cluster Drivers - Cluster System Layers - rexec, mpe, pbs - Optional SW ($) - Cluster Kernal Mods cs IP network CAN ... 5. Cluster Update button (future) - 2nd dialtone, CF engine, rolling update Endeavour Sys. Arch

6. REXEC / VEXEC • Resource Management, Autoconfig, Mechanism/Policy, Enforcement Node A Node B Node C Node D rexecd rexecd rexecd rexecd Cluster IP Multicast Channel vexecd(Policy A) vexecd(Policy B) “Nodes AB” run indexer on Nodes AB at 3 credits/min minimum $ rexec %rexec –n 2 –r 3 indexer Endeavour Sys. Arch

7. “Intelligent” Middle • Deployed many laptops with 802-11, 3 base stations, many PDAs with IR • Solved the PDA to IR-dongle and PDA to Annex serial port (J. Hill) • Deployed two kiosks: touch-LCD, IR-ppp • act as server for managment • Demonstrated key aspects of the service infrastructure • eg: get device applet from service point • eSticky notes appln • motivated xcoding-security infrastucture • => Need to harvest and extend Endeavour Sys. Arch

8. Small: Embedded Servers • Tested commercial products - promising • axis camera server • SOHO NAT, DHCP, Firewall server • plumbing = ethernet • Identified platform for building embedded servers • DIMM PC + Linux + … • 486 + 16 MB RAM + 16 MB Flash Disk Endeavour Sys. Arch

9. Low-power Wireless • Tested available options • RF monolithics (used in Smart Rocks) • “virtual wire” is brain-dead, but good tranceiver • RadioMetrics (used in ISI RF Tags) • simple, primitive packet controller, no pwr down • World Wireless • nice MAC, but only infrastructure mode • Selected RF monolithics • working with BSAC and ISI on building-block • new packet controller + MAC Endeavour Sys. Arch

10. Zillions of Little Devices • Connected device as client well-established • distiller in the infrastructure spoonfeeds client • powerful services in power-limited devices! • How to get the illusion of continuous connectivity? • What about sensors-based devices? • they should behave as servers • eg: camera server • How to scale tiny server to need? • How to get illusion of continuous connectivity? • use the infrastructure • First a demonstration: note server in a PDA Endeavour Sys. Arch

11. Assumptions • Computation and storage in the infrastructure is plentiful • Wired bandwidth is pervasive and essentially free • [ Multicast is widely accessible] => every device has a representative proxy in the infrastructure Endeavour Sys. Arch

12. Cautionary enabling thought • TCP protocol (acks, fsm, etc.) terminates in the kernel. • System specific protocol exists between kernel TCP agent and applications. • SSI clusters have another layer of network! • Think of device as an application (not big laptop) • Where is its kernel agent? app app app app TCP IP Endeavour Sys. Arch

13. Scalable, Available Ninja Base persistent named representative AP AP AP Dev MC The Internet Device Access Architecture • infra proxy provides name, state, queuing, etc. • extend toward AP as optimization low power local device link Clients Physical Device Services Endeavour Sys. Arch

14. Laptop w/ Ninja iSpace persistent named representative AP AP AP TCP The Internet Demo Mapping Laptop Browser serial PDA laptop w/ pppd BayStacker 802.11 Wireless Endeavour Sys. Arch

15. Key piece to build • Low power controller with 2 stream devices • X = sensor + actuator for devices • X = host interface for AP and Embedded server Application Tiny Kernel Tiny flow drivers RF tcvr X host a s s a svr s a Endeavour Sys. Arch