1 / 34

Preliminaries

CSE 291 course covers WebOS & Legion, supporting distributed, scalable services. Topics include resource discovery, process control, security, apps like Internet Chat & Rent-A-Server.

clintonn
Download Presentation

Preliminaries

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Preliminaries • Mailing list • Send me your email address if you haven’t already • Even if you are just going to sit in • List of students • Finding partners • Office hour? • Questions about papers, doing the presentations? • Evals • Great! Put them online? Anonymized? • Slides • Online also? CSE 291: WebOS & Legion

  2. WebOSOperating System Services for Wide Area Applications Amin Vahdat, Eshwar Belani, Paul Eastham, Chad Yoshikawa, Thomas Anderson, David Culler, and Michael Dahlin

  3. Vision • Framework for supporting wide-area, distributed, highly available, incrementally scalable, reconfigurable services and applications • Components • Resource discovery – which server do I use? • Persistent storage – wide-area file system • Remote process execution – fork on a remote machine • Authentication and security – provide trust CSE 291: WebOS & Legion

  4. Example App: Rent-A-Server • Set of generic servers distributed across Internet • Transparent, automatic replication of HTTP service (CNN) • Spawn new servers when demand is high • Do not have to statically allocate resources to match peak demand • Advantages of using WebOS to build Rent-A-Server • Better availability – transparently mask server failures • Better cost-performance – do not have to provide peak capacity for peak • Better burst behavior (handle peak demand) • Efficient use of a collection of resources CSE 291: WebOS & Legion

  5. Resource Discovery • How to choose the best server to send request? • HTTP redirect requires two round-trips • DNS is round-robin (performance insensitive) • WebOS uses Smart Clients • Applet chooses server based upon distance and server load • Servers periodically piggy-back load info back to applet at client to update load info • Load info grows stale, defaults to random CSE 291: WebOS & Legion

  6. Smart Clients • Algorithm • User requests a service (e.g., “service://chat”) • Meta-applet in browser handles “service:” URLs, contacts a “service:” server (e.g., by using AltaVista) • “Service” server returns a list of available hosts supporting the service (e.g., chat servers) + info (location, load) • Meta-applet chooses a host, downloads Smart Client applet • Smart Client applet runs in client browser • Smart Client chooses chat server based upon location, load • Smart Client updates its info as user continues to make requests to server CSE 291: WebOS & Legion

  7. Smart Proxies • Similar to Smart Clients, but run inside of Proxies • Aggregate client requests to servers, much better picture of server load • Get around Java applet restrictions (making network connections to arbitrary sites) CSE 291: WebOS & Legion

  8. Wide Area File System • WebFS • Persistent files named using URLs • Location dependent • Can use standard Web servers (backwards compatible) • Authenticated access when talking to other WebFS’s • Consistent • Application controlled; e.g., invalidation as in AFS • Caches data • Services using WebFS get caching for free • Implemented to the Vnode interface • Looks like another file system to applications CSE 291: WebOS & Legion

  9. Process Control • Fork processes on remote servers • As easily as forking on own machine • Resource Manager controls WebOS jobs • Does admission control • Authenticates the source of the job • Jobs run in restricted virtual machine (Janus) • Privileged system calls are intercepted • Configuration files determine whether calls are allowed to continue (c.f. Java security manager) • Resources limited using Unix resource controls CSE 291: WebOS & Legion

  10. Security and Authentication • Based on CRISIS [Belani98] • Not going to go into details, we’re going to read it later on • Entities • Principles – sources of trust • Objects – resources • Reference monitors – grant requests • Statements of trust represented by certifications • Authenticate principles • Validate privileges (access controls to resources) • Transfer privileges (very useful) CSE 291: WebOS & Legion

  11. Applications • Internet Chat • Each room is implemented as a consistent, shared file • Writes are appended to the file, updates multicast to clients • Remote Compute Engine • WebOS server on supercomputers • Applications are spawned inside WebOS VMs • Cooperative Web Caching • Smart Clients to choose best cache, WebFS to distribute data • Internet Weather • Central server maintains Internet performance database • Access server using an applet; use also contributes data CSE 291: WebOS & Legion

  12. Rent-A-Server Redux • WebOS killer app • Smart Clients choose best server to use • Process control allows dynamic spawning of servers • WebFS allows replication of data among servers • CRISIS allows trust among servers • Load balancer spawns/reaps servers depending upon load • How did you like this service? • Billing… CSE 291: WebOS & Legion

  13. Discussion • Questions about the paper, system? • Would you want to use WebOS to build wide-area services? CSE 291: WebOS & Legion

  14. What did I like? • Services motivated by application needs • Application requirements well understood • Design to meet those requirements • Leverage existing services • URLs for names, Java to implement client functionality, SSL for secure connections • Don’t try to reinvent the world • Implemented • WebOS services implemented • 5 applications implemented • Good systems paper CSE 291: WebOS & Legion

  15. The Core LegionObject ModelMike Lewis and Andrew Grimshaw

  16. Vision • Provide a single virtual machine model across all hosts connected by the Internet • Location and distribution are transparent to users and applications • “System boundaries, data location, faults are invisible” • Applications are run on Legion, not on a host • Contrast with systems where only the services are well defined • SMTP, MPI, HTTP, DCE-RPC, etc. • WebOS CSE 291: WebOS & Legion

  17. Target Applications • Compute intensive • Marshal CPU resources of many machines • Workstations, supercomputers (toasters?) • Highly parallel • Coarse grained • Has to be if wide-area • Applications? • Funny, not specified… • Large scale scientific apps – supercomputing++ • Highly parallel apps w/ minimal communiction • Cracking crypto keys CSE 291: WebOS & Legion

  18. Objectives (The Kitchen Sink) • Site autonomy • Extensible core (*) • Scalable architecture • Seamless computing environment • Highly parallel • Global name space (*) • Security for users and resources • Heterogeneous – many platforms • Multiple languages, interoperability • Fault tolerant CSE 291: WebOS & Legion

  19. Legion Objects • Interface specified using an IDL (CORBA, Mentat) • Implementation can be done using any language • Legion compiler wraps it up into a Legion Object • Coarse grained • Not fine-grained like C++, Java, Smalltalk • Execute in disjoint address spaces • Think of an object as being a process • For an app, probably one object per host CSE 291: WebOS & Legion

  20. Object Communication • Objects communicate via method calls • Non-blocking (not sure how to synchronize?) • Example methods mostly deal with control • Locating, creating, deleting, etc., objects • Not sure if methods are intended for data transport • Use Legion comm to locate and establish links with other Legion Objects • Once all objects are connected, use, e.g., MPI to shuttle data around? CSE 291: WebOS & Legion

  21. Core Objects • Core classes in Legion implement essential services • LegionObject • LegionClass • LegionHost • LegionVault • LegionBindingAgent CSE 291: WebOS & Legion

  22. LegionObject • Root of type and instance hierarchy • All objects derive from LegionObject • Classes are objects, so they derive, too • Interface for essential methods • Access – may_I() • Persistence – save_state(), restore_state() CSE 291: WebOS & Legion

  23. LegionClass • Responsible for managing object instances • create(), delete() • Subclasses • derive(), inherit_from() • Locating objects • get_binding() • Each class object can construct the address of all of its instances (including subclasses)…more later CSE 291: WebOS & Legion

  24. LegionHost • Represents a host • Subclasses represent different kinds of hosts • UnixHost, UnixSMMP, CM-5, etc. • All objects are created on a LegionHost • They must live somewhere and have backing store allocated for them • Used for resource control • LegionHost determines whether an object can execute on it • “Site autonomy” CSE 291: WebOS & Legion

  25. LegionVault • Interface to persistent storage • Objects have two states – Active, Inert • Inert objects stored as Object Persistent Representations (OPRs) • Objects save and restore state from their OPRs • LegionVaults store OPRs • E.g., on disk • Objects are migrated by storing to OPR, migrating OPR from one Vault to another, and restoring • Very coarse-grained migration CSE 291: WebOS & Legion

  26. LegionBindingAgent • Binds object identifiers to addresses • Think of it as a DNS server of sorts • …or something that maps URIs to URLs • Need to talk about LOIDs and Object Addresses… CSE 291: WebOS & Legion

  27. Legion Object IDs (LOIDs) • Universal, unique name of an object • Does not specify location • Format – defines the fields in the LOID • Class ID – class of object (from LegionObject) • Instance # – instance identifier (from LegionClass) • Public Key – used for security and authentication • Additional Fields – depends on format; extensible Format Class ID Instance # Public Key (Additional Fields) CSE 291: WebOS & Legion

  28. Object Addresses • Programs name objects using LOIDs, but you can’t find an object • Object Addresses (OA) define the location of an object • To use an object, its LOID must be mapped to an OA • Similar in spirit to binding process used in RPC • No session semantics, though • OAs contain Object Address Elements • List of addresses, can be used to provide multicast • Practically, an OA is an IP address and port # CSE 291: WebOS & Legion

  29. Binding • LegionBindingAgents (LBAs) provide the service that binds LOIDs to OAs (get_binding ()) • …with the help of LegionClass • A wants to get a binding for B (has class C) • A queries an LBA with LOID(B) • Recall classes know the OAs of their instances • LBA contacts class C to find OA for B • LBA can find C by contacting LegionClass (since C is an instance of LegionClass) • Caching (in object A, LBA, and class C) makes it scalable CSE 291: WebOS & Legion

  30. Discussion • Questions about the paper, system? • Move into discussion… • Single virtual machine model • Universal names CSE 291: WebOS & Legion

  31. Single Virtual Machine • Is a single virtual machine the best model? • The VM model provides transparency • Transparency is extremely useful…but also seductive • May not want transparency for the wide-area • RPC aims to be transparent • Is RPC successful? Butler Lampson says NO. • Can’t make faults transparent • Can’t make performance transparent • Well-defined service interfaces, protocols enough • E.g., HTTP • Difference between distributed systems, parallel systems? CSE 291: WebOS & Legion

  32. Universal Names • In favor of a universal naming mechanism • DNS is very successful • Orthogonal to single virtual machine model • Can be its own service, not tied to a particular system architecture (again, DNS) • One problem universal names should solve is object mobility • Legion LOIDs do not seem suited for this • What do you think? • Universal names a good idea? • LOIDs and OAs a good scheme? CSE 291: WebOS & Legion

  33. What did I like? • Like the goal of unique universal names • A service we should have • Funny way to justify system architecture, though • “Our system is more ambitious than other systems, so it must be better” • Btw, what are the apps again? CSE 291: WebOS & Legion

  34. WebOS/Legion • What do you think about WebOS vs. Legion? • Programming model vs. services? CSE 291: WebOS & Legion

More Related